更快的MD5替代品？ [英] Faster MD5 alternative?

问题描述

我正在研究一个程序，该程序可以搜索整个驱动​​器中的给定文件。目前，我计算已知文件的MD5散列值，然后递归扫描所有文件，查找匹配结果。

唯一的问题是MD5的速度很慢大文件。有没有更快的选择，我可以使用，同时保留误报的可能性很小？

所有代码都在C＃中。

谢谢。

更新

即使MD5可以非常快，磁盘I / O应该是限制因素。这使我相信我的代码可能不是最优的。这种方法有什么问题吗？

  MD5 md5 = MD5.Create（）; 
 StringBuilder sb = new StringBuilder（）; 
尝试
 {
使用（FileStream fs = File.Open（fileName，FileMode.Open，FileAccess.Read））
 {
 foreach（byte b in md5。 ComputeHash（fs））
 sb.Append（b.ToString（X2））; 
} 
 return sb.ToString（）; 
} 
 catch（例外）
 {
 return; 
 
  

解决方案

我希望你检查只有在文件大小已经匹配的情况下才能匹配MD5。

另一个优化是对第一个1K（或其他一些任意的，但相当小的数字）进行快速校验，并确保在处理整个文件之前进行匹配。

当然，所有这些都假设您只是为特定文件寻找匹配/不匹配的决定。 / p>

I'm working on a program that searches entire drives for a given file. At the moment, I calculate an MD5 hash for the known file and then scan all files recursively, looking for a match.

The only problem is that MD5 is painfully slow on large files. Is there a faster alternative that I can use while retaining a very small probablity of false positives?

All code is in C#.

Thank you.

Update

I've read that even MD5 can be pretty quick and that disk I/O should be the limiting factor. That leads me to believe that my code might not be optimal. Are there any problems with this approach?

        MD5 md5 = MD5.Create();
        StringBuilder sb = new StringBuilder();
        try
        {
            using (FileStream fs = File.Open(fileName, FileMode.Open, FileAccess.Read))
            {
                foreach (byte b in md5.ComputeHash(fs))
                    sb.Append(b.ToString("X2"));
            }
            return sb.ToString();
        }
        catch (Exception)
        {
            return "";
        }

解决方案

I hope you're checking for an MD5 match only if the file size already matches.

Another optimization is to do a quick checksum of the first 1K (or some other arbitrary, but reasonably small number) and make sure those match before working the whole file.

Of course, all this assumes that you're just looking for a match/nomatch decision for a particular file.

这篇关于更快的MD5替代品？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！