是否有可能做的CRC-32计算的分裂？ [英] Is it possible to do CRC-32 calculation in splits?
问题描述
我用这个平凡函数计算给定文件的CRC校验：
I use this trivial function to calculate the CRC checksum of a given file:
long i, j = 0;
int k = 0;
uint crc = 0xFFFFFFFF;
FileInfo file_info = new FileInfo(file);
byte[] file_buffer = new byte[32768];
FileStream file_stream = new FileStream(@file, FileMode.Open);
while ((i = file_stream.Read(file_buffer, 0, file_buffer.Count())) > 0)
{
for (j = 0; j < i; j++)
{
uint before = crc;
k = (int)((crc ^ file_buffer[j]) & 0x000000FFL);
uint after = (uint)((crc >> 8) & 0x00FFFFFFL) ^ crc32_table[k];
crc = after;
uint test = (uint)((crc << 8) & 0x00FFFFFFL) ^ crc32_table[k];
MessageBox.Show((~crc).ToString("X"));
}
}
file_stream.Close();
return ~crc;
我的问题是这样的：说我有一个大的文件，都说100MB。有第一50MB的CRC-32计算，最后50MB和100MB的文件的CRC-32计算之间的任何联系？
My question is this: Say I have a large file, of say 100MB. Is there any link between a CRC-32 calculation of the first 50MB and the last 50MB and the CRC-32 calculation of the 100MB file?
我之所以这么问中，我有一些非常大的文件（〜10GB给予或采取），它需要一些时间来生成，但是当他们正在生成，大部分部件保持静止，但是，部分的中间（已知点）和右在开始（标题，也称为部分/长度）。计算10GB的文件的CRC-32校验需要相当长一段时间，所以我想知道是否有任何办法做到这一点的块？
The reason I'm asking, is I have some very large files (~10GB give or take) which take some time to generate, but while they're being generated, most parts remain static, however, parts in the middle (known point) and right at the start (header, also known part/length). Calculating a CRC-32 checksum of a 10GB file takes quite some time, so I was wondering if there was any way to do it in chunks?
推荐答案
它的是的确实有可能并行的CRC-32计算，但细节混乱，我需要花费大约每天拿出的代码。
It is indeed possible to parallelize the CRC-32 computation, but the details are messy and I'd need to spend about a day to come up with the code.
让我们来看看基本的CRC算法，那里没有否定，也没有位反转。
Let's look at the basic CRC algorithm, where there is no negation and no bit reversal.
有关字节的字符串要计算的CRC，让我们称它为消息。其基本思想是，你把消息如 GF（2）多项式和你计算其剩余模CRC多项式。
For the string of bytes that you want to compute the CRC of, let's call it the message. The basic idea is that you treat the message as a polynomial in GF(2), and you compute its remainder modulo the CRC polynomial.
基本CRC算法是加/线性的。如果有两个消息和相同长度的b，则CRC（一个XOR B）= CRC（一）XOR CRC（b）中。
The basic CRC algorithm is additive/linear. If you have two messages a and b of the same length, then CRC(a XOR b) = CRC(a) XOR CRC(b).
此外，如果。垫右侧有n个零的消息，新的CRC将是老CRC次X ^ N模CRC多项式
Furthermore, if you pad the message on the right side with n zeros, the new CRC will be the old CRC times x^n mod the CRC polynomial.
&NBSP;
与所有的说，为您解决问题的唯一方法是真正理解CRC算法背后的数学和编写自己的自定义代码。下面是CRC的很长但很完整的解释： http://www.ross.net/crc /download/crc_v3.txt
With all that said, the only way to solve your problem is to really understand the mathematics behind the CRC algorithm and write your own custom code. Here's a long but very complete explanation of CRCs: http://www.ross.net/crc/download/crc_v3.txt
这篇关于是否有可能做的CRC-32计算的分裂？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
