如何缓存部分crc32校验和，这样我就不需要多次计算它了? [英] How to cache partial crc32 checksums so I don't need to calculate it multiple times?

问题描述

在我最近编写的一些代码中，我有这种模式:

从Zlib导入

  crc32new_data = get_some_input()crc32List ['stream1'] = crc32(new_data，crc32List ['stream1'])&0xffffffffLcrc32List ['stream2'] = crc32(new_data，crc32List ['stream2'])&0xffffffffL...crc32List ['streamN'] = crc32(new_data，crc32List ['streamN'])&0xffffffffL 

在我看来，那里有一些多余的计算，如果我能找到一个名为 magic(x，y)的函数来进行以下缓存，那我会很高兴的:

  crc32List ['cached'] = crc32(new_data，0)&0xffffffffLcrc32List ['stream1'] =魔术(crc32List ['cached']，crc32List ['stream1'])crc32List ['stream2'] =魔术(crc32List ['cached']，crc32List ['stream2'])...crc32List ['streamN'] =魔术(crc32List ['cached']，crc32List ['streamN']) 

' magic(x，y)'使用缓存的'x'crc32值，并返回与' crc32(new_data，y)&0xffffffffL '

当然，' stream [0:N] '以不同的值开始，并在任何时间点以不同的值结束，但是对于以下情况，几乎总是执行crc32计算(90％+):全部为N，并始终带有' new_data '

解决方案

您未提供有关标记使用哪种语言的提示，并且我对 crc32()的版本不熟悉

code>具有如图所示参数的函数.无论如何，我认为您正在寻找的是 zlib crc32_combine()函数>.

zlib(在C语言中)实际 crc32()函数的参数为​​ crc32(crc，buf，len)，其中 crc 是起始CRC-32值， buf 是指向要计算其CRC-32的字节的指针，而 len 是字节数.该函数返回更新后的CRC-32值.

鉴于:

  crc32(crc32(0，seq1，len1)，seq2，len2)== crc32_combine(crc32(0，seq1，len1)，crc32(0，seq2，len2)，len2) 

请注意， crc32_combine()需要知道第二个序列的长度以及两个CRC-32值才能将它们组合在一起.

In some code that I wrote recently, I had this pattern:

from zlib import crc32

new_data = get_some_input()

crc32List['stream1'] = crc32(new_data, crc32List['stream1']) & 0xffffffffL
crc32List['stream2'] = crc32(new_data, crc32List['stream2']) & 0xffffffffL
...
crc32List['streamN'] = crc32(new_data, crc32List['streamN']) & 0xffffffffL

It seems to me, that there's a bit of redundant computation going on there and if I can find a function called magic(x, y) that does the following caching, I would be happy:

crc32List['cached'] = crc32(new_data, 0) & 0xffffffffL

crc32List['stream1'] = magic(crc32List['cached'], crc32List['stream1'])
crc32List['stream2'] = magic(crc32List['cached'], crc32List['stream2'])
...
crc32List['streamN'] = magic(crc32List['cached'], crc32List['streamN'])

'magic(x, y)' uses the cached 'x' crc32 value and returns the same result as 'crc32(new_data, y) & 0xffffffffL'

Of course 'stream[0:N]' begin with different values and end up with different values at any point in time, but the crc32 computation is almost always executed (90%+) for all N and always with 'new_data'

解决方案

You did not provide a hint for what language this is with a tag, and I am not familiar with a version of a crc32() function that has arguments as shown. In any case, what I think you're looking for is the crc32_combine() function of zlib.

The arguments to the actual crc32() function in zlib (in C) are crc32(crc, buf, len), where crc is the starting CRC-32 value, buf is a pointer to the bytes to compute the CRC-32 of, and len is the number of bytes. The function returns the updated CRC-32 value.

Given that:

crc32(crc32(0, seq1, len1), seq2, len2) == crc32_combine(crc32(0, seq1, len1), crc32(0, seq2, len2), len2)

Note that crc32_combine() needs to know the length of the second sequence as well as the two CRC-32 values in order to combine them.

这篇关于如何缓存部分crc32校验和，这样我就不需要多次计算它了?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！