zlib的，放气：多少内存来分配？ [英] zlib, deflate: How much memory to allocate?

问题描述

我使用 zlib的以COM preSS文本数据流。文本数据进来块，每个块，放气（）被调用，以刷新设置为 Z_NO_FLUSH 。一旦所有的数据块被检索到，放气（）被称为平齐设置为 Z_FINISH 。

I am using zlib to compress a stream of text data. The text data comes in chunks, and for each chunk, deflate() is called, with flush set to Z_NO_FLUSH. Once all chunks have been retrieved, deflate() is called with flush set to Z_FINISH.

当然，放气（）不产生COM $ P $每次调用pssed输出。它在内部积累的数据达到了很高的COM pression率。这很好！每次放气（）产生COM pressed输出，该输出被添加到数据库字段 - 一个缓慢的过程。

Naturally, deflate() doesn't produce compressed output on each call. It internally accumulates data to achieve a high compression rate. And that's fine! Every time deflate() produces compressed output, that output is appended to a database field - a slow process.

然而，一旦放气（）产生COM pressed数据，该数据可能不适合所提供的输出缓冲器， deflate_out 。因此，多次打电话给放气（）是必需的。这就是我想避免什么：

However, once deflate() produces compressed data, that data may not fit into the provided output buffer, deflate_out. Therefore several calls to deflate() are required. And that is what I want to avoid:

有没有一种方法，使 deflate_out 总是足够大，以便放气（）可以将所有的COM $ p $它pssed数据，每一个时候，它决定产生输出？

Is there a way to make deflate_out always large enough so that deflate() can store all the compressed data in it, every times it decides to produce output?

注：

• 在uncom pressed数据的总大小的不的事先知道。如上述提到的，uncom pressed数据来自在块和COM pressed数据被追加到数据库字段，也以块

• The total size of the uncompressed data is not known beforehand. As mentioned above, the uncompressed data comes in chunks, and the compressed data is appended to a database field, also in chunks.

在包含文件 zconf.h 我发现下面的评论。那是什么也许我在找什么？即为（1 LT;＆LT;（windowBits + 2））+（1＆LT;＆LT;（memLevel + 9））在COM pressed的最大字节数该数据放气（）可能会产生？

In the include file zconf.h I have found the following comment. Is that perhaps what I am looking for? I.e. is (1 << (windowBits+2)) + (1 << (memLevel+9)) the maximum size in bytes of compressed data that deflate() may produce?

/* The memory requirements for deflate are (in bytes):
            (1 << (windowBits+2)) +  (1 << (memLevel+9))
 that is: 128K for windowBits=15  +  128K for memLevel = 8  (default values)
 plus a few kilobytes for small objects. For example, if you want to reduce
 the default memory requirements from 256K to 128K, compile with
     make CFLAGS="-O -DMAX_WBITS=14 -DMAX_MEM_LEVEL=7"
 Of course this will generally degrade compression (there's no free lunch).

   The memory requirements for inflate are (in bytes) 1 << windowBits
 that is, 32K for windowBits=15 (default value) plus a few kilobytes
 for small objects.
*/

推荐答案

deflateBound（）是有帮助的前提是你做的所有的COM pression的一步，如果你强迫紧缩对COM preSS目前市面上所有给它的输入数据，并发出COM $ p $为所有输入数据pssed。你会做到这一点与冲洗参数如Z_BLOCK，Z_PARTIAL_FLUSH等。

deflateBound() is helpful only if you do all of the compression in a single step, or if you force deflate to compress all of the input data currently available to it and emit compressed data for all of that input. You would do that with a flush parameter such as Z_BLOCK, Z_PARTIAL_FLUSH, etc.

如果你想使用Z_NO_FLUSH，那么它变得更加困难，以及效率低下的尝试predict输出放气用量最大的（）可能会发出的下一个电话。你不知道有多大的投入在COM pressed数据的最后一阵被发出的时间被消耗，因此，您需要承担几乎没有它，与缓冲区大小成长不必要。然而，你试图估算最大的产出，你会做很多不必要的mallocs或reallocs的毫无理由，这是低效的。

If you want to use Z_NO_FLUSH, then it becomes far more difficult as well as inefficient to attempt to predict the largest amount of output deflate() might emit on the next call. You don't know how much of the input was consumed at the time the last burst of compressed data was emitted, so you need to assume almost none of it, with the buffer size growing unnecessarily. However you attempt to estimate the maximum output, you will be doing a lot of unnecessary mallocs or reallocs for no good reason, which is inefficient.

有没有点避免调用放气（）以获得更多的输出。如果你只是在放气循环（），直到它有你没有更多的输出，那么你可以使用一个固定的输出缓冲malloced一次。那就是如何放气（）和膨胀（）接口被设计为可以使用。你可以看一下 http://zlib.net/zlib_how.html 获取如何使用界面上的证据充分的例子。

There is no point to avoid calling deflate() for more output. If you simply loop on deflate() until it has no more output for you, then you can use a fixed output buffer malloced once. That is how the deflate() and inflate() interface was designed to be used. You can look at http://zlib.net/zlib_how.html for a well-documented example of how to use the interface.

顺便说一句，没有在zlib的（1.2.6）的最新版本deflatePending（）函数，可以让你知道多少输出放气（）已等待交付。

By the way, there is a deflatePending() function in the latest version of zlib (1.2.6) that lets you know how much output deflate() has waiting to deliver.

这篇关于zlib的，放气：多少内存来分配？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！