GZipStream-写不写所有压缩的数据，即使冲洗? [英] GZipStream - write not writing all compressed data even with flush?

问题描述

针对.Net 3.5的gzipstream出现了令人讨厌的问题.这是我第一次使用gzipstream，但是我已经根据许多教程建模，包括此处，我仍然被困住.

I've got a pesky problem with gzipstream targeting .Net 3.5. This is my first time working with gzipstream, however I have modeled after a number of tutorials including here and I'm still stuck.

我的应用程序将数据表序列化为xml，然后插入数据库中，将压缩后的数据存储到varbinary(max)字段中，并将未压缩的缓冲区的原始长度存储在其中.然后，在需要时检索此数据并解压缩并重新创建数据表.解压缩似乎失败了.

My app serializes a datatable to xml and inserts into a database, storing the compressed data into a varbinary(max) field as well as the original length of the uncompressed buffer. Then, when I need it, I retrieve this data and decompress it and recreates the datatable. The decompress is what seems to fail.

可悲的是，按照建议将GetBuffer更改为ToArray之后，我的问题仍然存在.代码已在下面更新

Sadly after changing the GetBuffer to ToArray as suggested, my issue remains. Code Updated below

压缩代码:

DataTable dt = new DataTable("MyUnit");
//do stuff with dt
//okay...  now compress the table
using (MemoryStream xmlstream = new MemoryStream())
{
    //instead of stream, use xmlwriter?
    System.Xml.XmlWriterSettings settings = new System.Xml.XmlWriterSettings();
    settings.Encoding = Encoding.GetEncoding(1252);
    settings.Indent = false;
    System.Xml.XmlWriter writer = System.Xml.XmlWriter.Create(xmlstream, settings);
    try
    {
        dt.WriteXml(writer);
        writer.Flush();
    }
    catch (ArgumentException)
    {
        //likely an encoding issue...  okay, base64 encode it
        var base64 = Convert.ToBase64String(xmlstream.ToArray());
        xmlstream.Write(Encoding.GetEncoding(1252).GetBytes(base64), 0, Encoding.GetEncoding(1252).GetBytes(base64).Length);
    }

    using (MemoryStream zipstream = new MemoryStream())
    {
        GZipStream zip = new GZipStream(zipstream, CompressionMode.Compress);
        log.DebugFormat("Compressing commands...");
        zip.Write(xmlstream.GetBuffer(), 0, xmlstream.ToArray().Length);
        zip.Flush();
        float ratio = (float)zipstream.ToArray().Length / (float)xmlstream.ToArray().Length;
        log.InfoFormat("Resulting compressed size is {0:P2} of original", ratio);

        using (SqlCommand cmd = new SqlCommand())
        {
            cmd.CommandText = "INSERT INTO tinydup (lastid, command, compressedlength) VALUES (@lastid,@compressed,@length)";
            cmd.Connection = db;
            cmd.Parameters.Add("@lastid", SqlDbType.Int).Value = lastid;
            cmd.Parameters.Add("@compressed", SqlDbType.VarBinary).Value = zipstream.ToArray();
            cmd.Parameters.Add("@length", SqlDbType.Int).Value = xmlstream.ToArray().Length;
            cmd.ExecuteNonQuery();

        }
    }

解压缩代码:

/* This is an encapsulation of what I get from the database
 public class DupUnit{
    public uint lastid;
    public uint complength;
    public byte[] compressed;
}*/
  //I have already retrieved my list of work to do from the database in a List<Dupunit> dupunits
foreach (DupUnit unit in dupunits)
{
    DataSet ds = new DataSet();
    //DataTable dt = new DataTable();
    //uncompress and extract to original datatable
    try
    {
        using (MemoryStream zipstream = new MemoryStream(unit.compressed))
        {
            GZipStream zip = new GZipStream(zipstream, CompressionMode.Decompress);
            byte[] xmlbits = new byte[unit.complength];
            //WHY ARE YOU ALWAYS 0!!!!!!!!
            int bytesdecompressed = zip.Read(xmlbits, 0, unit.compressed.Length);
            MemoryStream xmlstream = new MemoryStream(xmlbits);
            log.DebugFormat("Uncompressed XML against {0} is: {1}", m_source.DSN, Encoding.GetEncoding(1252).GetString(xmlstream.ToArray()));
            try{
               ds.ReadXml(xmlstream);
            }catch(Exception)
            {
                //it may have been base64 encoded...  decode first.
               ds.ReadXml(Encoding.GetEncoding(1254).GetString(
                 Convert.FromBase64String(
                 Encoding.GetEncoding(1254).GetString(xmlstream.ToArray())))
                 );
            }
            xmlstream.Dispose();
        }
    }
    catch (Exception e)
    {
        log.Error(e);
        Thread.Sleep(1000);//sleep a sec!
        continue;
    }

请注意上面的注释... bytescompressed始终为0.有什么想法吗?我做错了吗?

Note the comment above... bytesdecompressed is always 0. Any ideas? Am I doing it wrong?

这很奇怪.我在解压缩例程中添加了以下调试代码:

So this is weird. I added the following debug code to the decompression routine:

   GZipStream zip = new GZipStream(zipstream, CompressionMode.Decompress);
   byte[] xmlbits = new byte[unit.complength];
   int offset = 0;
   while (zip.CanRead && offset < xmlbits.Length)
   {
       while (zip.Read(xmlbits, offset, 1) == 0) ;
       offset++;
   }

在调试时，有时该循环会完成，但有时会挂起.当我停止调试时，它将位于1616中的1600字节处.我会继续，但它根本不会移动.

When debugging, sometimes that loop would complete, but other times it would hang. When I'd stop the debugging, it would be at byte 1600 out of 1616. I'd continue, but it wouldn't move at all.

.该错误似乎在压缩代码中.无论出于何种原因，它都不会保存所有数据.当我尝试使用第三方gzip机制解压缩数据时，我只会得到原始数据的一部分.

EDIT 3: The bug appears to be in the compress code. For whatever reason, it is not saving all of the data. When I try to decompress the data using a third party gzip mechanism, I only get part of the original data.

我会开始赏金，但到目前为止我真的没有什么好名望的:-(

I'd start a bounty, but I really don't have much reputation to give as of now :-(

推荐答案

终于找到了答案.压缩的数据不完整，因为GZipStream.Flush()绝对不会做任何事情来确保所有数据都在缓冲区之外-您需要将GZipStream.Close()用作

Finally found the answer. The compressed data wasn't complete because GZipStream.Flush() does absolutely nothing to ensure that all of the data is out of the buffer - you need to use GZipStream.Close() as pointed out here. Of course, if you get a bad compress, it all goes downhill - if you try to decompress it, you will always get 0 returned from the Read().

这篇关于GZipStream-写不写所有压缩的数据，即使冲洗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！