如何使用GZipStream将gz文件解压缩到内存中? [英] How do you unzip a gz file in memory using GZipStream?

问题描述

我可能在这里显然很愚蠢.请指出！

I'm probably doing something obviously stupid here. Please point it out!

我有一些C#代码正在从SFTP中拉出一堆.gz文件(使用

I have some C# code that is pulling down a bunch of .gz files from SFTP (using the SSH.NET Nuget package - works great!). Each gz contains only a single .CSV file inside of them. I want to keep these files in memory without hitting disk (yes, I know, server memory management concerns exist - that's fine as these files are fairly small), decompress them in memory to extract the CSV file inside, and then return a collection of CSV files in a custom DTO (FtpFile).

我的问题是，尽管我的SFTP连接中的MemoryStream中包含数据，但似乎从来没有在我的GZipStream中填充它，或者从GZipStream到我的输出MemoryStream的副本失败了.我已经尝试过使用自己的缓冲区对Read进行更传统的循环，但其结果与此代码相同.

My problem is that while my MemoryStream from the SFTP connection has data in it, either it doesn't ever seem to be populated in my GZipStream or the copy from the GZipStream to my output MemoryStream is failing. I have tried with the more traditional looping over Read with my own buffer but it had the same results as this code.

除了连接详细信息(连接成功，因此无后顾之忧)之外，这是我的所有代码:

Aside from connection details (it connects successfully, so no worries there), here's all of my code:

逻辑:

    public static List<FtpFile> Foo()
    {
        var connectionInfo = new ConnectionInfo("example.com",
            "username",
            new PasswordAuthenticationMethod("username", "password"));
        using (var client = new SftpClient(connectionInfo))
        {
            client.Connect();

            var searchResults = client.ListDirectory("/testdir")
                .Where(obj => obj.IsRegularFile
                              && obj.Name.ToLowerInvariant().StartsWith("test_")
                              && obj.Name.ToLowerInvariant().EndsWith(".gz"))
                .Take(2)
                .ToList();

            var fileResults = new List<FtpFile>();

            foreach (var file in searchResults)
            {
                var ftpFile = new FtpFile { FileName = file.Name, FileSize = file.Length };

                using (var fileStream = new MemoryStream())
                {
                    client.DownloadFile(file.FullName, fileStream); // Success! All is good here, so far. :)

                    using (var gzStream = new GZipStream(fileStream, CompressionMode.Decompress))
                    {
                        using (var outputStream = new MemoryStream())
                        {
                            gzStream.CopyTo(outputStream);
                            byte[] outputBytes = outputStream.ToArray(); // No data. Sad panda. :'(
                            ftpFile.FileContents = Encoding.ASCII.GetString(outputBytes);
                            fileResults.Add(ftpFile);
                        }
                    }
                }
            }

            return fileResults;
        }
    }

FtpFile (我正在填充的只是一个简单的DTO):

FtpFile (just a simple DTO I'm populating):

public class FtpFile
{
    public string FileName { get; set; }
    public long FileSize { get; set; }
    public string FileContents { get; set; }
}

---

PSA 如果有人来复制此代码，请注意，这不是很好的代码，因为此代码可能会导致一些严重内存管理问题！最好的做法是将其流式传输到磁盘上，这在代码中不不会完成！我的需求非常具体，因为我必须同时将这些文件存储在内存中，以用于与它们一起构建.

---

PSA If anybody comes and copies this code, be aware that this is NOT good code in that you could have some serious memory management problems with this code! It's best practice to instead stream it to disk, which is not being done in this code! My needs are very specific in that I have to have these files simultaneously in memory for what I'm building with them.

推荐答案

如果要将数据插入流中，请确保在解压缩之前先找回其源头.

If you are inserting data into the stream, make sure to seek back to its origin before un-gzipping it.

以下内容可以解决您的问题:

The following should fix your troubles:

            using (var fileStream = new MemoryStream())
            {
                client.DownloadFile(file.FullName, fileStream); // Success! All is good here, so far. :)
                fileStream.Seek(0, SeekOrigin.Begin);

                using (var gzStream = new GZipStream(fileStream, CompressionMode.Decompress))
                {
                    using (var outputStream = new MemoryStream())
                    {
                        gzStream.CopyTo(outputStream);
                        byte[] outputBytes = outputStream.ToArray(); // No data. Sad panda. :'(
                        ftpFile.FileContents = Encoding.ASCII.GetString(outputBytes);
                        fileResults.Add(ftpFile);
                    }
                }
            }

这篇关于如何使用GZipStream将gz文件解压缩到内存中?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！