从GCS下载对象时如何检查MD5 [英] How to check MD5 when downloading object from GCS

问题描述

从GCS下载文件时，我想进行MD5检查.但是，似乎我没有得到正确的MD5……..一个例子得到了:

I wanted to do MD5 check when I download file from GCS. However, it seems that I didn't get the correct MD5 my on side..... One example s that got :

local 1B2M2Y8AsgTpgAmY7PhCfg==, cloud JWSLJAR+M3krp1RiOAJzOw==

但是我很确定文件没有损坏...

But I'm pretty sure the file isn't corrupted...

以下代码与C#7.0配合使用，使用System.Security.Cryptography;

The following code are with C#7.0, using System.Security.Cryptography;

           using (var memStream = new MemoryStream())
            {
                _StorageClient.DownloadObject(bucketName, gcsObj.Name, memStream);
                try
                {
                    using (var md5 = MD5.Create())
                    {
                        var hash = md5.ComputeHash(memStream);
                        localMd5 = Convert.ToBase64String(hash);
                    }
                    Console.WriteLine($"local {localMd5}, cloud {gcsObj.Md5Hash}");
                }
                catch
                {
                    Console.WriteLine("Error getting md5 checksum");
                }
            }

另一个问题是: 我尝试获取文件的CRC32C值的 c#lib 仅返回一个uint类型，但GCS对象的Crc32C值是一个字符串.如何比较它们?

Another question is: the c# lib that I tried to get the CRC32C value of a file only return an uint type, but the GCS object's Crc32C value is a string. How to compare them?

推荐答案

从您的示例中，我假设您的示例哈希来自x-goog-hash标头?

From your sample, I'm assuming your sample hash comes from the x-goog-hash header?

如果是这种情况，您可以检查同一文件的值x-goog-stored-content-encoding是什么吗?如果它是gzip，则将压缩副本上载到GCS，并以gzip格式存储.在这种情况下，x-goog-hash是存储在GCS上的 压缩副本 的MD5.

If that is the case, can you check what is the value x-goog-stored-content-encoding for the same file? If it is gzip, you uploaded a compressed copy to GCS and it is stored in gzip format. In that case, x-goog-hash is the MD5 of the gzipped copy stored on GCS.

要验证该版本，您必须下载压缩版本(不确定使用的C#库是否可以实现此功能)，然后检查该版本的MD5哈希值.

To verify it you'd have to download the compressed version (not sure if that's possible with the C# library you're using), and check the MD5 hash of that.

对于CRC32C，您可以使用以下方法:

For the CRC32C, you can use this:

Convert.ToBase64String(BitConverter.GetBytes(crc32c))

但是同样的情况也适用:如果是gzip压缩版本，则为gzip压缩版本的CRC32C.

But the same thing applies: if it is gziped, this is the CRC32C of the gzipped version.

要检查对象元数据，可以使用:

To check object metadata you can use:

gsutil stat gs://some-bucket/some-object

示例输出:

Creation time:          Sat, 20 Jan 2018 11:09:11 GMT
Update time:            Sat, 20 Jan 2018 11:09:11 GMT
Storage class:          MULTI_REGIONAL
Content-Encoding:       gzip
Content-Length:         5804
Content-Type:           application/msword
Hash (crc32c):          kxvpkw==
Hash (md5):             bfH75gryTXKgNosp1Smxvw==
ETag:                   CO7sotCz5tgCEAE=
Generation:             1516446551684718
Metageneration:         1

该对象以gzip格式存储. MD5/CRC32C都不会与解压缩后的副本匹配.

This object is stored in gzip format. Neither MD5/CRC32C will match those of the decompressed copy.

这篇关于从GCS下载对象时如何检查MD5的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！