KMS加密后，AWS S3 etag与md5不匹配 [英] Aws S3 etag not matching md5 after KMS encryption
问题描述
全部-我们正在努力迁移一些机密信息；从本地UNIX文件系统到S3的法规信息.
All- We are working on migrating some confidential & regulatory information from Local UNIX file system to S3.
使用"aws s3 cp--sse aws:kms --sse-kms-key-id ......."命令，使用AWS EC2实例将文件复制到S3.
The files are copied using AWS EC2 instance into S3 using "aws s3 cp--sse aws:kms --sse-kms-key-id....... " command.
我注意到的是etag与unix md5sum不同.如果我不使用kms密钥加密数据，则完全相同.
What i have noticed is the etag is different from the unix md5sum. It is exactly the same if i don't encrypt the data using kms keys.
我需要验证上传内容，以确保上传到S3时数据没有损坏，我如何验证我的文件是完整的，因为加密导致etag不匹配.
I need to validate the upload to make sure data is not corrupt while uploading to S3, how do i validate my file is intact as etag won't match due to encryption.
我们非常感谢您的帮助！
Any help is really appreciated!
PS:我的文件不是> 5GB，我知道分段上传的问题，它不适用于我....
PS: my files are not > 5gb, i am aware of the issue with multipart upload and it is not applicable for me....
推荐答案
在AWS S3中，etag不是MD5校验和.如果只是这种情况，过去就是这种情况，但是AWS警告不要依赖此方法进行完整性检查.
In AWS S3 the etag is not an MD5 checksum. If just happens that this is the case in the past but AWS warns not to rely on this method for integrity checks.
以下链接是我指的文字:
In the following link is the text that I am referring to:
The ETag may or may not be an MD5 digest of the object data.
实体标签是对象的哈希. ETag仅反映更改 对象的内容，而不是其元数据. ETag可能会或可能会 不是对象数据的MD5摘要.是否存在取决于 关于如何创建对象以及如何按照所述方法对其进行加密的信息 下方:
The entity tag is a hash of the object. The ETag reflects changes only to the contents of an object, not its metadata. The ETag may or may not be an MD5 digest of the object data. Whether or not it is depends on how the object was created and how it is encrypted as described below:
-
由PUT对象，POST对象或复制操作或通过AWS管理控制台创建的对象，并由SSE-S3或 纯文本，具有ETag，这些ETag是其对象数据的MD5摘要.
Objects created by the PUT Object, POST Object, or Copy operation, or through the AWS Management Console, and are encrypted by SSE-S3 or plaintext, have ETags that are an MD5 digest of their object data.
由PUT对象，POST对象或复制操作或通过AWS管理控制台创建的对象，并由SSE-C或 SSE-KMS的ETag不是其对象数据的MD5摘要.
Objects created by the PUT Object, POST Object, or Copy operation, or through the AWS Management Console, and are encrypted by SSE-C or SSE-KMS, have ETags that are not an MD5 digest of their object data.
如果通过分段上传"或分段复制"操作创建了对象，则无论采用哪种方法，ETag都不是MD5摘要. 加密.
If an object is created by either the Multipart Upload or Part Copy operation, the ETag is not an MD5 digest, regardless of the method of encryption.
这篇关于KMS加密后，AWS S3 etag与md5不匹配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
