AWS S3:有什么方法可以计算上传文件的行数? [英] AWS S3 : Is there any way to count rows of uploaded file?

问题描述

如果要确定上传任务正确结束，我们想检查AWS S3上已上传文件的行数.

We'd like to check the row count of uploaded files on AWS S3, if the uploading task is sure to be end correctly.

现在，在从SQL Server导出后，我们只需按照以下Windows AWS CLI命令发送导出的文件；

Now we're simply sending exported files by Windows AWS CLI command as follows, after exporting from SQL Server;

aws s3 cp !SEND_FILE! %S3_DIR%/ 

请告知我们是否有任何方法可以检查S3存储桶中已上传文件的数量. 如果没有适当的方法对它们进行计数，那么也希望通过替代方法来获取任何上传错误代码.

Kindly let us know if there is any way to check the count of uploaded files on S3 bucket. If there is no appropriate way to count them, to get any uploading error code as altanative way would be also appreciated.

谢谢您的建议.

推荐答案

如果不重新下载文件，则无法直接计算S3上的行数.但是，这不是必需的.从REST API 文档:

Without re-downloading the file there isn't a way to count the lines on S3 directly. However, this is not needed. From the REST API documentation:

Amazon S3从未添加部分对象；如果您获得成功 作为响应，Amazon S3将整个对象添加到存储桶中.

Amazon S3 never adds partial objects; if you receive a success response, Amazon S3 added the entire object to the bucket.

完成此操作的主要方法是验证在客户端(即运行CLI的位置)上计算出的MD5哈希与它存储的对象相同.来自同一文档:

The primary way this is accomplished is by verifying the MD5 hash that is calculated on the client (i.e. where you're running the CLI) is the same as the object it stored. From the same docs:

为确保遍历网络的数据没有损坏，请使用 Content-MD5标头.使用此标头时，Amazon S3会检查 反对提供的MD5值，如果不匹配， 返回错误.

To ensure that data is not corrupted traversing the network, use the Content-MD5 header. When you use this header, Amazon S3 checks the object against the provided MD5 value and, if they do not match, returns an error.

CLI将自动为您计算MD5哈希并将其发送到S3.因此，如果对象(文件)未正确上传，您将在CLI中返回一个错误，通知您.

The CLI will automatically calculate the MD5 hash for you and send it to S3. So if the object (file) did not upload correctly you would get an error back in the CLI letting you know that.

这篇关于AWS S3:有什么方法可以计算上传文件的行数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！