在S3 Amazon上合并文件 [英] Merging files on S3 Amazon

问题描述

我有一个应用程序，我想将s3上存在的两个文件合并到第三个文件中.我想到了使用分段上传来使用复制对象的方法.下面是代码.

I have an application where i want to merge two files present on s3 into the third file. I thought of using the Copy Object using multipart upload. Below is the code.

       AmazonS3Config config = new AmazonS3Config();
        AmazonS3 s3Client = new AmazonS3Client(accessKeyID, secretAccessKey, config);

        // List to store upload part responses.
        List<UploadPartResponse> uploadResponses =
            new List<UploadPartResponse>();

        List<CopyPartResponse> copyResponses =
                  new List<CopyPartResponse>();
        InitiateMultipartUploadRequest initiateRequest =
               new InitiateMultipartUploadRequest()
                .WithBucketName(targetBucket)
                .WithKey(targetObjectKey);

        InitiateMultipartUploadResponse initResponse =
            s3Client.InitiateMultipartUpload(initiateRequest);
        String uploadId = initResponse.UploadId;

        try
        {
            // Get object size.
            GetObjectMetadataRequest metadataRequest = new GetObjectMetadataRequest();
            metadataRequest.BucketName = sourceBucket;
            metadataRequest.Key = sourceObjectKey1;

            GetObjectMetadataResponse metadataResponse = s3Client.GetObjectMetadata(metadataRequest);
            long objectSize1 = metadataResponse.ContentLength; // in bytes

            // Get object size.
            GetObjectMetadataRequest metadataRequest2 = new GetObjectMetadataRequest();
            metadataRequest2.BucketName = sourceBucket;
            metadataRequest2.Key = sourceObjectKey2;

            GetObjectMetadataResponse metadataResponse2 = s3Client.GetObjectMetadata(metadataRequest2);
            long objectSize2 = metadataResponse2.ContentLength; // in bytes

            long bytePosition = 0;


            CopyPartRequest copyRequest1 = new CopyPartRequest()
               .WithDestinationBucket(targetBucket)
               .WithDestinationKey(targetObjectKey)
               .WithSourceBucket(sourceBucket)
               .WithSourceKey(sourceObjectKey1)
               .WithUploadID(uploadId)
               .WithFirstByte(bytePosition)
               .WithLastByte( objectSize1 - 1 )
               .WithPartNumber(1);

            copyResponses.Add(s3Client.CopyPart(copyRequest1));


            CopyPartRequest copyRequest2 = new CopyPartRequest()
               .WithDestinationBucket(targetBucket)
               .WithDestinationKey(targetObjectKey)
               .WithSourceBucket(sourceBucket)
               .WithSourceKey(sourceObjectKey2)
               .WithUploadID(uploadId)
               .WithFirstByte(bytePosition)
               .WithLastByte(objectSize2 - 1)
               .WithPartNumber(2);

            copyResponses.Add(s3Client.CopyPart(copyRequest2));

            ////
            CompleteMultipartUploadRequest completeRequest =
                  new CompleteMultipartUploadRequest()
                     .WithBucketName(targetBucket)
                     .WithKey(targetObjectKey)
                     .WithUploadId(initResponse.UploadId)
                     .WithPartETags(GetETags(copyResponses));

            CompleteMultipartUploadResponse completeUploadResponse =
                s3Client.CompleteMultipartUpload(completeRequest);

        }
        catch (Exception e)
        {
            Console.WriteLine(e.Message);
        }

但是它在CompleteMultipartUpload的最后一行抛出异常.以下是S3的例外情况:您建议的上传大小小于允许的最小大小

But it is throwing exception at the last line CompleteMultipartUpload. Below is the S3 exception: Your proposed upload is smaller than the minimum allowed size

好像我只上传copyRequest1一样，效果很好.

Where as if i only upload copyRequest1 it works fine.

感谢您的帮助！

关于， 长谷ena

推荐答案

无法使用S3 API合并上传的文件，因此我正在使用FTP下载和合并.

It is not possible to merge uploaded files using S3 API, so I am using FTP to download and Merge.

这篇关于在S3 Amazon上合并文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！