您是否可以仅使用访问S3子文件夹而不访问根文件夹的凭据来设置从S3存储桶子文件夹到GCS存储桶的传输? [英] Can you set up a transfer from an S3 bucket subfolder to a GCS bucket with only the credentials to access the S3 subfolder, not the root folder?

问题描述

我希望设置一个传输作业，以获取存储在S3存储桶中的文件并将其加载到GCS存储桶中.我拥有的凭据可以访问包含我需要的文件的文件夹，但不能访问更高级别的文件夹.

I'm looking to set up a transfer job to take files stored within an S3 bucket and load them to a GCS bucket. The credentials that I have give me access to the folder that contains the files that I need from S3 but not to the higher level folders.

当我尝试使用"Amazon S3存储桶"下的S3存储桶名称以及访问密钥ID&设置传输作业时，机密访问密钥已填写，由于我的凭据限制，访问被拒绝，正如您所期望的那样.但是，如果我将额外的路径信息添加为前缀项(例如'Production/FTP/CompanyName')，并且确实可以访问此文件夹，则访问将被 still 拒绝.

When I try to set up the transfer job with the S3 bucket name under 'Amazon S3 bucket' and the access key ID & secret access key filled-in, the access is denied as you would expect given the limits of my credentials. However, access is still denied if I add the extra path information as a Prefix item (e.g. 'Production/FTP/CompanyName') and I do have access to this folder.

似乎我无法克服我无权访问根目录的事实.有什么办法解决吗?

It seems as though I can't get past the fact that I do not have access to the root directory. Is there any way around this?

推荐答案

根据文档链接:

Storage Transfer Service使用项目-[$ PROJECT_NUMBER] @ storage-transfer-service.iam.gserviceaccount.com服务帐户以从Cloud Storage源存储桶中移动数据.

The Storage Transfer Service uses the project-[$PROJECT_NUMBER]@storage-transfer-service.iam.gserviceaccount.com service account to move data from a Cloud Storage source bucket.

服务帐户必须对源具有以下权限桶:

The service account must have the following permissions for the source bucket:

storage.buckets.get允许服务帐户获取以下位置:桶.始终需要.

storage.buckets.get Allows the service account to get the location of the bucket. Always required.

storage.objects.list允许服务帐户列出对象中的对象.桶.始终需要.

storage.objects.list Allows the service account to list objects in the bucket. Always required.

storage.objects.get允许服务帐户读取对象中的对象.桶.始终需要.

storage.objects.get Allows the service account to read objects in the bucket. Always required.

storage.objects.delete允许服务帐户删除其中的对象桶.如果您设置deleteObjectsFromSourceAfterTransfer，则为必需变为真实.

storage.objects.delete Allows the service account to delete objects in the bucket. Required if you set deleteObjectsFromSourceAfterTransfer to true.

roles/storage.objectViewer和role/storage.legacyBucketReader角色一起包含始终需要的权限.这role/storage.legacyBucketWriter角色包含storage.objects.delete权限.该服务帐户过去曾必须为执行转移的人员分配所需的角色.

The roles/storage.objectViewer and roles/storage.legacyBucketReader roles together contain the permissions that are always required. The roles/storage.legacyBucketWriter role contains the storage.objects.delete permissions. The service account used to perform the transfer must be assigned the desired roles.

您必须在AWS存储桶上设置此权限.

You have to set this permissions on your AWS bucket.

这篇关于您是否可以仅使用访问S3子文件夹而不访问根文件夹的凭据来设置从S3存储桶子文件夹到GCS存储桶的传输?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！