如何将s3中的对象从帐户A复制到具有更新的对象所有权的帐户B到对象A [英] How to copy the object in s3 from account A to Account B with updated object ownership to object A
问题描述
我的代码将对象从帐户A复制到帐户B
import json
import boto3
from datetime import datetime, timedelta
def lambda_handler(event, context):
# TODO implement
SOURCE_BUCKET = 'Bucket-A'
DESTINATION_BUCKET = 'Bucket-B'
s3_client = boto3.client('s3')
# Create a reusable Paginator
paginator = s3_client.get_paginator('list_objects_v2')
# Create a PageIterator from the Paginator
page_iterator = paginator.paginate(Bucket=SOURCE_BUCKET)
# Loop through each object, looking for ones older than a given time period
for page in page_iterator:
if "Contents" in page:
for object in page['Contents']:
if object['LastModified'] < datetime.now().astimezone() - timedelta(minutes=5): # <-- Change time period here
print(f"Moving {object['Key']}")
# Copy object
s3_client.copy_object(
ACL='bucket-owner-full-control',
Bucket=DESTINATION_BUCKET,
Key=object['Key'],
CopySource={'Bucket':SOURCE_BUCKET, 'Key':object['Key']}
)
# Delete original object
s3_client.delete_object(Bucket=SOURCE_BUCKET, Key=object['Key'])
else:
print("No Contents key for page!")
lambda函数角色策略为:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:PutObject*",
"s3:List*",
"s3:GetObject*",
"s3:GetBucketLocation",
"s3:DeleteObject*"
],
"Resource": [
"arn:aws:s3:::Bucket-A/*",
"arn:aws:s3:::bucket-A"
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:PutObjectAcl",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::bucket-B/*"
}
]
}
aws s3api get-object-acl --bucket bucket-b --key key1
{
"Owner": {
"DisplayName": "accountA",
"ID": "MYIDA"
},
"Grants": [
{
"Grantee": {
"DisplayName": "accountA",
"ID": "MyIDA",
"Type": "CanonicalUser"
},
"Permission": "FULL_CONTROL"
},
{
"Grantee": {
"DisplayName": "accountb",
"ID": "MyIDB",
"Type": "CanonicalUser"
},
"Permission": "FULL_CONTROL"
}
]
}
从帐户A复制到帐户B到帐户B时，如何更改对象的所有者
{
"Owner": {
"DisplayName": "accountB",
"ID": "MYIDB"
},
我将首先尝试解决您的问题，然后为用例提供更好的方法.
首先，如已正确识别，您要查找的是对象ACL. Boto3为您提供了一种方式来检索和更新对象ACL，从而为对象所有者提供方式来检索对象ACL，从而使对象所有者以及更新ACL.在官方文档中详细了解此内容这里.
作为参考，这是示例请求语法:
response = object_acl.put(
ACL='private'|'public-read'|'public-read-write'|'authenticated-read'|'aws-exec-read'|'bucket-owner-read'|'bucket-owner-full-control',
AccessControlPolicy={
'Grants': [
{
'Grantee': {
'DisplayName': 'string',
'EmailAddress': 'string',
'ID': 'string',
'Type': 'CanonicalUser'|'AmazonCustomerByEmail'|'Group',
'URI': 'string'
},
'Permission': 'FULL_CONTROL'|'WRITE'|'WRITE_ACP'|'READ'|'READ_ACP'
},
],
'Owner': {
'DisplayName': 'string',
'ID': 'string'
}
},
GrantFullControl='string',
GrantRead='string',
GrantReadACP='string',
GrantWrite='string',
GrantWriteACP='string',
RequestPayer='requester',
VersionId='string'
)
现在可以找到一种更好的方法来实现此目的.看一下AWS跨区域复制.在公告帖子此处此处了解更多信息. 或参考文档.. >
要使用文档中的描述，请执行以下操作:
通过复制，可以跨Amazon S3存储桶自动，异步地复制对象.为对象复制配置的存储桶可以由同一AWS账户或不同账户拥有.您可以在不同的AWS区域之间或同一区域内复制对象.
以不同的所有权维护对象副本-无论谁拥有源对象，您都可以告诉Amazon S3将副本所有权更改为拥有目标存储桶的AWS账户.这称为所有者覆盖选项.您可以使用此选项来限制对对象副本的访问.
基本上，您可以使用LifeCycle策略并自动执行整个过程.您还可以配置要使用新所有者创建的目标对象.这样，您就可以将管理工作转移到AWS上，并使流程具有响应性.从长远来看，这将帮助您节省人力成本和资源使用成本.
My code copies the object from Account A to Account B
import json
import boto3
from datetime import datetime, timedelta
def lambda_handler(event, context):
# TODO implement
SOURCE_BUCKET = 'Bucket-A'
DESTINATION_BUCKET = 'Bucket-B'
s3_client = boto3.client('s3')
# Create a reusable Paginator
paginator = s3_client.get_paginator('list_objects_v2')
# Create a PageIterator from the Paginator
page_iterator = paginator.paginate(Bucket=SOURCE_BUCKET)
# Loop through each object, looking for ones older than a given time period
for page in page_iterator:
if "Contents" in page:
for object in page['Contents']:
if object['LastModified'] < datetime.now().astimezone() - timedelta(minutes=5): # <-- Change time period here
print(f"Moving {object['Key']}")
# Copy object
s3_client.copy_object(
ACL='bucket-owner-full-control',
Bucket=DESTINATION_BUCKET,
Key=object['Key'],
CopySource={'Bucket':SOURCE_BUCKET, 'Key':object['Key']}
)
# Delete original object
s3_client.delete_object(Bucket=SOURCE_BUCKET, Key=object['Key'])
else:
print("No Contents key for page!")
The lambda function role policy is :
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:PutObject*",
"s3:List*",
"s3:GetObject*",
"s3:GetBucketLocation",
"s3:DeleteObject*"
],
"Resource": [
"arn:aws:s3:::Bucket-A/*",
"arn:aws:s3:::bucket-A"
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:PutObjectAcl",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::bucket-B/*"
}
]
}
aws s3api get-object-acl --bucket bucket-b --key key1
{
"Owner": {
"DisplayName": "accountA",
"ID": "MYIDA"
},
"Grants": [
{
"Grantee": {
"DisplayName": "accountA",
"ID": "MyIDA",
"Type": "CanonicalUser"
},
"Permission": "FULL_CONTROL"
},
{
"Grantee": {
"DisplayName": "accountb",
"ID": "MyIDB",
"Type": "CanonicalUser"
},
"Permission": "FULL_CONTROL"
}
]
}
How can I change the owner of the object while copying from Account A to account B to Account B
{
"Owner": {
"DisplayName": "accountB",
"ID": "MYIDB"
},
I'll try to address your question first and then provide a better approach for the use case.
Firstly, as you have correctly identified, what you are looking for are Object ACL. Boto3 gives you a way to retrieve and update Object ACLs and thus the object owners way to retrieve Object ACLs and thus the object owners as well as update ACL. Read more about this in the official docs here. To read about Object ACLs, you can refer to the docs here.
For reference, here's a sample Request Syntax:
response = object_acl.put(
ACL='private'|'public-read'|'public-read-write'|'authenticated-read'|'aws-exec-read'|'bucket-owner-read'|'bucket-owner-full-control',
AccessControlPolicy={
'Grants': [
{
'Grantee': {
'DisplayName': 'string',
'EmailAddress': 'string',
'ID': 'string',
'Type': 'CanonicalUser'|'AmazonCustomerByEmail'|'Group',
'URI': 'string'
},
'Permission': 'FULL_CONTROL'|'WRITE'|'WRITE_ACP'|'READ'|'READ_ACP'
},
],
'Owner': {
'DisplayName': 'string',
'ID': 'string'
}
},
GrantFullControl='string',
GrantRead='string',
GrantReadACP='string',
GrantWrite='string',
GrantWriteACP='string',
RequestPayer='requester',
VersionId='string'
)
Now coming to a better way to implement this. Have a look at AWS Cross Region Replication. Read more about it in the announcement post here or refer to the docs.
To use the description from the docs:
Replication enables automatic, asynchronous copying of objects across Amazon S3 buckets. Buckets that are configured for object replication can be owned by the same AWS account or by different accounts. You can copy objects between different AWS Regions or within the same Region.
Maintain object copies under different ownership — Regardless of who owns the source object, you can tell Amazon S3 to change replica ownership to the AWS account that owns the destination bucket. This is referred to as the owner override option. You can use this option to restrict access to object replicas.
Essentially you can use LifeCycle Policies and automate the whole process. You can also configure the destination objects to be created with a new Owner. With this you are offloading the management to AWS and also making the process reactive. In the long run, this will help you save costs both in terms of man-power costs and resource usage.
这篇关于如何将s3中的对象从帐户A复制到具有更新的对象所有权的帐户B到对象A的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
