我有以下复制命令:
copy ink.contact_trace_records
from 's3://mybucket/mykey'
iam_role 'arn:aws:iam::accountnum:role/rolename'
json 'auto';
角色具有对 S3 的完全访问权限和对 RS 策略的完全访问权限(我知道这不是一个好主意,但我只是在这里丢失了它:)(。群集具有附加的角色。集群具有 VPC 增强型路由。我得到以下信息:
[2019-05-28 14:07:34] [XX000][500310] [Amazon](500310) Invalid operation: S3ServiceException:Access Denied,Status 403,Error AccessDenied,Rid 370F75618922AFC0,ExtRid dp2mcnlofFzt4dz00Lcm188/+ta3OEKVpuFjnZSYsC0pJPMiULk7I6spOpiZwXc04VVRdxizIj4=,CanRetry 1
[2019-05-28 14:07:34] Details:
[2019-05-28 14:07:34] -----------------------------------------------
[2019-05-28 14:07:34] error: S3ServiceException:Access Denied,Status 403,Error AccessDenied,Rid 370F75618922AFC0,ExtRid dp2mcnlofFzt4dz00Lcm188/+ta3OEKVpuFjnZSYsC0pJPMiULk7I6spOpiZwXc04VVRdxizIj4=,CanRetry 1
[2019-05-28 14:07:34] code: 8001
[2019-05-28 14:07:34] context: S3 key being read : s3://redacted
[2019-05-28 14:07:34] query: 7794423
[2019-05-28 14:07:34] location: table_s3_scanner.cpp:372
[2019-05-28 14:07:34] process: query3_986_7794423 [pid=13695]
[2019-05-28 14:07:34] -----------------------------------------------;
我在这里错过了什么?集群对 S3 具有完全访问权限,它甚至不在自定义 VPC 中,而是在默认 VPC 中。思潮?
检查对象所有者。该对象可能归其他帐户所有。这就是 S3 403 在许多情况下的原因,其中具有完全权限的角色或账户无法访问对象。典型的指标是您可以list
对象,但不能get
或head
它们。
在以下示例中,我尝试从第二个帐户访问 Redshift 审核日志。使用以下策略向账户012345600000
授予对999999999999
拥有的存储桶的访问权限:
{"Version": "2012-10-17",
"Statement": [
{"Action": [
"s3:Get*",
"s3:ListBucket*"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::my-audit-logs",
"arn:aws:s3:::my-audit-logs/*"
],
"Principal": {
"AWS": [
"arn:aws:iam::012345600000:root"
]}}]
}
现在我尝试列出(s3 ls
(然后复制(s3 cp
(单个日志对象:
aws --profile 012345600000 s3 ls s3://my-audit-logs/AWSLogs/999999999999/redshift/us-west-2/2019/05/25/
# 2019-05-28 14:49:46 376 …connectionlog_2019-05-25T19:03.gz
aws --profile 012345600000 s3 cp s3://my-audit-logs/AWSLogs/999999999999/redshift/us-west-2/2019/05/25/…connectionlog_2019-05-25T19:03.gz ~/Downloads/…connectionlog_2019-05-25T19:03.gz
# fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
然后,我从拥有存储桶的账户中检查对象所有权。
aws --profile 999999999999 s3api get-object-acl --bucket my-audit-logs --key AWSLogs/999999999999/redshift/us-west-2/2019/05/25/…connectionlog_2019-05-25T19:03.gz
# "Owner": {
# "DisplayName": "aws-cm-prod-auditlogs-uswest2", ## Not my account!
# "ID": "b2b456ce30a967fb1877b3c9594773ae0275fee248e3ebdbff43d66907b89144"
# },
然后,我使用 --acl bucket-owner-full-control
将对象复制到同一存储桶中。这使我成为新对象的所有者。请注意更改的SharedLogs/
前缀。
aws --profile 999999999999 s3 cp
s3://my-audit-logs/AWSLogs/999999999999/redshift/us-west-2/2019/05/25/…connectionlog_2019-05-25T19:03.gz
s3://my-audit-logs/SharedLogs/999999999999/redshift/us-west-2/2019/05/25/…connectionlog_2019-05-25T19:03.gz
--acl bucket-owner-full-control
现在,我可以下载(或加载到 Redshift!(从同一存储桶共享的新对象。
aws --profile 012345600000 s3 cp s3://my-audit-logs/SharedLogs/999999999999/redshift/us-west-2/2019/05/25/…connectionlog_2019-05-25T19:03.gz ~/Downloads/…connectionlog_2019-05-25T19:03.gz
# download: …