用于将 CloudWatch 日志导出到 S3 的 Python3.7 脚本

Question

我正在使用以下代码将CloudWatch日志复制到S3：-

import boto3
import collections
from datetime import datetime, date, time, timedelta
region = 'eu-west-1'
def lambda_handler(event, context):
yesterday = datetime.combine(date.today()-timedelta(1),time())
today = datetime.combine(date.today(),time())
unix_start = datetime(1970,1,1)
client = boto3.client('logs')
response = client.create_export_task(
taskName='Export_CloudwatchLogs',
logGroupName='/aws/lambda/stop-instances',
fromTime=int((yesterday-unix_start).total_seconds() * 1000),
to=int((today -unix_start).total_seconds() * 1000),
destination='bucket',
destinationPrefix='bucket-{}'.format(yesterday.strftime("%Y-%m-%d"))
)
return 'Response from export task at {} :n{}'.format(datetime.now().isoformat(),response)

我给角色提供了以下政策：-

policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:DescribeLogStreams",
"logs:CreateExportTask",
"logs:DescribeExportTasks",
"logs:DescribeLogGroups"
],
"Resource": [
"arn:aws:logs:*:*:*"
]
}
]
}
EOF

第二项政策：-

policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:PutObject",
"s3:PutObjectAcl",
"s3:GetBucketAcl"
],
"Effect": "Allow",
"Resource": ["arn:aws:s3:::${var.source_market}-${var.environment}-${var.bucket}/*"],
"Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } }
}
]
}
EOF

如果我在 AWS 控制台中执行此操作，我会收到以下错误：-

{ "errorMessage"： "调用 CreateExportTask 操作时出错 (InvalidParameterException(：给定存储桶上的 GetBucketAcl 调用失败。请检查云观看日志是否已被授予执行此操作的权限。 "错误类型"： "无效参数异常">

在附加了适当的策略后，我已经引用了许多块。

Answer 1

这似乎是 s3 存储桶权限的问题。您需要将此策略附加到 s3 存储桶。请通过更改 CloudWatch 的存储桶名称和 AWS 区域来修改策略。

{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:GetBucketAcl",
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-exported-logs",
"Principal": { "Service": "logs.us-west-2.amazonaws.com" }
},
{
"Action": "s3:PutObject" ,
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-exported-logs/random-string/*",
"Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } },
"Principal": { "Service": "logs.us-west-2.amazonaws.com" }
}
]}

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasksConsole.html

Answer 2

检查存储桶上的加密设置。我遇到了同样的问题，这是因为我将其设置为 AWS-KMS。我以与您拥有的相同权限收到此错误，然后在我将加密切换到 AES-256 后它就开始工作

Answer 3

我遇到了同样的错误，问题是我在策略上放了像桶/东西这样的"目标"参数，而我只有桶，所以删除参数上的前缀解决了问题，所以检查策略和参数是否匹配。