AWS Client在使用 Lambda 和 S3 将数据插入存储桶时出错



我正在尝试使用 lambda 将 json blob 放入 S3 存储桶中,但在查看云监视日志时出现以下错误

[ERROR] ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Traceback (most recent call last):
File "/var/task/main.py", line 147, in lambda_handler
save_articles_and_comments(sub, submissions)
File "/var/task/main.py", line 125, in save_articles_and_comments
object.put(Body=json.dumps(articles))
File "/var/task/boto3/resources/factory.py", line 520, in do_action
response = action(self, *args, **kwargs)
File "/var/task/boto3/resources/action.py", line 83, in __call__
response = getattr(parent.meta.client, operation_name)(*args, **params)
File "/var/task/botocore/client.py", line 316, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/task/botocore/client.py", line 635, in _make_api_call
raise error_class(parsed_response, operation_name)

所有阻止公有访问设置都设置为"off",代码中的存储桶名称与 S3 中的存储桶名称相同。这是将 json blob 放入我的 S3 存储桶和 lambda 处理程序中的可修复文件夹中

的代码
def save_articles_and_comments(sub, submissions):
"""
"""
s3 = boto3.resource('s3')
now = dt.datetime.utcnow()
formatted_date = now.strftime("%Y-%m-%d-%H-%M-%S")
articles, comments = data_for_subreddit(submissions)
print("Number of articles, comments {}, {}".format(len(articles), len(comments)))
articles_name = 'articles/' + formatted_date + '_' + sub + '_articles.json'
comments_name = 'comments/' + formatted_date + '_' + sub + '_comments.json'
object = s3.Object('diegos-reddit-bucket', articles_name)
object.put(Body=json.dumps(articles))
print("Finished writing articles to {}".format(articles_name))
object = s3.Object('diegos-reddit-bucket', comments_name)
object.put(Body=json.dumps(comments))
print("Finished writing comments to {}".format(comments_name))

def lambda_handler(x, y):
"""
"""
import time
import random
idx = random.randint(0, len(SUBREDDITS)-1)
start = time.time()
assert PRAW_KEY is not None
sub = SUBREDDITS[idx]
red = reddit_instance()
subreddit = red.subreddit(sub)
print("Pulling posts from {}, {}.".format(sub, "hot"))
submissions = subreddit.hot()
save_articles_and_comments(sub, submissions)
print("="*50)
print("Pulling posts from {}, {}.".format(sub, "new"))
submissions = subreddit.new()
save_articles_and_comments(sub, submissions)
print("="*50)
print("Pulling posts from {}, {}.".format(sub, "top"))
submissions = subreddit.top()
save_articles_and_comments(sub, submissions)
print("="*50)
print("Pulling posts from {}, {}.".format(sub, "rising"))
submissions = subreddit.rising()
save_articles_and_comments(sub, submissions)
end = time.time()
print("Elapsed time {}".format(end - start))

我看不出代码中有什么问题让我得到所说的错误。 将我的lambda_handler函数换成了一个 main 在本地测试。使用主,它可以工作,并写入 S3 存储桶及其受关注的文件夹。当我尝试通过 AWS Lambda 运行时,在函数完成从第一个 subreddit 中提取帖子并尝试将 json blob 放入 S3 存储桶中的文件夹中后,我收到上述错误。这就是我的输出应该是什么样子的

Pulling posts from StockMarket, hot.
Number of articles, comments 101, 909
Finished writing articles to articles/2020-06-03-02-48-44_StockMarket_articles.json
Finished writing comments to comments/2020-06-03-02-48-44_StockMarket_comments.json
==================================================
Pulling posts from StockMarket, new.
Number of articles, comments 101, 778
Finished writing articles to articles/2020-06-03-02-49-10_StockMarket_articles.json
Finished writing comments to comments/2020-06-03-02-49-10_StockMarket_comments.json
==================================================
Pulling posts from StockMarket, top.
Number of articles, comments 101, 5116
Finished writing articles to articles/2020-06-03-02-49-36_StockMarket_articles.json
Finished writing comments to comments/2020-06-03-02-49-36_StockMarket_comments.json
==================================================
Pulling posts from StockMarket, rising.
Number of articles, comments 24, 170
Finished writing articles to articles/2020-06-03-02-52-10_StockMarket_articles.json
Finished writing comments to comments/2020-06-03-02-52-10_StockMarket_comments.json
Elapsed time 215.6588649749756

我的代码中是否存在问题,还是 AWS 方面的问题?

出现此问题的原因是您没有将对象写入存储桶的权限:

放置对象操作:访问被拒绝

要纠正此问题,必须查看 lambda 执行角色:它是否有权写入 S3?还可以检查存储桶策略。

使用主,它可以工作,并写入 S3 存储桶及其受关注的文件夹。当我尝试通过 AWS Lambda 运行时,出现上述错误

当您在本地进行测试时,您的代码将使用您自己的权限(您的 IAM 用户(写入 S3。因此它有效。当您在 lambda 上执行代码时,您的函数不会使用您的权限。相反,它使用lambda 执行角色中定义的权限。

最新更新