Env
文件:
AWS_ACCESS_KEY_ID=AAAAAAAAAAAAAAAAAA
AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxx
AWS_DEFAULT_REGION=eu-west-1
AWS_DEFAULT_OUTPUT=json
Config
file:
[profile a]
credential_source = Environment
region=eu-west-1
role_arn=arn
[profile b]
credential_source = Environment
region=eu-west-1
role_arn=arn
[profile c]
credential_source = Environment
region=eu-west-1
role_arn=arn
Dockerfile
含量:
FROM apache/airflow:2.1.2-python3.8
ARG AIRFLOW_USER_HOME=/opt/airflow
ENV PYTHONPATH "${PYTHONPATH}:/"
RUN pip install -r requirements.pip
ADD ./my_path/aws/config /home/airflow/.aws/config
音量:
- ./my_path/aws:/home/airflow/.aws
如果我运行:
docker exec -it my-container bash
aws configure list
我看到:
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key *******************A env
secret_key *******************x env
region eu-west-1 env AWS_DEFAULT_REGION
使用boto3
运行一些Python3
代码时,我得到的错误是:
Error when retrieving credentials from Environment: No credentials found in credential_source referenced in profile.
如果设置了acces_key和secret,并且配置文件表明源是环境,为什么会发生这种情况?
额外测试:
我在容器内运行:aws s3 ls
,它列出了所有内容,因此凭证是从env文件正确设置的。
而且引发这个问题的python代码是:
class Boto3AwsClient(object):
def __init__(self, localhost, profile):
self.running_localhost = localhost
self.profile = profile
if self.running_localhost == 1:
self.session = boto3.Session(profile_name=self.profile)
else:
self.session = boto3.Session()
def aws_client_connect(self, service=None):
if service is None:
raise ValueError('Service is not defined in new boto3 session.')
service_client = self.session.client(service)
return service_client
我也尝试像这样设置config
文件:
[default]
credential_source = Environment
region=eu-west-1
output=json
[profile a]
region=eu-west-1
role_arn=arn
[profile b]
region=eu-west-1
role_arn=arn
[profile c]
region=eu-west-1
role_arn=arn
也不工作。唯一可行的方法是使用配置文件和凭证文件,而不设置ENV变量。问题是,这不是一个好的做法。
为访问s3的实例分配正确的IAM角色