我正在运行一个在Docker上运行的MWAA实例的本地环境。一切都很好，我尝试将SecretsManager设置为仅用于连接的备用后端。
我已经将其添加到airflow.cfg文件中：
[secrets]
backend = airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend
backend_kwargs = {"connections_prefix" : "airflow/connections", "variables_prefix" : null, "config_prefix" : null}
然后，我在Airflow Connections中配置了一个aws_default连接：类型：亚马逊网络服务
名称：aws_default
登录：<aws_access_key>
密码：<aws_secret_access_key>
我使用一个函数验证了凭据是否正常工作，该函数使用此连接ID启动AWSHook。
我向凭据所属的用户添加了SecretsManager readWrite策略。
然后，我尝试使用SnowflakeOperator对Snowflake运行查询，以便从SecretsManager获取连接，但我得到这个错误，就好像后端SecretsMnager服务找不到使用API和查询机密的凭据一样。xxxx_snowflake_operator和xxxx_snowflake_book只是snowflake_operator和snowflake_hook的包装器，做完全相同的事情而没有任何更改(只有UI的颜色(
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1138, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1311, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1341, in _execute_task
result = task_copy.execute(context=context)
File "/usr/local/airflow/plugins/operators/xxxx_snowflake_operator.py", line 84, in execute
raise ex
File "/usr/local/airflow/plugins/operators/xxxx_snowflake_operator.py", line 78, in execute
parameters=self.parameters)
File "/usr/local/lib/python3.7/site-packages/airflow/hooks/dbapi.py", line 173, in run
with closing(self.get_conn()) as conn:
File "/usr/local/lib/python3.7/site-packages/airflow/providers/snowflake/hooks/snowflake.py", line 215, in get_conn
conn_config = self._get_conn_params()
File "/usr/local/airflow/plugins/hooks/xxxx_snowflake_hook.py", line 22, in _get_conn_params
conn = self.get_connection(self.snowflake_conn_id)
File "/usr/local/lib/python3.7/site-packages/airflow/hooks/base.py", line 67, in get_connection
conn = Connection.get_connection_from_secrets(conn_id)
File "/usr/local/lib/python3.7/site-packages/airflow/models/connection.py", line 351, in get_connection_from_secrets
conn = secrets_backend.get_connection(conn_id=conn_id)
File "/usr/local/lib/python3.7/site-packages/airflow/secrets/base_secrets.py", line 64, in get_connection
conn_uri = self.get_conn_uri(conn_id=conn_id)
File "/usr/local/lib/python3.7/site-packages/airflow/providers/amazon/aws/secrets/secrets_manager.py", line 115, in get_conn_uri
return self._get_secret(self.connections_prefix, conn_id)
File "/usr/local/lib/python3.7/site-packages/airflow/providers/amazon/aws/secrets/secrets_manager.py", line 153, in _get_secret
SecretId=secrets_path,
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 663, in _make_api_call
operation_model, request_dict, request_context)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 682, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 132, in _send_request
request = self.create_request(request_dict, operation_model)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 116, in create_request
operation_name=operation_model.name)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/signers.py", line 90, in handler
return self.sign(operation_name, request)
File "/usr/local/lib/python3.7/site-packages/botocore/signers.py", line 162, in sign
auth.add_auth(request)
File "/usr/local/lib/python3.7/site-packages/botocore/auth.py", line 373, in add_auth
raise NoCredentialsError()
botocore.exceptions.NoCredentialsError: Unable to locate credentials
为什么SecretsManager Airflow后端找不到凭据？它不使用默认的AWS连接ID来运行其botocore API请求吗？
UPDATE：我通过在~/.aws/config中设置一个包含aws访问权限和密钥的配置文件来实现它。我想知道我是否可以在Airflow中使用aws_default连接。
您可以使用以下参数更新airflow.cfg：
backend_kwargs = {"connections_prefix" : "airflow/connections", "variables_prefix" : "airflow/variables", "aws_access_key_id":"XXXXXXXXXXXXXX", "aws_secret_access_key":"XXXXXXXXXXXXXX","region_name":"XXXXXXXXXXXXXX"}
重建图像
./mwaa-local-env build-image
然后再次启动气流
./mwaa-local-env start
请注意，在AWS上设置MWAA环境时，不应设置aws_access_key_id、aws_secret_access_key和region_name。