我正试图在具有云功能的composer环境中触发Airflow DAG。为了做到这一点,我需要获得这里描述的客户端id。我尝试过使用curl命令,但它没有返回任何值。使用python脚本,我不断收到以下错误:
Traceback (most recent call last):
File "get_client_id.py", line 55, in <module>
get_client_id(
File "get_client_id.py", line 40, in get_client_id
print(query_string['client_id'][0])
KeyError: 'client_id'
Python代码:
import google.auth
import google.auth.transport.requests
import requests
import six.moves.urllib.parse
# Authenticate with Google Cloud.
# See: https://cloud.google.com/docs/authentication/getting-started
credentials, _ = google.auth.default(
scopes=['https://www.googleapis.com/auth/cloud-platform'])
authed_session = google.auth.transport.requests.AuthorizedSession(
credentials)
project_id = 'my-project'
location = 'my-region'
composer_environment = 'my-env'
environment_url = (
'https://composer.googleapis.com/v1beta1/projects/{}/locations/{}'
'/environments/{}').format(project_id, location, composer_environment)
composer_response = authed_session.request('GET', environment_url)
environment_data = composer_response.json()
airflow_uri = environment_data['config']['airflowUri']
# The Composer environment response does not include the IAP client ID.
# Make a second, unauthenticated HTTP request to the web server to get the
# redirect URI.
redirect_response = requests.get(airflow_uri, allow_redirects=False)
redirect_location = redirect_response.headers['location']
# Extract the client_id query parameter from the redirect.
parsed = six.moves.urllib.parse.urlparse(redirect_location)
query_string = six.moves.urllib.parse.parse_qs(parsed.query)
print(query_string['client_id'][0])
cURL命令:
curl -v my_airflow_url 2>&1 >/dev/null | grep -o "client_id=[A-Za-z0-9-]*.apps.googleusercontent.com"
有人知道如何获得IAM代理的Gettheclient_id
吗?
发布此Community Wiki
以获得更好的visibility
。
如@LEC
在评论部分所述,该配置与Cloud Composer V1
兼容,CCD_5可在GCP文档中找到,该文档使用云功能触发DAG。
此时可以找到两个选项卡Cloud Composer 1 Guides
和Cloud Composer 2 Guides
。在Cloud Composer 1
下是OP使用的代码,但如果您将在Manage DAGs
>用云函数触发DAG,你会得到还没有适当文档的信息。
Cloud Composer 2的此文档页尚不可用。请使用Cloud Composer 1的页面。
作为解决方案,请使用Cloud Composer V1
。
以下是指南,其中包含说明如何在Composer 2/Airflow中触发DAG-https://cloud.google.com/composer/docs/composer-2/access-airflow-api