如何通过apache气流运行shell脚本

Question

我试图通过气流运行shell脚本，shell脚本在本地执行时工作。

这是我的Dag代码:

dag = DAG(dag_id='Phase1_dag_v1', default_args=args, schedule_interval=None)
with dag:
test= BashOperator(
task_id='test',
bash_command='bash -i /home/lnxuser/airflow/files/test_ns/env.sh'
)
test

这是来自气流的错误日志:

[2021-02-04 19:21:08,127] {taskinstance.py:1396} ERROR - bash -i /home/lnxuser/airflow/files/pam_ns/env.sh
Traceback (most recent call last):
File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1086, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1224, in _prepare_and_execute_task_with_callbacks
self.render_templates(context=context)
File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1684, in render_templates
self.task.render_template_fields(context)
File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 857, in render_template_fields
self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 870, in _do_render_template_fields
rendered_content = self.render_template(content, context, jinja_env, seen_oids)
File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 905, in render_template
return jinja_env.get_template(content).render(**context)
File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 187, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: bash -i /home/lnxuser/airflow/files/pam_ns/env.sh
[2021-02-04 19:21:08,169] {taskinstance.py:1433} INFO - Marking task as FAILED. dag_id=Phase1_dag_v1, task_id=PAM, execution_date=20210204T192105, start_date=20210204T192107, end_date=20210204T192108
[2021-02-04 19:21:08,382] {local_task_job.py:118} INFO - Task exited with return code 1

我是一个新手，所以不确定哪里出了问题，我可能用完全错误的方式使用气流。

有人知道这是怎么回事吗?

Answer 1

您需要在脚本名称后添加空格，因为您没有使用模板，并且脚本不在dag文件的相对路径中。

bash_command='bash -i /home/lnxuser/airflow/files/test_ns/env.sh '

需要空格的解释可以在docs

中找到。