在一个BigQueryInsertJobOperator气流中包含多个SQL查询

Question

语法是这样的，

configuration={
"query": {
"query": "{% include ['query_2.sql'] %}",
"useLegacySql": False,
}

我有什么办法可以做到吗?

"query": "{% include ['query_2.sql', 'query_3.sql', 'query_4.sql'] %}"

Answer 1

configuration={

"query": {
"query": "{% for f in ['query_1.sql','query_2.sql'] %}"
"{% include f %}"
";"
"{% endfor %}",
"useLegacySql": False }

Answer 2

您可以创建一个存储过程，列出所需的所有查询，然后像这样调用它:

t1 = BigQueryInsertJobOperator(
task_id="exec_sp",
project_id=PROJECT_ID,
configuration={
"query": {
"query": f"CALL `{PROJECT_ID}.{DATASET}.stored_procedure_name` ();", 
"useLegacySql": False,
"timeoutMs": 3600000,
}
}
)