气流"This connection is closed"运行中，但未在测试中

Question

我使用的是气流1.7.1.3和python 2.7

我创建了一个DAG，当我使用

分别运行每个任务时，它可以完美地工作。

气流测试[myDAG] [myTask] 2016-10-14

然而,

气流trigger_dag [myDAG]

或

气流运行[myDAG] [myTask] 2016-10-14

都抛出"This connection is closed" SQLalchemy错误。

[...]
    with self.engine.connect() as connection:
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2016, in connect
    return self._connection_cls(self, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 72, in __init__
    if connection is not None else engine.raw_connection()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2102, in raw_connection
    self.pool.unique_connection, _connection)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2072, in _wrap_pool_connect
    return fn()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 318, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 746, in _checkout
    raise exc.InvalidRequestError("This connection is closed")
InvalidRequestError: This connection is closed
[2016-10-14 15:49:30,704] {models.py:1306} INFO - Marking task as FAILED.
[2016-10-14 15:49:30,712] {models.py:1327} ERROR - This connection is closed

这是通过SQLalchemy连接到Oracle 12数据库，当我在脚本中使用session.commit()时，会引发这个错误。

有人知道是什么解释了这种差异和错误吗?

Answer 1

这是一个已知的bug。在此错误修复之前，您将无法通过SQLAlchemy连接到Oracle。

这个问题是由于源代码中的一些SQL语法。你不能在Oracle中说"SELECT 1"，你需要说"SELECT 1 FROM DUAL"。

也许可以考虑使用气流中的一个挂钩:https://github.com/apache/incubator-airflow/tree/master/airflow/hooks

似乎有一个oracle_hook可以帮助你。好运。