在docker容器中有气流的情况下发送失败/重试电子邮件

Question

我正在尝试使用LocalExecutor.yml文件运行puckel airflow docker容器，该文件位于以下位置：

https://github.com/puckel/docker-airflow

我无法让气流在失败或重试时向我发送电子邮件。

我试过以下几种：

1. 使用smtp主机名编辑配置文件

[smtp]
# If you want airflow to send emails on retries, failure, and you want to use
# the airflow.utils.email.send_email_smtp function, you have to configure an
# smtp server here
smtp_host = [e-mail]
smtp_starttls = True
smtp_ssl = False
# Uncomment and set the user/pass settings if you want to use SMTP AUTH
# smtp_user = airflow
# smtp_password = airflow
smtp_port = 25
smtp_mail_from = [e-mail]

2. 在repo中包含的entrypoint.sh脚本中编辑环境变量：

: "${AIRFLOW__SMTP__SMTP_HOST:="smtp-host"}"
: "${AIRFLOW__SMTP__SMTP_PORT:="25"}"
# Defaults and back-compat
: "${AIRFLOW_HOME:="/usr/local/airflow"}"
: "${AIRFLOW__CORE__FERNET_KEY:=${FERNET_KEY:=$(python -c "from cryptography.fernet import Fernet; FERNET_KEY = Fernet.generate_key().decode(); print(FERNET_KEY)")}}"
: "${AIRFLOW__CORE__EXECUTOR:=${EXECUTOR:-Sequential}Executor}"

export 
AIRFLOW_HOME 
AIRFLOW__CELERY__BROKER_URL 
AIRFLOW__CELERY__RESULT_BACKEND 
AIRFLOW__CORE__EXECUTOR 
AIRFLOW__CORE__FERNET_KEY 
AIRFLOW__CORE__LOAD_EXAMPLES 
AIRFLOW__CORE__SQL_ALCHEMY_CONN 
AIRFLOW__SMTP__SMTP_HOST 
AIRFLOW__SMTP__SMTP_PORT 
if [ "$AIRFLOW__SMTP__SMTP_HOST" != "smtp-host" ]; then
AIRFLOW__SMTP__SMTP_HOST="smtp-host"
AIRFLOW__SMTP__SMTP_PORT=25
fi

我目前有一个dag运行故意失败，但我从未收到重试或失败的警报。

Answer 1

您不需要执行步骤1和2。在docker-compose.yml中指定环境变量。此外，请确保您已在Operator中启用失败时发送电子邮件，就像在这里：https://gist.githubusercontent.com/aawgit/0753b3ef1d715257e442ceafbc8583d3/raw/fa45caa9150e08654d476c2d619ab50615942b46/email-notification-on-task-failure.py

Answer 2

设置以下变量以使邮件功能正常工作。这取决于SMTP邮件服务器的配置。应正确设置用户名和密码。

• 如果使用docker-compose，在docker-compase.yml中设置以下变量将自动更新气流变量：

- AIRFLOW__SMTP__SMTP_HOST=smtp_host
- AIRFLOW__SMTP__SMTP_PORT=25_?_the_smtp_port_used_smtp_host
- AIRFLOW__SMTP__SMTP_USER=mail_user_name_of_your_app
- AIRFLOW__SMTP__SMTP_PASSWORD=mail_user_password_of_your_app
- AIRFLOW__SMTP__SMTP_MAIL_FROM=[e-mail]

• 类似地，如果手动打开docker，则将以下变量设置为环境变量：

AIRFLOW__SMTP__SMTP_HOST=smtp_host
AIRFLOW__SMTP__SMTP_PORT=25_?_the_smtp_port_used_smtp_host
AIRFLOW__SMTP__SMTP_USER=mail_user_name_of_your_app
AIRFLOW__SMTP__SMTP_PASSWORD=mail_user_password_of_your_app
AIRFLOW__SMTP__SMTP_MAIL_FROM=[e-mail]
export AIRFLOW__SMTP__SMTP_HOST AIRFLOW__SMTP__SMTP_PORT AIRFLOW__SMTP__SMTP_USER AIRFLOW__SMTP__SMTP_PASSWORD AIRFLOW__SMTP__SMTP_MAIL_FROM
# Then bring up docker:
docker run your_docker...