我在/var/tasks/tasks.py
中定义了任务(用于Celery)。
我在/var/tasks/venv
有一个virtualenv,它应该用于运行/var/tasks/tasks.py
。
我可以手动启动一个工人来处理这样的任务:
cd /var/tasks
. venv/bin/activate
celery worker -A tasks -Q queue_1
现在,我想守护它。
我从GitHub复制了init.d
脚本,并在/etc/default/celeryd
:中使用以下配置文件
# name(s) of nodes to start
CELERYD_NODES="worker1"
# absolute or relative path to celery binary
CELERY_BIN="/var/tasks/venv/bin/celery"
# app instance
CELERY_APP="tasks"
# change to directory on upstart
CELERYD_CHDIR="/var/tasks"
# options
CELERYD_OPTS="-Q queue_1 --concurrency=8"
# %N will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERYD_PID_FILE="/var/run/celery/%N.pid"
# unprivileged user/group
CELERYD_USER="celery"
CELERYD_GROUP="celery"
# create pid and log directories, if missing
CELERY_CREATE_DIRS=1
当我启动服务(通过init.d
脚本)时,它会显示:
celery init v10.1.
Using config script: /etc/default/celeryd
但是,它不处理队列中的任何任务,日志文件中也没有任何内容。
我做错了什么?
Supervisor可能是一个不错的选择,但如果你想使用Celery Init.d,Script会建议你从他们的Github Source复制它。
sudo vim /etc/init.d/celeryd
从复制代码https://github.com/celery/celery/blob/master/extra/generic-init.d/celeryd中的。有关详细信息,请参阅守护进程教程。
sudo chmod 755 /etc/init.d/celeryd
sudo chown root:root /etc/init.d/celeryd
sudo nano /etc/default/celeryd
复制粘贴以下配置并相应更改
#Where your Celery is present
CELERY_BIN="/home/shivam/Desktop/deploy/bin/celery"
# App instance to use
CELERY_APP="app.celery"
# Where to chdir at start
CELERYD_CHDIR="/home/shivam/Desktop/Project/demo/"
# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"
# %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
# Workers should run as an unprivileged user.
# You need to create this user manually (or you can choose
# A user/group combination that already exists (e.g., nobody).
CELERYD_USER="shivam"
CELERYD_GROUP="shivam"
# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1
export SECRET_KEY="foobar"
保存并退出
sudo /etc/init.d/celeryd start
sudo /etc/init.d/celeryd status
这将在引导上自动启动Celery
sudo update-rc.d celeryd defaults
如果您使用systemd,您应该启用芹菜服务。它将在启动时激活您的芹菜守护程序。
sudo systemctl enable yourcelery.service
我最终使用了Supervisor和/etc/Supervisor/conf.d/selery.conf中的脚本,类似于以下内容:
https://github.com/celery/celery/blob/3.1/extra/supervisord/celeryd.conf
这可以很好地自动处理妖魔化。