烧瓶 芹菜作为守护程序

Question

我尝试倾斜python烧瓶并想要使用芹菜。分布式任务正常运行，但是现在我想按照芹菜文档中解释的守护程序进行配置。但是我得到celery_worker_1 exited with code 0错误。

项目结构：

celery
|-- flask-app
|   `-- app.py
|-- worker
|   |-- celeryd
|   |-- celeryd.conf
|   |-- Dockerfile
|   |-- start.sh
|   `-- tasks.py
`-- docker-compose.yml

flask-app/app.py：

from flask import Flask
from flask_restful import Api, Resource
from celery import Celery
celery = Celery(
                'tasks',
                broker='redis://redis:6379',
                backend='redis://redis:6379'
)
app = Flask(__name__)
api = Api(app)
class add_zahl(Resource):
    def get(self):
        zahl = 54
        task = celery.send_task('mytasks.add', args=[zahl])
        return {'message': f"Prozess {task.id} gestartet, input {zahl}"}, 200
api.add_resource(add_zahl, "/add")
if __name__ == '__main__':
    app.run(host="0.0.0.0", debug=True)

工人：tasks.py

from celery import Celery
import requests
import time
import os
from dotenv import load_dotenv
basedir = os.path.abspath(os.path.dirname(__file__))
load_dotenv(os.path.join(basedir, '.env'))
celery = Celery(
                'tasks',
                broker='redis://redis:6379',
                backend='redis://redis:6379'
)
@celery.task(name='mytasks.add')
def send_simple_message(zahl):
    time.sleep(5)
    result = zahl * zahl
    return result
if __name__ == '__main__':
    celery.start()

dockerfile：

FROM python:3.6-slim
RUN mkdir /worker
COPY requirements.txt /worker/
RUN pip install --no-cache-dir -r /worker/requirements.txt
COPY . /worker/
COPY celeryd /etc/init.d/celeryd
RUN chmod +x /etc/init.d/celeryd
COPY celeryd.conf /etc/default/celeryd
RUN chown root:root /etc/default/celeryd
RUN useradd -N -M --system -s /bin/bash celery
RUN addgroup celery
RUN adduser celery celery
RUN mkdir -p /var/run/celery
RUN mkdir -p /var/log/celery
RUN chown -R celery:celery /var/run/celery
RUN chown -R celery:celery /var/log/celery
RUN chmod u+x /worker/start.sh
ENTRYPOINT /worker/start.sh

Celeryd.conf：

CELERYD_NODES="worker1"
CELERY_BIN="/worker/tasks"
CELERY_APP="worker.tasks:celery"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_USER="celery"
CELERYD_GROUP="celery"
CELERY_CREATE_DIRS=1

start.sh

#!/bin/sh
exec celery multi start worker1 -A worker --app=worker.tasks:celery 

芹菜：https://github.com/celery/celery/blob/3.1/extra/generic-init.d/celeryd

Docker检查日志：

Docker inspect 50fbe00fdc5de56dafaf4268f24baed3b47c8519a689f0733e41ec7fdbc86765
[
    {
        "Id": "50fbe00fdc5de56dafaf4268f24baed3b47c8519a689f0733e41ec7fdbc86765",
        "Created": "2019-02-21T23:20:15.017156266Z",
        "Path": "/bin/sh",
        "Args": [
            "-c",
            "/worker/start.sh"
        ],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2019-02-21T23:20:40.375566345Z",
            "FinishedAt": "2019-02-21T23:20:41.162618701Z"
        },

对不起"垃圾邮件"，但我无法解决这个问题。

edit edit edit

我添加了提到的CMD线，现在工人没有启动。我正在努力寻找解决方案。有提示吗？谢谢大家。

FROM python:3.6-slim
RUN mkdir /worker
COPY requirements.txt /worker/
RUN pip install --no-cache-dir -r /worker/requirements.txt
COPY . /worker/
COPY celeryd /etc/init.d/celeryd
RUN chmod +x /etc/init.d/celeryd
COPY celeryd.conf /etc/default/celeryd
RUN chown -R root:root /etc/default/celeryd
RUN useradd -N -M --system -s /bin/bash celery
RUN addgroup celery
RUN adduser celery celery
RUN mkdir -p /var/run/celery
RUN mkdir -p /var/log/celery
RUN chown -R celery:celery /var/run/celery
RUN chown -R celery:celery /var/log/celery
CMD ["celery", "worker", "--app=worker.tasks:celery"]

Answer 1

每当docker容器的入口点退出时（或者，如果没有入口点，则是其主命令），则会出现容器。推论的是，容器中的主过程不能是像celery multi这样的命令，它会产生一些背景工作并立即返回。您需要使用在前景中运行的celery worker之类的命令。

我可能会用：

替换您的Dockerfile中的最后几行

CMD ["celery", "worker", "--app=worker.tasks:celery"]

保留入口点脚本并将其更改为等效的前景celery worker命令也应该执行工作。

Answer 2

您也可以使用Substisord来运行芹菜工人。奖金是主管也将监视并重新启动您的工人，如果有些事情发生了。以下是我为您的情况提取的工作图像的示例...

文件sustecord.conf

[supervisord]
nodaemon=true
[program:celery]
command=celery worker -A proj --loglevel=INFO
directory=/path/to/project
user=nobody
numprocs=1
stdout_logfile=/var/log/celery/worker.log
stderr_logfile=/var/log/celery/worker.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600
stopasgroup=true
priority=1000

文件start.sh

#!/bin/bash
set -e
exec /usr/bin/supervisord -c /etc/supervisor/supervisord.conf

文件Dockerfile

# Your other Dockerfile content here
ENTRYPOINT ["/entrypoint.sh"]
CMD ["/start.sh"]