Django芹菜周期任务仅运行一次

Question

我正在尝试安排使用Django 1.9.8，芹菜4.0.2，RabbitMQ 2.1.4，Redis 2.10.5每10分钟运行一次的任务。这些都在Linux（Fedora 25）的Docker容器中运行。我尝试了许多在芹菜文档和此网站中发现的东西的组合。迄今为止唯一起作用的组合在下面。但是，它仅在应用程序启动时最初运行周期任务，但此后忽略了时间表。我绝对确认计划的任务在初始时间之后不会再次运行。

我的（几乎是在工作）设置仅运行一次：

settings.py：

INSTALLED_APPS = (
   ...
   'django_celery_beat',
   ...
)
BROKER_URL = 'amqp://{user}:{password}@{hostname}/{vhost}/'.format(
    user=os.environ['RABBIT_USER'],
    password=os.environ['RABBIT_PASS'],
    hostname=RABBIT_HOSTNAME,
    vhost=os.environ.get('RABBIT_ENV_VHOST', '')
# We don't want to have dead connections stored on rabbitmq, so we have to negotiate using heartbeats
BROKER_HEARTBEAT = '?heartbeat=30'
if not BROKER_URL.endswith(BROKER_HEARTBEAT):
    BROKER_URL += BROKER_HEARTBEAT
BROKER_POOL_LIMIT = 1
BROKER_CONNECTION_TIMEOUT = 10
# Celery configuration
# configure queues, currently we have only one
CELERY_DEFAULT_QUEUE = 'default'
CELERY_QUEUES = (
    Queue('default', Exchange('default'), routing_key='default'),
)
# Sensible settings for celery
CELERY_ALWAYS_EAGER = False
CELERY_ACKS_LATE = True
CELERY_TASK_PUBLISH_RETRY = True
CELERY_DISABLE_RATE_LIMITS = False
# By default we will ignore result
# If you want to see results and try out tasks interactively, change it to False
# Or change this setting on tasks level
CELERY_IGNORE_RESULT = True
CELERY_SEND_TASK_ERROR_EMAILS = False
CELERY_TASK_RESULT_EXPIRES = 600
# Set redis as celery result backend
CELERY_RESULT_BACKEND = 'redis://%s:%d/%d' % (REDIS_HOST, REDIS_PORT, REDIS_DB)
CELERY_REDIS_MAX_CONNECTIONS = 1
# Don't use pickle as serializer, json is much safer
CELERY_TASK_SERIALIZER = "json"
CELERY_RESULT_SERIALIZER = "json"
CELERY_ACCEPT_CONTENT = ['application/json']
CELERYD_HIJACK_ROOT_LOGGER = False
CELERYD_PREFETCH_MULTIPLIER = 1
CELERYD_MAX_TASKS_PER_CHILD = 1000

芹菜conf.py

coding=UTF8
from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "web_portal.settings")
app = Celery('web_portal')
CELERY_TIMEZONE = 'UTC'
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

tasks.py

from celery.schedules import crontab
from .celeryconf import app as celery_app
@celery_app.on_after_finalize.connect
def setup_periodic_tasks(sender, **kwargs):
    # Calls email_scanner every 10 minutes
    sender.add_periodic_task(
        crontab(hour='*', 
                minute='*/10', 
                second='*', 
                day_of_week='*',  
                day_of_month='*'),
        email_scanner.delay(),
    )
@app.task
def email_scanner():
    dispatch_list = scanning.email_scan()
    for dispatch in dispatch_list:
        validate_dispatch.delay(dispatch)
    return

run_celery.sh-用于从docker-compose.yml

启动芹菜任务

#!/bin/sh
# wait for RabbitMQ server to start
sleep 10
cd web_portal
# run Celery worker for our project myproject with Celery configuration stored in Celeryconf
su -m myuser -c "celery beat -l info --pidfile=/tmp/celerybeat-web_portal.pid -s /tmp/celerybeat-schedule &"
su -m myuser -c "celery worker -A web_portal.celeryconf -Q default -n default@%h"

---

我还尝试在设置中使用celerybeat_scheduler来代替 @celery_app.onp.on_after finalize_connect decorator和blocks.pys.py，但是调度程序甚至从未运行过一次。

settings.py（在所有情况下都不起作用）

（与以下内容相同，除了以下内容）

CELERYBEAT_SCHEDULE = {
    'email-scanner-every-5-minutes': {
        'task': 'tasks.email_scanner',
        'schedule': timedelta(minutes=10)
    },
}

芹菜4.0.2在线文档假设我本能地知道很多givens，但是在这种环境下我是新的。如果有人知道我可以在哪里找到docs.celeryproject.org和http://django-celery-beat.readthedocs.io/en/latest/，这两个都假设我已经是Django大师，我会很高兴。或当然让我知道，如果您在我的设置中看到明显错误的事情。谢谢！

Answer 1

我找到了一个有效的解决方案。我无法让Celerybeat_schedule或芹菜任务装饰器工作，我怀疑这可能至少部分应与我启动芹菜节拍任务的方式有关。

工作解决方案都使用了整个9码，以利用Django数据库调度程序。我下载了GitHub项目" https://github.com/celery/django-celery-beat"，并将所有代码纳入我的项目中的另一个"应用程序"。这启用了Django-Admin访问，以通过浏览器维护CRON/INTERTAL/周期任务。我还修改了我的run_celery.sh，如下所示：

#!/bin/sh
# wait for RabbitMQ server to start
sleep 10
# run Celery worker for our project myproject with Celery configuration stored in Celeryconf
celery beat -A web_portal.celeryconf -l info --pidfile=/tmp/celerybeat- web_portal.pid -S django --detach
su -m myuser -c "celery worker -A web_portal.celeryconf -Q default -n default@%h -l info "

通过Django-Admin Web界面添加计划任务后，调度程序开始工作正常。