芹菜每个周期都会发送任务，但仅接收和执行每隔一个时期

Question

我将周期设置为1分钟。芹菜确实每分钟按预期发送任务，但是每隔一分钟就接收并执行任务。

[2016-11-24 15:47:48,653: INFO/MainProcess] mingle: searching for neighbors
[2016-11-24 15:47:49,669: INFO/MainProcess] mingle: sync with 1 nodes
[2016-11-24 15:47:49,670: INFO/MainProcess] mingle: sync complete
[2016-11-24 15:47:49,691: WARNING/MainProcess] celery@testTask ready.
[2016-11-24 15:48:00,011: INFO/Beat] Scheduler: Sending due task every-minute (tasks.download_reports)
[2016-11-24 15:49:00,053: INFO/Beat] Scheduler: Sending due task every-minute (tasks.download_reports)
[2016-11-24 15:49:00,057: INFO/MainProcess] Received task: tasks.download_reports[a927e0a5-8c45-4a2b-b05f-ab1e19156ada]
[2016-11-24 15:49:00,061: WARNING/Worker-17] this is a task
[2016-11-24 15:49:00,115: INFO/MainProcess] Task tasks.download_reports[a927e0a5-8c45-4a2b-b05f-ab1e19156ada] succeeded in 0.0551409143955s: None
[2016-11-24 15:50:00,053: INFO/Beat] Scheduler: Sending due task every-minute (tasks.download_reports)
[2016-11-24 15:51:00,076: INFO/Beat] Scheduler: Sending due task every-minute (tasks.download_reports)
[2016-11-24 15:51:00,079: INFO/MainProcess] Received task: tasks.download_reports[0015bc12-4452-486e-98b8-b8b61f1eb56c]
[2016-11-24 15:51:00,081: WARNING/Worker-10] this is a task
[2016-11-24 15:51:00,127: INFO/MainProcess] Task tasks.download_reports[0015bc12-4452-486e-98b8-b8b61f1eb56c] succeeded in 0.0465096402913s: None
... ...

这是我的芹菜文件：

# CELERY
BROKER_URL = 'amqp://guest@localhost//'
CELERY_RESULT_BACKEND = 'amqp'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'US/Eastern'
from celery.schedules import crontab
CELERYBEAT_SCHEDULE = {
    'every-minute': {
        'task': 'tasks.download_reports',
        'schedule': crontab(minute='*/1'),
        'args': (),
    },
}

这是任务文件：

from celery import Celery
celery = Celery('tasks')
celery.config_from_object('celeryconfig')
@celery.task
def download_reports():
  print 'this is a task'

Answer 1

我发现问题是我同时跑了两个芹菜工人。因此，我在消息队列中失去了50％的工作。

我通过使用一个芹菜工人来运行两个任务解决了这一点。

芹菜饼文件：

CELERYBEAT_SCHEDULE = {
    'every-minute': {
        'task': 'tasks.download_reports',
        'schedule': crontab(minute='*/1'),
        'args': (),
    },
    'every-5-minute': {
        'task': 'tasks.download_another_report',
        'schedule': crontab(minute='*/5'),
        'args': (),         
    }
}

任务文件

@celery.task
def download_reports():
  print 'this is a task'
@celery.task
def download_another_report():
  print 'this is another task'