Django Model.objects.all() 在芹菜任务中返回空的 QuerySet

Question

project/project/settings.py

...
CELERY_BEAT_SCHEDULE = {
'find-subdomains': {
'task': 'subdiscovery.tasks.mytask',
'schedule': 10.0
}
}

项目/子发现/任务.py

from __future__ import absolute_import, unicode_literals
from celery import shared_task
from subdiscovery.models import Domain
@shared_task
def mytask():
print(Domain.objects.all())
return 99

芹菜工作线程显示一个空的查询集：

celery_worker_1  | [2019-08-12 07:07:44,229: WARNING/ForkPoolWorker-2] <QuerySet []>
celery_worker_1  | [2019-08-12 07:07:44,229: INFO/ForkPoolWorker-2] Task subdiscovery.tasks.mytask[60c59024-cd19-4ce9-ae69-782a3a81351b] succeeded in 0.004897953000181587s: 99

但是，导入相同的模型在python shell中工作：

./manage.py shell
>>> from subdiscovery.models import Domain
>>> Domain.objects.all()
<QuerySet [<Domain: example1.com>, <Domain: example2.com>, <Domain: example3.com>]>

我应该提到它在 docker 堆栈中运行

编辑：

好的，进入正在运行的 docker 容器

docker exec -it <web service container id> /bin/sh

和跑步

$ celery -A project worker -l info

按预期工作：

[2019-08-13 05:12:28,945: INFO/MainProcess] Received task: subdiscovery.tasks.mytask[7b2760cf-1e7f-41f8-bc13-fa4042eedf33]  
[2019-08-13 05:12:28,957: WARNING/ForkPoolWorker-8] <QuerySet [<Domain: uber.com>, <Domain: example1.com>, <Domain: example2.com>, <Domain: example3.com>]>

这是docker-compose.yml的样子。

version: '3'
services:
web:
build: .
image: app-image
ports:
- 80:8000
volumes:
- .:/app
command: gunicorn -b 0.0.0.0:8000 project.wsgi
redis:
image: "redis:alpine"
ports:
- 6379:6379
celery_worker:
working_dir: /app
command: sh -c './wait-for web:8000 && ./wait-for redis:6379 -- celery -A project worker -l info'
image: app-image
depends_on:
- web
- redis
celery_beat:
working_dir: /app
command: sh -c 'celery -A project beat -l info'
image: app-image
depends_on:
- celery_worker

知道为什么工作线程从 docker-compose 开始不起作用，但进入正在运行的容器并启动工作线程可以吗？

Answer 1

从reddit转贴 https://www.reddit.com/r/docker/comments/cpoedr/different_behavior_when_starting_a_celery_worker/ewqx3mp?utm_source=share&utm_medium=web2x

您的问题是您的芹菜工人看不到 sqlite 数据库。您需要切换到不同的数据库或使您的./app卷可见。

version: '3'
services:
...
celery_worker:
working_dir: /app
command: sh -c './wait-for web:8000 && ./wait-for redis:6379 -- celery -A project worker -l info'
image: app-image
volumes: # <-here
- .:/app
depends_on:
- web
- redis
...

我建议切换到更多生产就绪的数据库，例如 postgres