芹菜连接错误

Question

我正在部署一个带有芹菜的Django应用程序到Heroku，并且正在努力解决连接错误。我的AMQP提供者声称他能够访问他端的资源，并且我有足够的连接。我认为我的设置是在我的特定工作进程中找到芹菜应用程序，但未能在shell中获得适当的设置。是否有一种方法可以告诉调用任务的连接URL是什么?我的celery_app检查下面处理事情正确吗?

调用

问题:

$ heroku run bash
~$ python <app>/manage.py shell
>>> from <app>.management.tasks.tasks import <task>
>>> t = <task>()
>>> dt = '20140101'
>>> t.delay(dt=dt)

异常堆栈

:

Traceback (most recent call last):
  File "<input>", line 4, in <module>
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/task.py", line 453, in delay
    return self.apply_async(args, kwargs)
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/task.py", line 555, in apply_async
    **dict(self._get_exec_options(), **options)
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/base.py", line 353, in send_task
    reply_to=reply_to or self.oid, **options
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/amqp.py", line 305, in publish_task
    **kwargs
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/messaging.py", line 168, in publish
    routing_key, mandatory, immediate, exchange, declare)
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 457, in _ensured
    interval_max)
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 369, in ensure_connection
    interval_start, interval_step, interval_max, callback)
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/utils/__init__.py", line 243, in retry_over_time
    return fun(*args, **kwargs)
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 237, in connect
    return self.connection
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 741, in connection
    self._connection = self._establish_connection()
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 696, in _establish_connection
    conn = self.transport.establish_connection()
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 112, in establish_connection
    conn = self.Connection(**opts)
  File "/app/.heroku/python/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
    self.transport = create_transport(host, connect_timeout, ssl)
  File "/app/.heroku/python/lib/python2.7/site-packages/amqp/transport.py", line 294, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/app/.heroku/python/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
    raise socket.error(last_err)
error: [Errno 111] Connection refused

检查shell配置:

~$ python <app>/manage.py shell
>>> from celery_app import Config
>>> Config.BROKER_URL
'<correct amqp resource>'

Env var:

CELERYD_CONCURRENCY=1
CELERY_IGNORE_RESULT=True
CELERYD_TASK_TIME_LIMIT=60
CLOUDAMQP_URL=<amqp url>
RESULT_EXPIRY_RATE=600
BROKER_CONNECTION_TIMEOUT=10
PWD=/app
DJANGO_SETTINGS_MODULE=settings.production
DJANGO_PROJECT_DIR=/app/<app>
BROKER_POOL_LIMIT=1
HOME=/app
PYTHONPATH=/app:/app/<app>:/app/<app>/<app>

Procfile调用(works)

web: gunicorn <app>.<app>.wsgi -w 1 --log-file -
worker: celery worker --app=<app>.<app> -E -Q <app>,celery --loglevel=INFO  -c 1  --workdir=<app>

我的芹菜应用程序:

from __future__ import absolute_import
from os import getenv
from kombu import Exchange, Queue
from django.conf import settings
from celery import Celery

app = Celery('<app>')
class Config(object):
    # List of modules to import when celery starts.
    CELERY_IMPORTS = ("<imports>",)

    BROKER_CONNECTION_RETRY = True
    API_RATE_LIMIT = getenv('API_RATE_LIMIT')
    BROKER_POOL_LIMIT = int(getenv('BROKER_POOL_LIMIT', 1))
    BROKER_URL = getenv('CLOUDAMQP_URL')
    BROKER_CONNECTION_TIMEOUT = int(getenv('BROKER_CONNECTION_TIMEOUT'))
    CELERYD_CONCURRENCY = int(getenv('CELERYD_CONCURRENCY'))

app.config_from_object(Config)
if __name__ == '__main__':
    app.start()

在同一目录下导入init.py中的应用程序:

from __future__ import absolute_import
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery_app import app as celery_app  # nolint

Answer 1

核心问题是django、芹菜和gunicorn对DJANGO_SETTINGS_MODULE的期望格式不同。我通过将DJANGO_SETTINGS_MODULE=settings.production更改为DJANGO_SETTINGS_MODULE=<app>.settings.production来解决这个问题，这修复了shell代理连接，但破坏了我的web和工作进程。一个有效的Procfile规范是

web: cd <app> && gunicorn <app>.wsgi -w 1 --log-file -
worker: celery worker --app=<app> -E -Q <app>,celery --loglevel=INFO  -c 1  --workdir=<app>

Answer 2

当我将Django项目部署到Tornado Server时，我遇到了这个异常。下面是部署代码:

import os
import tornado.httpserver
import tornado.ioloop
import tornado.wsgi
from django.core.wsgi import get_wsgi_application
# add this line when use celery.
# import app
def main():
    os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings' # path to your settings module
    application = get_wsgi_application()
    container = tornado.wsgi.WSGIContainer(application)
    http_server = tornado.httpserver.HTTPServer(container)
    http_server.listen(8888)
    tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
    main()

当我添加import app(包含芹菜app init文件的应用程序)时，一切都运行良好。