Heroku + Celery: Critical Worker Timeout (503)

Question

我在Heroku上部署了一个Flask应用程序，我使用芹菜+Redis进行异步处理。当我发送请求时，测试页面没有加载，并且我收到这个错误:

2021-10-17T13:32:59.278920+00:00 heroku[router]: at=error code=H12 desc="Request timeout" method=GET path="/test_route" host=www.mysite.dev request_id=f387a680-dca6-4e20-8f97-7f4d4a45d17b fwd="73.252.142.129,108.162.215.101" dyno=web.1 connect=0ms service=30000ms status=503 bytes=0 protocol=https
2021-10-17T13:32:59.776324+00:00 app[web.1]: [2021-10-17 13:32:59 +0000] [19] [CRITICAL] WORKER TIMEOUT (pid:23)
2021-10-17T13:32:59.778326+00:00 app[web.1]: [2021-10-17 13:32:59 +0000] [23] [INFO] Worker exiting (pid: 23)

我的测试路线:

@main_bp.route('/test_route')
def test_route():
celery_tasks.test_task.apply_async()
return render_template('main/test.html')

芹菜的任务:

@celery_app.task()
def test_task():
print('Test task succeeded')
return 1

在任务被触发之前，worker看起来很好:

2021-10-17T13:54:28.080039+00:00 app[worker.1]: [2021-10-17 13:54:28,079: INFO/MainProcess] mingle: all alone
2021-10-17T13:54:28.080111+00:00 app[worker.1]: [2021-10-17 13:54:28,080: DEBUG/MainProcess] ^-- substep ok
2021-10-17T13:54:28.080178+00:00 app[worker.1]: [2021-10-17 13:54:28,080: DEBUG/MainProcess] | Consumer: Starting Tasks
2021-10-17T13:54:28.086858+00:00 app[worker.1]: [2021-10-17 13:54:28,086: DEBUG/MainProcess] ^-- substep ok
2021-10-17T13:54:28.086956+00:00 app[worker.1]: [2021-10-17 13:54:28,086: DEBUG/MainProcess] | Consumer: Starting Control
2021-10-17T13:54:28.094411+00:00 app[worker.1]: [2021-10-17 13:54:28,094: DEBUG/MainProcess] ^-- substep ok
2021-10-17T13:54:28.094481+00:00 app[worker.1]: [2021-10-17 13:54:28,094: DEBUG/MainProcess] | Consumer: Starting Gossip
2021-10-17T13:54:28.100571+00:00 app[worker.1]: [2021-10-17 13:54:28,100: DEBUG/MainProcess] ^-- substep ok
2021-10-17T13:54:28.100633+00:00 app[worker.1]: [2021-10-17 13:54:28,100: DEBUG/MainProcess] | Consumer: Starting event loop
2021-10-17T13:54:28.100745+00:00 app[worker.1]: [2021-10-17 13:54:28,100: DEBUG/MainProcess] | Worker: Hub.register Pool...
2021-10-17T13:54:28.101184+00:00 app[worker.1]: [2021-10-17 13:54:28,101: INFO/MainProcess] celery@82da4ee3-bdcc-46c6-b65b-d6221ba1693e ready.
2021-10-17T13:54:28.101315+00:00 app[worker.1]: [2021-10-17 13:54:28,101: DEBUG/MainProcess] basic.qos: prefetch_count->32

• 我从路由中删除了芹菜任务(.apply_async)，并确认没有它的页面加载
• 在本地，我将time.sleep(30)添加到芹菜任务中，以确认任务卸载到worker，并且在任务仍在运行时加载页面。

我不确定为什么页面加载在生产中会停滞，因为异步处理的目的是不阻塞请求。我也不确定如何更深入地挖掘和调试这个错误-在文档中没有找到太多。

我在文档:调试请求超时中发现了这个，但它只指向长时间运行的任务和无限循环，而这里的情况都不是

Answer 1

这可以通过更改celery_tasks.py中使用的芹菜应用程序来解决。

这行不通:

#celery_tasks.py
from celery import Celery
celery_app = Celery()
@celery_app.task()
def test_task():
...

如此:

#celery_tasks.py
from extensions import celery
celery_app = celery
@celery_app.task()
def test_task():
...

地点:

#extensions.py
import flask
from celery import Celery
class FlaskCelery(Celery):
def __init__(self, *args, **kwargs):
super(FlaskCelery, self).__init__(*args, **kwargs)
self.patch_task()
if 'app' in kwargs:
self.init_app(kwargs['app'])
def patch_task(self):
TaskBase = self.Task
_celery = self
class ContextTask(TaskBase):
abstract = True
def __call__(self, *args, **kwargs):
if flask.has_app_context():
return TaskBase.__call__(self, *args, **kwargs)
else:
with _celery.app.app_context():
return TaskBase.__call__(self, *args, **kwargs)
self.Task = ContextTask
def init_app(self, app):
self.app = app
self.config_from_object(app.config)

celery = FlaskCelery()

和__init__.py使用相同的类