我正在 Elastic Beanstalk 上设置celeryd和 redis 作为守护程序，根据 如何使用 AWS Elastic Beanstalk 运行工作线程？。
我想我快到了。Redis 在 elasticcache 上运行并可访问。
但是要设置celeryd的.ebextesions文件在特定行上失败。请参阅下面我正在运行的整个脚本.ebextensions。
线：command=/opt/python/run/venv/bin/celery worker -A myappname --loglevel=INFO.
我已经尝试了该命令的几种不同变体。
试：
-
command=/opt/python/run/venv/bin/celery worker -A /opt/python/current/app/app.py --loglevel=INFO(我的 app.py 目录) -
command=/opt/python/run/venv/bin/celery worker -A app.py --loglevel=INFO(我 app.py 的名字) -
command=/opt/python/run/venv/bin/celery worker -A app --loglevel=INFO(不带扩展名的名称) -
command=/opt/python/run/venv/bin/celery worker -A flask-celery-01 --loglevel=INFO(弹性豆茎中的应用名称) -
command=/opt/python/run/venv/bin/celery worker -A myappname --loglevel=INFO(原始未编辑)
芹菜日志中的回溯：
Traceback (most recent call last):
File "/opt/python/run/venv/bin/celery", line 11, in <module>
sys.exit(main())
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/__main__.py", line 30, in main
main()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/celery.py", line 81, in main
cmd.execute_from_commandline(argv)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/base.py", line 305, in execute_from_commandline
argv = self.setup_app_from_commandline(argv)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/base.py", line 465, in setup_app_from_commandline
self.app = self.find_app(app)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/base.py", line 485, in find_app
return find_app(app, symbol_by_name=self.symbol_by_name)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/app/utils.py", line 229, in find_app
sym = symbol_by_name(app, imp=imp)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/base.py", line 488, in symbol_by_name
return symbol_by_name(name, imp=imp)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/kombu/utils/__init__.py", line 96, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/utils/imports.py", line 101, in import_from_cwd
return imp(module, package=package)
File "/opt/python/run/venv/lib64/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 948, in _find_and_load_unlocked
ModuleNotFoundError: No module named '/opt/python/current/app/app'
.ebextensions script：
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_celeryd.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
# Get django environment variables
celeryenv=`cat /opt/python/current/env | tr 'n' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g'`
celeryenv=${celeryenv%?}
# Create celery configuraiton script
celeryconf="[program:celeryd]
; Set full path to celery program if using virtualenv
command=/opt/python/run/venv/bin/celery worker -A /opt/python/current/app/app.py --loglevel=INFO
directory=/opt/python/current/app
user=nobody
numprocs=1
stdout_logfile=/var/log/celery-worker.log
stderr_logfile=/var/log/celery-worker.log
autostart=true
autorestart=true
startsecs=10
; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
; When resorting to send SIGKILL to the program to terminate it
; send SIGKILL to its whole process group instead,
; taking care of its children as well.
killasgroup=true
; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=998
environment=$celeryenv"
# Create the celery supervisord conf script
echo "$celeryconf" | tee /opt/python/etc/celery.conf
# Add configuration script to supervisord conf (if not there already)
if ! grep -Fxq "[include]" /opt/python/etc/supervisord.conf
then
echo "[include]" | tee -a /opt/python/etc/supervisord.conf
echo "files: celery.conf" | tee -a /opt/python/etc/supervisord.conf
fi
# Reread the supervisord config
supervisorctl -c /opt/python/etc/supervisord.conf reread
# Update supervisord in cache without restarting all services
supervisorctl -c /opt/python/etc/supervisord.conf update
# Start/Restart celeryd through supervisord
supervisorctl -c /opt/python/etc/supervisord.conf restart celeryd
可能是因为我的烧瓶应用程序没有拆分为模块(一切都在app.py中)。而且我还没有像在某些教程中看到的那样"安装"该应用程序。刚刚使用 EB 部署，Web UI 可以正常工作。
Flask和AWS对我来说都是新的。并且很乐意以任何方式将其投入生产。
每条评论更新：
试：
/opt/python/run/venv/bin/celery worker -A app:app --loglevel=INFO
Traceback (most recent call last):
File "/opt/python/run/venv/bin/celery", line 11, in <module>
sys.exit(main())
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/__main__.py", line 30, in main
main()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/celery.py", line 81, in main
cmd.execute_from_commandline(argv)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/base.py", line 305, in execute_from_commandline
argv = self.setup_app_from_commandline(argv)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/base.py", line 473, in setup_app_from_commandline
user_preload = tuple(self.app.user_options['preload'] or ())
AttributeError: 'Flask' object has no attribute 'user_options'
试：
/opt/python/run/venv/bin/celery worker -A app:app.celery --loglevel=INFO
Traceback (most recent call last):
File "/opt/python/run/venv/bin/celery", line 11, in <module>
sys.exit(main())
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/__main__.py", line 30, in main
main()
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/celery.py", line 81, in main
cmd.execute_from_commandline(argv)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/base.py", line 305, in execute_from_commandline
argv = self.setup_app_from_commandline(argv)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/base.py", line 465, in setup_app_from_commandline
self.app = self.find_app(app)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/bin/base.py", line 485, in find_app
return find_app(app, symbol_by_name=self.symbol_by_name)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/app/utils.py", line 232, in find_app
sym = imp(app)
File "/opt/python/run/venv/local/lib/python3.6/site-packages/celery/utils/imports.py", line 101, in import_from_cwd
return imp(module, package=package)
File "/opt/python/run/venv/lib64/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 936, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 948, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'app:app'
我在本地运行：
celery worker -A app.celery --loglevel=info
这行得通。
app.py
import os
import random
import time
from flask import Flask, request, render_template, session, flash, redirect,
url_for, jsonify
from flask.ext.mail import Mail, Message
from celery import Celery
import psycopg2 as pg
app = Flask(__name__)
REDIS_CLUSTER = os.environ.get('CLUSTER_URL')
# celery config
app.config['CELERY_BROKER_URL'] = REDIS_CLUSTER
app.config['CELERY_RESULT_BACKEND'] = REDIS_CLUSTER
# initialize extensions
mail = Mail(app)
# initialize celery
celery = Celery(app.name, broker=app.config['CELERY_BROKER_URL'])
celery.conf.update(app.config)
DB_NAME = os.environ.get('DB_NAME')
DB_USER = os.environ.get('DB_USER')
DB_PASSWORD = os.environ.get('DB_PASSWORD')
DB_HOST = os.environ.get('DB_HOST')
@celery.task
def do_sth_to_db():
"""insert record into db"""
with app.app_context():
# local: fintrosapp_ml
conn = pg.connect(
dbname=DB_NAME,
user=DB_USER,
password=DB_PASSWORD,
host=DB_HOST
)
cursor = conn.cursor()
query = """
INSERT INTO examples(first_name, last_name)
VALUES ('First', 'Last');
"""
cursor.execute(query)
conn.commit()
@app.route('/', methods=['GET', 'POST'])
def index():
if request.method == 'GET':
return render_template('index.html', email=session.get('email', ''))
# backgrounded task
do_sth_to_db.delay()
return redirect(url_for('index'))
###############################################################################
if __name__ == '__main__':
app.run() # debug=True
你需要像下面这样使用它
celery worker -A app:celery --loglevel=info
A.B.C:D语法意味着导入A.B.C并使用同一对象的D