DockeR语言 Celery 无法连接到 redis



项目结构:

client
nginx
web/
celery_worker.py
project
config.py
api/

我的码头工人编写中有以下服务:

version: '3.6'
services:
web:
build:
context: ./services/web
dockerfile: Dockerfile-dev
volumes:
- './services/web:/usr/src/app'
ports:
- 5001:5000
environment:
- FLASK_ENV=development
- APP_SETTINGS=project.config.DevelopmentConfig
- DATABASE_URL=postgres://postgres:postgres@web-db:5432/web_dev 
- DATABASE_TEST_URL=postgres://postgres:postgres@web-db:5432/web_test
- SECRET_KEY=my_precious  
depends_on:  
- web-db
- redis
celery:
image: dev3_web
restart: always
volumes:
- ./services/web:/usr/src/app
- ./services/web/logs:/usr/src/app
command: celery worker -A celery_worker.celery --loglevel=INFO -Q cache
environment:
- CELERY_BROKER=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
depends_on:
- web
- redis
links:
- redis:redis

redis:
image: redis:5.0.3-alpine
restart: always
expose:
- '6379'
ports:
- '6379:6379'

monitor:
image: dev3_web
ports:
- 5555:5555
command:  flower -A celery_worker.celery --port=5555 --broker=redis://redis:6379/0
depends_on:
- web
- redis

web-db:  
build:
context: ./services/web/project/db
dockerfile: Dockerfile
ports:
- 5435:5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres

nginx:
build:
context: ./services/nginx
dockerfile: Dockerfile-dev
restart: always
ports:
- 80:80
- 8888:8888
depends_on:
- web
- client
- redis

client:
build:
context: ./services/client
dockerfile: Dockerfile-dev
volumes:
- './services/client:/usr/src/app'
- '/usr/src/app/node_modules'
ports:
- 3007:3000
environment:
- NODE_ENV=development
- REACT_APP_WEB_SERVICE_URL=${REACT_APP_WEB_SERVICE_URL}
depends_on:
- web
- redis

芹菜原木

但是,芹菜无法从以下日志连接:

celery_1   | [2019-03-29 03:09:32,111: ERROR/MainProcess] consumer: Cannot connect to redis://localhost:6379/0: Error 99 connecting to localhost:6379. Address not available..
celery_1   | Trying again in 2.00 seconds...

网络日志

因此,服务(运行后端)不是web,通过相同的日志:

web_1      | Waiting for postgres...
web_1      | PostgreSQL started
web_1      |  * Environment: development
web_1      |  * Debug mode: on
web_1      |  * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
web_1      |  * Restarting with stat
web_1      |  * Debugger is active!
web_1      |  * Debugger PIN: 316-641-271
web_1      | 172.21.0.9 - - [29/Mar/2019 03:03:17] "GET /users HTTP/1.0" 200 -
web_1      | 172.21.0.9 - - [29/Mar/2019 03:03:26] "POST /auth/register HTTP/1.0" 500 -
web_1      | Traceback (most recent call last):
web_1      |   File "/usr/lib/python3.6/site-packages/redis/connection.py", line 492, in connect
web_1      |     sock = self._connect()
web_1      |   File "/usr/lib/python3.6/site-packages/redis/connection.py", line 550, in _connect
web_1      |     raise err
web_1      |   File "/usr/lib/python3.6/site-packages/redis/connection.py", line 538, in _connect
web_1      |     sock.connect(socket_address)
web_1      | OSError: [Errno 99] Address not available
web_1      | 
web_1      | During handling of the above exception, another exception occurred:
web_1      | 
web_1      | Traceback (most recent call last):
web_1      |   File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 431, in _reraise_as_library_errors
web_1      |     yield
web_1      |   File "/usr/lib/python3.6/site-packages/celery/app/base.py", line 744, in send_task
web_1      |     self.backend.on_task_call(P, task_id)
web_1      |   File "/usr/lib/python3.6/site-packages/celery/backends/redis.py", line 265, in on_task_call
web_1      |     self.result_consumer.consume_from(task_id)
web_1      |   File "/usr/lib/python3.6/site-packages/celery/backends/redis.py", line 125, in consume_from
web_1      |     return self.start(task_id)
web_1      |   File "/usr/lib/python3.6/site-packages/celery/backends/redis.py", line 107, in start
web_1      |     self._consume_from(initial_task_id)
web_1      |   File "/usr/lib/python3.6/site-packages/celery/backends/redis.py", line 132, in _consume_from
web_1      |     self._pubsub.subscribe(key)
web_1      |   File "/usr/lib/python3.6/site-packages/redis/client.py", line 3096, in subscribe
web_1      |     ret_val = self.execute_command('SUBSCRIBE', *iterkeys(new_channels))
web_1      |   File "/usr/lib/python3.6/site-packages/redis/client.py", line 3003, in execute_command
web_1      |     self.shard_hint
web_1      |   File "/usr/lib/python3.6/site-packages/redis/connection.py", line 994, in get_connection
web_1      |     connection.connect()
web_1      |   File "/usr/lib/python3.6/site-packages/redis/connection.py", line 497, in connect
web_1      |     raise ConnectionError(self._error_message(e))
web_1      | redis.exceptions.ConnectionError: Error 99 connecting to localhost:6379. Address not available.
web_1      | 
web_1      | During handling of the above exception, another exception occurred:
web_1      | 
web_1      | Traceback (most recent call last):
web_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 2309, in __call__
web_1      |     return self.wsgi_app(environ, start_response)
web_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 2295, in wsgi_app
web_1      |     response = self.handle_exception(e)
web_1      |   File "/usr/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function
web_1      |     return cors_after_request(app.make_response(f(*args, **kwargs)))
web_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 1741, in handle_exception
web_1      |     reraise(exc_type, exc_value, tb)
web_1      |   File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
web_1      |     raise value
web_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
web_1      |     response = self.full_dispatch_request()
web_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
web_1      |     rv = self.handle_user_exception(e)
web_1      |   File "/usr/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function
web_1      |     return cors_after_request(app.make_response(f(*args, **kwargs)))
web_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception

雷迪斯日志

然而,Redis似乎正在起作用:

redis_1    | 1:C 29 Mar 2019 02:33:32.722 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1    | 1:C 29 Mar 2019 02:33:32.722 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1    | 1:C 29 Mar 2019 02:33:32.722 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1    | 1:M 29 Mar 2019 02:33:32.724 * Running mode=standalone, port=6379.
redis_1    | 1:M 29 Mar 2019 02:33:32.724 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1    | 1:M 29 Mar 2019 02:33:32.724 # Server initialized
redis_1    | 1:M 29 Mar 2019 02:33:32.724 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1    | 1:M 29 Mar 2019 02:33:32.725 * DB loaded from disk: 0.000 seconds
redis_1    | 1:M 29 Mar 2019 02:33:32.725 * Ready to accept connections

config.py

class DevelopmentConfig(BaseConfig):
"""Development configuration"""
DEBUG_TB_ENABLED = True 
DEBUG = True
BCRYPT_LOG_ROUNDS = 4 
#set key
#sqlalchemy
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL')
#SQLALCHEMY_DATABASE_URI= "sqlite:///models/data/database.db"
# mail
MAIL_SERVER='smtp.gmail.com'
MAIL_PORT = 587
MAIL_USE_TLS = True
MAIL_DEBUG = True
MAIL_USERNAME = 'me@gmail.com'
MAIL_PASSWORD = 'MEfAc6w74WGx'
SEVER_NAME = 'http://127.0.0.1:8080'
# celery broker
REDIS_HOST = "0.0.0.0"
REDIS_PORT = 6379
BROKER_URL = os.environ.get('REDIS_URL', "redis://{host}:{port}/0".format(
host=REDIS_HOST, 
port=str(REDIS_PORT)))
INSTALLED_APPS = ['routes']
# celery config
CELERYD_CONCURRENCY = 10
CELERY_BROKER_URL = BROKER_URL
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
CELERY_IMPORTS = ('project.api.routes.background',)

我在这里错过了什么?

TL;灾难恢复更改redis://localhost:6379/0redis://redis:6379/0

当您运行 docker-compose 时,它会创建一个新网络,所有容器都在该网络下运行。Docker 引擎还创建了一个内部路由,允许所有容器使用它们的名称相互引用。

在您的情况下,您的网络和芹菜容器试图通过本地主机访问 redis。但在容器内部,localhost意味着自己的本地主机。您需要更改配置以将主机名指向容器的name

如果您没有使用 docker,而是为每个容器提供了不同的计算机,则localhost意味着它们自己的服务器。为了连接到 redis 服务器,您需要传递运行 redis 的计算机的 IP 地址。在 docker 中,由于上面讨论的引擎路由,您可以只传递容器的名称,而不是 IP 地址。

请注意,您仍然可以为每个容器分配静态 IP 地址,并使用这些 IP 地址而不是container_names。有关更多详细信息,请阅读 docker 文档的网络部分。

相关内容

  • 没有找到相关文章

最新更新