如何修复Kafka Docker容器抛出0.0.0.0/0.0.0.0:2181:Connection refused?

Question

我正在尝试为Kafka更改数据捕获设置Docker组合文件环境，我遇到了这个错误:

打开套接字连接到服务器0.0.0.0/0.0.0.0:2181。不尝试使用SASL进行身份验证(未知错误)

Socket error occurred: 0.0.0.0/0.0.0.0:2181: Connection refused

我一直在遵循这个教程https://hevodata.com/learn/kafka-cdc-postgres/，但它是直接使用链接选项运行docker命令，而不是使用docker-compose。yml文件。

我试图转换这些:

docker run -it --name kafka -p 9092:9092 --link zookeeper:zookeeper debezium/kafka

到下面的docker-compose。yml文件。然而，它似乎完全忽略了kafka_zookeper_connect环境变量，因为这是我在日志中看到的:

使用ZOOKEEPER_CONNECT = 0.0.0.0:2181

尽管如此，文档https://github.com/debezium/docker-images/tree/master/kafka/1.5指出它应该可以工作。

当我使用docker运行而不创建docker-compose文件时，它完全可以正常工作。它显示了我的本地计算机的IP地址，端口为2181，而不是0.0.0.0:2181。

docker-compose.yml:

version: "3.7"
services:
postgres:
image: debezium/postgres:10
container_name: postgres
ports:
- "5000:5432"
environment:
POSTGRES_HOST_AUTH_METHOD: trust
# POSTGRES_USER: db_user
# POSTGRES_PASSWORD: db_password
zookeeper:
image: debezium/zookeeper:1.5
container_name: zookeeper
ports:
- "2181:2181"
- "2888:2888"
- "3888:3888"
kafka:
image: debezium/kafka:1.5
container_name: kafka
ports:
- "9092:9092"
depends_on:
- zookeeper
environment:
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
connect:
image: debezium/connect:1.5
container_name: connect
ports:
- "8083:8083"
environment:
GROUP_ID: 1
CONFIG_STORAGE_TOPIC: my-connect-configs
OFFSET_STORAGE_TOPIC: my-connect-offsets
depends_on: 
- postgres
- kafka
- zookeeper
networks:
default:
name: kafkaCDC

Zookeeper容器上的zoo.cfg:

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/zookeeper/data
dataLogDir=/zookeeper/txns
clientPort=2181
autopurge.snapRetainCount=3
autopurge.purgeInterval=1

已经关注这个问题好几次了;然而，我完全迷路了。尤其是，很多例子都在使用链接。

这是GitHub的帖子，让我想到使用kafka_zookeper_connect。https://github.com/wurstmeister/kafka-docker/issues/512 issuecomment - 505905161

除了我觉得这是https://github.com/debezium/docker-images/blob/master/kafka/1.5/docker-entrypoint.sh的问题，忽略了环境变量，但它可能只是我不理解的东西，有一个conf错误。

Answer 1

要使debezium/kafka:1.5映像在docker组合中工作，您可以尝试传递以下环境变量:

ZOOKEEPER_CONNECT: "zookeeper:2181"

它解决了我的问题，示例docker撰写yaml如下:

version: "3.9"
services:
zookeeper:
image: debezium/zookeeper:1.5
ports:
- "2181:2181"
- "2888:2888"
kafka:
image: debezium/kafka:1.5
ports:
- "9092:9092"
environment:
ZOOKEEPER_CONNECT: "zookeeper:2181"
depends_on:
- zookeeper

Answer 2

不要用KAFKA_作为ENVIRONMENT变量的前缀

下面是我的工作集群:

version: '2'
services:
postgres:
image: debezium/postgres:13-alpine
container_name: postgres
hostname: postgres
environment:
POSTGRES_USER: nikamooz
POSTGRES_PASSWORD: nikamooz

ports:
- 5432:5432

zookeeper:
image: debezium/zookeeper
container_name: zookeeper
hostname: zookeeper
environment:
ZOOKEEPER_SERVER_ID: 1
ports:
- 2182:2181
- 2888:2888
- 3888:3888
volumes: 
- ./data/zoo/data:/zookeeper/data
- ./data/zoo/log:/zookeeper/txns
kafka:
image: debezium/kafka
container_name: kafka
hostname: kafka
depends_on:
- zookeeper
ports:
- 9092:9092
environment:
ZOOKEEPER_CONNECT: zookeeper:2181
BOOTSTRAP_SERVERS: kafka:9092
volumes: 
- ./data/kafka/data:/kafka/data
- ./data/kafka/logs:/kafka/logs

connect:
image: debezium/connect
container_name: connect
hostname: connect
depends_on:
- kafka
- postgres
ports:
- 8083:8083
environment:
GROUP_ID: holding_group
CONFIG_STORAGE_TOPIC: holding_storage_topic
OFFSET_STORAGE_TOPIC: holding_offset_topic
BOOTSTRAP_SERVERS: kafka:9092

Answer 3

我可以通过设置zookeeper连接地址到docker容器的IP地址来修复它

要获取IP地址，运行

docker inspect <container-name> --format='{{ .NetworkSettings.IPAddress }}'

和start启动kafka，如下所示

docker run --name some-kafka -p 9092:9092 -e KAFKA_ZOOKEEPER_CONNECT=<zookeeper-ip>:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 confluentinc/cp-kafka

这个github注释帮助我找到了我错过的东西