当我在Docker容器中使用kafka运行spring-boot应用程序时，Broker不可用

Question

我是docker的新手，在docker容器中运行Kafka代理时遇到问题。我与kafka、生产者和消费者在同一应用程序中开发了spring-boot应用程序。应用程序在没有docker的情况下成功运行，但当我试图通过主机上的docker容器运行它时，会出现以下错误：

org.apache.kafka.clients.NetworkClient   : [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-group_id-1, groupId=group_id] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected

我用的是弹簧靴2.4.2和kafka_2.7.0

我遵循的步骤：

1-I为spring-boot应用程序构建docker镜像。

2-我创建docker-compose.yml来运行zookeeper和kafka服务，如下所示：

version: '3'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"     
kafka:
image: wurstmeister/kafka
build:
context: .
dockerfile: Dockerfile
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_MESSAGE_MAX_BYTES: 2000000
KAFKA_CREATE_TOPICS: "Topic1:1:1"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- zookeeper

3-然后我执行docker-compose.yml：docker组合-d

4-最后我运行了spring-boot应用程序镜像：docker运行-p 8080:8080 spring-boot应用程序。

这是我的客户端配置(春季启动应用程序)：application.properties:

消费者配置

#spring.kafka.consumer.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id==group_id
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.key-deserializer= 
org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer= org.apache.kafka.common.serialization.StringDeserializer

生产者配置

#spring.kafka.producer.bootstrap-servers=localhost:9092
spring.kafka.producer.key-serializer=   org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer= org.apache.kafka.common.serialization.StringSerializer

Answer 1

您需要为外部连接(docker容器外部)配置kafka代理。

编辑您的广告听众如下：

KAFKA_ADVERTISED_LISTENERS: 
INTERNAL://kafka:19092,
EXTERNAL://localhost:9092

关注这篇文章了解更多见解。

要在远程服务器上配置kafka集群，请阅读本文。

我使用了以下yml，它对我有效：

version: '3'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"     
kafka:
image: wurstmeister/kafka
build:
context: .
dockerfile: Dockerfile
ports:
- "9092:9092"
environment:
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka:19092,LISTENER_DOCKER_EXTERNAL://localhost:9092
KAFKA_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka:19092,LISTENER_DOCKER_EXTERNAL://localhost:9092
KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_EXTERNAL
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_MESSAGE_MAX_BYTES: 2000000
KAFKA_CREATE_TOPICS: "Topic1:1:1"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- zookeeper

Answer 2

以下是如何添加应用程序来编写文件的示例。在运行此文件之前，请创建您的映像。我添加了网络，将它们组合在同一个网络部分。

version: '3'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"     
networks:
- app-network
kafka:
image: wurstmeister/kafka
build:
context: .
dockerfile: Dockerfile
ports:
- "9092:9092"
environment:
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_MESSAGE_MAX_BYTES: 2000000
KAFKA_CREATE_TOPICS: "Topic1:1:1"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- zookeeper
networks:
- app-network
app:
image: spring-boot-app
ports:
- "8080:8080"
depends_on:
- kafka
networks:
- app-network

networks:
app-network:
driver: bridge