无法从其他容器连接到 Kafka Docker 容器:连接被拒绝 - 1/1 代理关闭



我正在尝试将数据聚合器连接到kafka代理 目的是从HTTP端点收集数据,并定期将其保存到数据库中。两者都在 docker 中运行,使用以下 docker-compose.yml:

version: "3"
services:
zk:
image: confluentinc/cp-zookeeper:5.0.0
container_name: zk
ports:
- 2181:2181
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
restart: on-failure:5
kafka:
image: confluentinc/cp-kafka:5.0.0
container_name: kafka
ports:
- 9092:9092
- 9094:9094
environment:
KAFKA_ZOOKEEPER_CONNECT: zk:2181
KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9092,OUTSIDE://localhost:9094
KAFKA_LISTENERS: INSIDE://0.0.0.0:9092,OUTSIDE://0.0.0.0:9094
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
KAFKA_NUM_PARTITIONS: 4
restart: on-failure:5
tinygator-metric-aggregator:
image: tinygator.aggregator:latest
container_name: tg-aggregator
environment:
KAFKA_BOOTSTRAP_SERVER: localhost:9094
KAFKA_TOPIC: test
METRIC_ENDPOINT: "http://localhost:15672/api/queues/%2F/Test?sort=message_stats.publish_details.rate&sort_reverse=true&columns=name,message_stats.publish_details.rate,message_stats.deliver_get_details.rate"
METRIC_METHOD: GET
METRIC_INTERVAL: 5000
METRIC_HEADERS: '{"authorization":"Basic Z3Vlc3Q6Z3Vlc3Q="}'
restart: on-failure:5

卡夫卡和zk似乎正常启动。但是,我收到聚合器的以下错误:

tg-aggregator                  | %3|1576348274.043|FAIL|rdkafka#producer-1| [thrd:0.0.0.0:9094/bootstrap]: 0.0.0.0:9094/bootstrap: Connect to ipv4#0.0.0.0:9094 failed: Connection refused (after 4704ms in state CONNECT)
tg-aggregator                  | %3|1576348274.048|ERROR|rdkafka#producer-1| [thrd:0.0.0.0:9094/bootstrap]: 0.0.0.0:9094/bootstrap: Connect to ipv4#0.0.0.0:9094 failed: Connection refused (after 4704ms in state CONNECT)
tg-aggregator                  | %3|1576348274.048|ERROR|rdkafka#producer-1| [thrd:0.0.0.0:9094/bootstrap]: 1/1 brokers are down
tg-aggregator                  | Unhandled exception. System.Net.Http.HttpRequestException: Cannot assign requested address
tg-aggregator                  |  ---> System.Net.Sockets.SocketException (99): Cannot assign requested address
tg-aggregator                  |    at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
tg-aggregator                  |    --- End of inner exception stack trace ---
tg-aggregator                  |    at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
tg-aggregator                  |    at MetricAggregator.Program.Main(String[] args) in /app/Program.cs:line 83
tg-aggregator                  |    at MetricAggregator.Program.<Main>(String[] args)

现在,我和一位同事已经坐了几个小时了(我们都是Kafka的新手(。在MacOS上编写聚合器(可以在 https://github.com/Azer0s/TinyGator 下找到(的人不会出现此问题。我在Windows 10上,我的同事在Arch上。我们尝试了很多不同的端口号,设置Kafka侦听器和通告侦听器的方法等。简而言之,我们完全不知所措。

有人有想法吗?

更新:

在阅读了 Kafka 侦听器之后,我现在已经修改了我的 docker-compose,更改了聚合器尝试连接的地址:

KAFKA_BOOTSTRAP_SERVER: kafka:9092

但错误仍然是一样的。

该错误来自指标终结点,该终结点在发送 Kafka 消息之前调用。

这表明您的指标终端节点 URL 不正确,因为localhost容器本身(它正在拉取指标、没有指标本身或公开 HTTP 接口(

该示例看起来像是在提取 RabbitMQ 队列统计信息


无论如何,kafka:9092在您的上下文中是正确的

编写聚合器的人不会发生此问题

我敢肯定,如果他给出了无效的指标端点,它会的

最新更新