我一直在尝试在Docker容器上运行Zookeeper和Kafka。我得到了很多错误，[error occurred during error reporting , id 0xb]和[Too many errors, abort]在我的终端。然后是library initialization failed - unable to allocate file descriptor table - out of memory
我使用下面的docker-compose。yml文件
version: "3.8"
services:
zookeeper:
image: confluentinc/cp-zookeeper:5.5.4
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
kafka:
image: confluentinc/cp-kafka:5.5.4
# If you want to expose these ports outside your dev PC,
# remove the "127.0.0.1:" prefix
ports:
- 127.0.0.1:9092:9092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
我通过在docker.service中重写命令参数ExecStart修复了这个问题
sudo systemctl edit docker
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --default-ulimit nofile=65536:65536 -H fd://
保存并运行sudo systemctl daemon-reload和sudo systemctl restart docker。
尝试像这样添加无限制配置到zookeeper
---
version: '3'
services:
zookeeper:
image: wurstmeister/zookeeper
ulimits:
nofile:
soft: 65536
hard: 65536
ports:
- "2181:2181"
我认为你的kafka代理需要依赖于你的zookeeper。
---
version: '2'
services:
zookeeper:
image: confluentinc/cp-zookeeper:6.2.0
hostname: zookeeper
container_name: zookeeper
ports:
- "2181:2181"
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
broker:
image: confluentinc/cp-server:6.2.0
hostname: broker
container_name: broker
depends_on:
- zookeeper
ports:
- "9092:9092"
- "9101:9101"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_CONFLUENT_LICENSE_TOPIC_REPLICATION_FACTOR: 1
KAFKA_CONFLUENT_BALANCER_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_JMX_PORT: 9101
KAFKA_JMX_HOSTNAME: localhost
CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: broker:29092
CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
CONFLUENT_METRICS_ENABLE: 'true'
CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous'
version: "3.9"
networks:
nrixmw:
driver: bridge
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
ulimits:
nofile:
soft: 65536
hard: 65536
hostname: zookeeper
ports:
- 2181:2181
environment:
ZOOKEEPER_CLIENT_PORT: '2181'
ZOOKEEPER_TICK_TIME: '2000'
networks:
- nrixmw
broker:
image: confluentinc/cp-kafka:5.4.2
ulimits:
nofile:
soft: 65536
hard: 65536
hostname: broker
depends_on:
- zookeeper
ports:
- 29092:29092
- 9092:9092
- 9101:9101
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
# KAFKA_ADVERTISED_HOST_NAME: 'broker'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_LISTENERS: 'PLAINTEXT://:29092,PLAINTEXT_HOST://:9092'
KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
KAFKA_DELETE_TOPIC_ENABLE: 'true'
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
networks:
- nrixmw
mem_limit: 6g
restart: always
schema-registry:
image: confluentinc/cp-schema-registry:5.5.1
ulimits:
nofile:
soft: 65536
hard: 65536
depends_on:
- zookeeper
- broker
ports:
- "8081:8081"
environment:
SCHEMA_REGISTRY_HOST_NAME: schema-registry
SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: 'zookeeper:2181'
networks:
- nrixmw
我有一台薄荷机，在那里我不需要所有这些无限制，它仍然工作，这是arch操作系统的问题可能。不确定。