Kafka授权仅在端口9092上失败

Question

我使用confluent kafka docker镜像，并通过以下配置启用了身份验证和授权。KAFKA_ADVERTISED_LISTENERS =明文://:9092年,SASL_SSL://: 9093

=比;9093 SASL_SSL
=>9092年明文

容器环境变量

- KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND=false
- KAFKA_SSL_CLIENT_AUTH=required
- KAFKA_SECURITY_INTER_BROKER_PROTOCOL=SASL_SSL
- KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
- KAFKA_SASL_ENABLED_MECHANISMS=PLAIN
- KAFKA_AUTHORIZER_CLASS_NAME=kafka.security.authorizer.AclAuthorizer
- KAFKA_SUPER_USERS="User:admin"
- KAFKA_OPTS=-Djava.security.auth.login.config={{ kafka_secrets_dir }}/kafka_jaas.conf

kafka_jaas.conf

KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="admin"
password="admin"
user_admin="admin"
user_second_user="read_user";
};
Client {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="admin"
password="admin";
};

配置消费者acl

bin/kafka-acls --authorizer-properties zookeeper.connect=my.host1:2181,host2:2181,host3:2181 --add --allow-principal User:second_user --consumer --topic '*' --group '*'

配置生产者acl

kafka-acls --authorizer-properties zookeeper.connect=my.host1:2181,host2:2181,host3:2181 --add --allow-principal User:second_user --producer --topic '*'

我想在两个端口上使用kafka。9093有SSL加密，9092没有。因此，我用一个简单的主机消费者/生产者来测试它。端口9093工作正常，我可以使用和生成消息。问题是它不能在端口9092上工作。我总是得到一个认证错误TopicAuthorizationException: Not authorized to access topics: [test_topic]。我用"second_user"进行了测试。甚至还有超级用户"admin"。为什么它只适用于安全端口?我错过了什么配置吗?

Console消费通过端口9093 (working)

#consumer.properties
ssl.endpoint.identification.algorithm=
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
ssl.truststore.location=/home/vagrant/kafka-2.8.0/ssl/kafka.truststore.jks
ssl.truststore.password=changeme
ssl.protocol=TLS
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required 
username="admin" 
password="admin";

# create consumer => This is working!
/bin/kafka-console-consumer.sh --bootstrap-server host1:9093,host2:9093,host3:9093 --topic test_topic --from-beginning --consumer.config consumer.properties

Console消费通过端口9092 (不工作)

#consumer.properties
security.protocol=PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required 
username="admin" 
password="admin";
#create consumer
kafka-console-consumer.sh --bootstrap-server host1:9092,host2:9092,host3:9092 --topic test_topic --from-beginning --consumer.config consumer.properties
=>TopicAuthorizationException: Not authorized to access topics: [test_topic]

我还用python和confluence -kafka-python包测试了它(不工作)。
test.py

self.consumer = Consumer({
'bootstrap.servers': "host1:9092,host2:9092,host3:9092",
'group.id': f"test",
'security.protocol': "PLAINTEXT",
'sasl.mechanism': 'PLAIN',
'sasl.username': 'admin',
'sasl.password': "admin"
})
=> FindCoordinator response error: Group authorization failed

Answer 1

您没有在9092端口上启用身份验证，

与的组合KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND

，您正在获得授权失败，

要修复它，您应该更改为SASL_PLAINTEXT以允许SASL身份验证而不需要TLS加密

明文://:9092→SASL_PLAINTEXT://: 9092