在使用 Kerberos 身份验证通过 SpringBoot 连接到 Kafka 时遇到问题。我正在使用自定义 Kafka 连接管理器，其中包含以下详细信息 -
bootstrap-servers-sasl: node1:9094, node2:9094, node3:9094
protocol: SASL_SSL
mechanism: GSSAPI
kerberos:
service:
name: kfkusr
jaas:
config: "com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="#keytab-name#" principal="abc/node2@domain.NET";"
其中#keytab-name#将在运行时替换为以下值，如 -
我的本地电脑 -C:/Users/MyPC/AppData/Local/Temp/abc.node2_d2254866264751402128.keytab
聚氯乙烯 -/home/vcap/tmp/abc.node2_d2215947326380395062.keytab
在本地，应用程序运行良好，消息将发送到 Kafka。但是当在 PCF 上运行失败并出现以下异常时 -
2019-08-09T14:40:46.481-05:00 [APP/PROC/WEB/0] [OUT] WARN [9f-3868cbe47d81] org.apache.kafka.clients.NetworkClient o.a.k.c.NetworkClient.processDisconnection(NetworkClient.java:585) - ||||||||||||||Connection to node -1 terminated during authentication. This may indicate that authentication failed due to invalid credentials.
...
...
Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.: org.springframework.kafka.core.KafkaProducerException: Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms
...
...
Exception thrown when sending a message with key='null' and payload='<my payload>' to topic <test_topic> :: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
更新 1 -
添加 krb5.conf 文件后，它具有默认领域
[libdefaults]
default_realm = mydomain.NET
身份验证错误消失，但仍有以下错误
Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.: org.springframework.kafka.core.KafkaProducerException: Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms
一旦为org.apache.kafka: DEBUG启用调试，就会显示新错误，如 -
2019-08-14T09:49:51.947-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] org.apache.kafka.clients.NetworkClient o.a.k.c.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:907) - ||||||||||||||Initialize connection to node node1:9094 (id: -1 rack: null) for sending metadata request
2019-08-14T09:49:51.947-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] org.apache.kafka.clients.NetworkClient o.a.k.c.NetworkClient.initiateConnect(NetworkClient.java:762) - ||||||||||||||Initiating connection to node node1:9094 (id: -1 rack: null)
2019-08-14T09:49:51.948-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] o.a.k.c.s.a.SaslClientAuthenticator o.a.k.c.s.a.SaslClientAuthenticator.setSaslState(SaslClientAuthenticator.java:209) - ||||||||||||||Set SASL client state to SEND_HANDSHAKE_REQUEST
2019-08-14T09:49:51.948-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] o.a.k.c.s.a.SaslClientAuthenticator o.a.k.c.s.a.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:134) - ||||||||||||||Creating SaslClient: client=abc/node1@domain.net;service=kfkusr;serviceHostname=node1;mechs=[GSSAPI]
2019-08-14T09:49:51.949-05:00 [APP/PROC/WEB/0] [OUT] INFO [d3-5b28248c661c] o.a.k.common.network.SaslChannelBuilder o.a.k.c.n.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:119) - ||||||||||||||Failed to create channel due to : org.apache.kafka.common.KafkaException: Failed to configure SaslClientAuthenticator at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:125) at org.apache.kafka.common.network.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:116) at org.apache.kafka.common.network.Selector.connect(Selector.java:203) at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:764) at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:60) at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:908) at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:819) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:431) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:224) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:162) at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.kafka.common.KafkaException: Failed to create SaslClient with mechanism GSSAPI at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:140) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:123) ... 10 common frames omittedCaused by: javax.security.sasl.SaslException: Failure to initialize security context at com.sun.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:149) at com.sun.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:63) at javax.security.sasl.Sasl.createSaslClient(Sasl.java:384) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:136) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:131) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:131) ... 11 common frames omittedCaused by: org.ietf.jgss.GSSException: Invalid name provided (Mechanism level: KrbException: Cannot locate default realm) at sun.security.jgss.krb5.Krb5NameElement.getInstance(Krb5NameElement.java:129) at sun.security.jgss.krb5.Krb5MechFactory.getNameElement(Krb5MechFactory.java:95) at sun.security.jgss.GSSManagerImpl.getNameElement(GSSManagerImpl.java:203) at sun.security.jgss.GSSNameImpl.getElement(GSSNameImpl.java:477) at sun.security.jgss.GSSNameImpl.init(GSSNameImpl.java:201) at sun.security.jgss.GSSNameImpl.<init>(GSSNameImpl.java:170) at sun.security.jgss.GSSManagerImpl.createName(GSSManagerImpl.java:138) at com.sun.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:107) ... 18 common frames omitted
2019-08-14T09:49:51.949-05:00 [APP/PROC/WEB/0] [OUT] DEBUG [d3-5b28248c661c] org.apache.kafka.clients.NetworkClient o.a.k.c.NetworkClient.initiateConnect(NetworkClient.java:773) - ||||||||||||||Error connecting to node abcNode:9094 (id: -1 rack: null): java.io.IOException: Channel could not be created for socket java.nio.channels.SocketChannel[closed] at org.apache.kafka.common.network.Selector.connect(Selector.java:210) at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:764) at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:60) at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:908) at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:819) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:431) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:224) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:162) at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: Failed to configure SaslClientAuthenticator at org.apache.kafka.common.network.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:120) at org.apache.kafka.common.network.Selector.connect(Selector.java:203) ... 8 common frames omittedCaused by: org.apache.kafka.common.KafkaException: Failed to configure SaslClientAuthenticator at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:125) at org.apache.kafka.common.network.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:116) ... 9 common frames omittedCaused by: org.apache.kafka.common.KafkaException: Failed to create SaslClient with mechanism GSSAPI at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:140) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:123) ... 10 common frames omittedCaused by: javax.security.sasl.SaslException: Failure to initialize security context at com.sun.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:149) at com.sun.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:63) at javax.security.sasl.Sasl.createSaslClient(Sasl.java:384) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:136) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:131) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:131) ... 11 common frames omittedCaused by: org.ietf.jgss.GSSException: Invalid name provided (Mechanism level: KrbException: Cannot locate default realm) at sun.security.jgss.krb5.Krb5NameElement.getInstance(Krb5NameElement.java:129) at sun.security.jgss.krb5.Krb5MechFactory.getNameElement(Krb5MechFactory.java:95) at sun.security.jgss.GSSManagerImpl.getNameElement(GSSManagerImpl.java:203) at sun.security.jgss.GSSNameImpl.getElement(GSSNameImpl.java:477) at sun.security.jgss.GSSNameImpl.init(GSSNameImpl.java:201) at sun.security.jgss.GSSNameImpl.<init>(GSSNameImpl.java:170) at sun.security.jgss.GSSManagerImpl.createName(GSSManagerImpl.java:138) at com.sun.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:107) ... 18 common frames omitted
好的。我为此寻求解决方案..可能对遇到此问题的其他人有所帮助。
首先，我在生产者配置属性中设置 krb5.conf 文件。此文件包含使用 kerberos 发现服务所需的主机、领域和其他详细信息
System.setProperty("java.security.krb5.conf", <path to conf file>);
我没有直接在属性中设置 jaas 配置，而是创建了 jaas conf 文件并在系统属性中设置它 -
在《代码》是这样的之前——
props.put("sasl.jaas.config", jaasConfig);
更改后 -
System.setProperty("java.security.auth.login.config", jaasFile.getPath());
示例 Jaas 配置文件jaas_client.conf结构 -
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
keyTab="#keytab-name#"
principal="#principal#";
};
附言jaas_client.conf文件中，在将#keytab-name#更新为 keytab 文件的实际路径并用实际值#principal#值后，我在设置为SystemProperties之前重写了文件
可以使用以下属性设置调试模式，以便在启用调试时查看 JaaS 身份验证和票证日志 -
System.setProperty("sun.security.krb5.debug", ""+<true/false>);