在Windows上运行Apache Kafka 3时AccessDeniedException &g

Question

Apache Kafka 3.0昨天发布了，我决定在Windows 10上尝试一下。虽然ZooKeeper启动正常，但Kafka在记录几个异常后关闭。

我理解AccessDeniedException是不言自明的，但权限不是问题，因为Kafka能够创建AccessDeniedException被记录的文件。

[2021-09-22 23:58:18,295] ERROR Failed to write meta.properties due to (kafka.server.BrokerMetadataCheckpoint)
java.nio.file.AccessDeniedException: D:tmpkafka-logs
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:121)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
at org.apache.kafka.common.utils.Utils.flushDir(Utils.java:953)
at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:941)
at kafka.server.BrokerMetadataCheckpoint.liftedTree1$1(BrokerMetadataCheckpoint.scala:214)
at kafka.server.BrokerMetadataCheckpoint.write(BrokerMetadataCheckpoint.scala:204)
at kafka.server.KafkaServer.$anonfun$checkpointBrokerMetadata$2(KafkaServer.scala:772)
at kafka.server.KafkaServer.$anonfun$checkpointBrokerMetadata$2$adapted(KafkaServer.scala:770)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:889)
at kafka.server.KafkaServer.checkpointBrokerMetadata(KafkaServer.scala:770)
at kafka.server.KafkaServer.startup(KafkaServer.scala:322)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2021-09-22 23:58:18,301] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.nio.file.AccessDeniedException: D:tmpkafka-logs
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:121)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
at org.apache.kafka.common.utils.Utils.flushDir(Utils.java:953)
at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:941)
at kafka.server.BrokerMetadataCheckpoint.liftedTree1$1(BrokerMetadataCheckpoint.scala:214)
at kafka.server.BrokerMetadataCheckpoint.write(BrokerMetadataCheckpoint.scala:204)
at kafka.server.KafkaServer.$anonfun$checkpointBrokerMetadata$2(KafkaServer.scala:772)
at kafka.server.KafkaServer.$anonfun$checkpointBrokerMetadata$2$adapted(KafkaServer.scala:770)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:889)
at kafka.server.KafkaServer.checkpointBrokerMetadata(KafkaServer.scala:770)
at kafka.server.KafkaServer.startup(KafkaServer.scala:322)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)

和

[2021-09-22 23:58:19,115] INFO Shutting down. (kafka.log.LogManager)
[2021-09-22 23:58:19,200] ERROR Error while writing to checkpoint file D:tmpkafka-logsrecovery-point-offset-checkpoint (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: D:tmpkafka-logs
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:121)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
at org.apache.kafka.common.utils.Utils.flushDir(Utils.java:953)
at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:941)
at kafka.server.checkpoints.CheckpointFile.liftedTree1$1(CheckpointFile.scala:114)
at kafka.server.checkpoints.CheckpointFile.write(CheckpointFile.scala:92)
at kafka.server.checkpoints.OffsetCheckpointFile.write(OffsetCheckpointFile.scala:67)
at kafka.log.LogManager.$anonfun$checkpointRecoveryOffsetsInDir$1(LogManager.scala:675)
at kafka.log.LogManager.$anonfun$checkpointRecoveryOffsetsInDir$1$adapted(LogManager.scala:671)
at scala.Option.foreach(Option.scala:437)
at kafka.log.LogManager.checkpointRecoveryOffsetsInDir(LogManager.scala:671)
at kafka.log.LogManager.$anonfun$shutdown$9(LogManager.scala:542)
at kafka.log.LogManager.$anonfun$shutdown$9$adapted(LogManager.scala:535)
at kafka.utils.Implicits$MapExtensionMethods$.$anonfun$forKeyValue$1(Implicits.scala:62)
at scala.collection.mutable.HashMap$Node.foreachEntry(HashMap.scala:633)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:499)
at kafka.log.LogManager.shutdown(LogManager.scala:535)
at kafka.server.KafkaServer.$anonfun$shutdown$18(KafkaServer.scala:701)
at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:68)
at kafka.server.KafkaServer.shutdown(KafkaServer.scala:701)
at kafka.server.KafkaServer.startup(KafkaServer.scala:435)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2021-09-22 23:58:19,202] ERROR Disk error while writing recovery offsets checkpoint in directory D:tmpkafka-logs: Error while writing to checkpoint file D:tmpkafka-logsrecovery-point-offset-checkpoint (kafka.log.LogManager)
[2021-09-22 23:58:19,233] ERROR Error while writing to checkpoint file D:tmpkafka-logslog-start-offset-checkpoint (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: D:tmpkafka-logs
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:121)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
at org.apache.kafka.common.utils.Utils.flushDir(Utils.java:953)
at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:941)
at kafka.server.checkpoints.CheckpointFile.liftedTree1$1(CheckpointFile.scala:114)
at kafka.server.checkpoints.CheckpointFile.write(CheckpointFile.scala:92)
at kafka.server.checkpoints.OffsetCheckpointFile.write(OffsetCheckpointFile.scala:67)
at kafka.log.LogManager.$anonfun$checkpointLogStartOffsetsInDir$1(LogManager.scala:698)
at kafka.log.LogManager.$anonfun$checkpointLogStartOffsetsInDir$1$adapted(LogManager.scala:694)
at scala.Option.foreach(Option.scala:437)
at kafka.log.LogManager.checkpointLogStartOffsetsInDir(LogManager.scala:694)
at kafka.log.LogManager.$anonfun$shutdown$9(LogManager.scala:545)
at kafka.log.LogManager.$anonfun$shutdown$9$adapted(LogManager.scala:535)
at kafka.utils.Implicits$MapExtensionMethods$.$anonfun$forKeyValue$1(Implicits.scala:62)
at scala.collection.mutable.HashMap$Node.foreachEntry(HashMap.scala:633)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:499)
at kafka.log.LogManager.shutdown(LogManager.scala:535)
at kafka.server.KafkaServer.$anonfun$shutdown$18(KafkaServer.scala:701)
at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:68)
at kafka.server.KafkaServer.shutdown(KafkaServer.scala:701)
at kafka.server.KafkaServer.startup(KafkaServer.scala:435)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2021-09-22 23:58:19,234] ERROR Disk error while writing log start offsets checkpoint in directory D:tmpkafka-logs: Error while writing to checkpoint file D:tmpkafka-logslog-start-offset-checkpoint (kafka.log.LogManager)
[2021-09-22 23:58:19,236] INFO Shutdown complete. (kafka.log.LogManager)

ZooKeeper scripts -

binwindowskafka-server-start.bat .configserver.properties

Kafka启动脚本-

binwindowskafka-server-start.bat .configserver.properties

我在以下环境中使用ZooKeeper和Kafka的默认配置文件-

JDK 11.0.2
Kafka 3.0
Windows 10

Answer 1

现在在Windows 10中可以使用wsl2来运行kafka。请遵循以下指南:

https://www.confluent.io/blog/set-up-and-run-kafka-on-windows-linux-wsl-2/enable-windows-subsystem

，然后你可以编写代码并在wsl

中启动它有时windows和wsl通信有问题，所以你必须在windows命令提示符中发出wsl——shutdown，然后重新启动它(再次发出wsl)，之后你可以调用你的wsl启动的应用程序，并从windows调用其暴露的服务