我在k8s集群上运行Flink 1.11,在尝试更新log4j-console.properties文件时出现以下错误:
Starting Task Manager
Enabling required built-in plugins
Linking flink-s3-fs-hadoop-1.11.1.jar to plugin directory
Successfully enabled flink-s3-fs-hadoop-1.11.1.jar
sed: couldn't open temporary file /opt/flink/conf/sedl2dH0X: Read-only file system
sed: couldn't open temporary file /opt/flink/conf/sedPLYAzY: Read-only file system
/docker-entrypoint.sh: 72: /docker-entrypoint.sh: cannot create /opt/flink/conf/flink-conf.yaml: Permission denied
sed: couldn't open temporary file /opt/flink/conf/sede0G5LW: Read-only file system
/docker-entrypoint.sh: 120: /docker-entrypoint.sh: cannot create /opt/flink/conf/flink-conf.yaml.tmp: Read-only file system
Starting taskexecutor as a console application on host flink-taskmanager-c765c947c-qx68t.
Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ser/FilterProvider
at org.apache.logging.log4j.core.layout.JsonLayout.<init>(JsonLayout.java:158)
at org.apache.logging.log4j.core.layout.JsonLayout.<init>(JsonLayout.java:69)
at org.apache.logging.log4j.core.layout.JsonLayout$Builder.build(JsonLayout.java:102)
at org.apache.logging.log4j.core.layout.JsonLayout$Builder.build(JsonLayout.java:77)
at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:122)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:1002)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:942)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:934)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:934)
at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:552)
at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:241)
at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:288)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:579)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:651)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:668)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:253)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:153)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:138)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:45)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:48)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:30)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:329)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:349)
at org.apache.flink.runtime.taskexecutor.TaskManagerRunner.<clinit>(TaskManagerRunner.java:89)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.ser.FilterProvider
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)
at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
我的log4j-console.properties:
rootLogger.level = INFO
#rootLogger.appenderRef.console.ref = ConsoleAppender
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n
appender.kafka.type = Kafka
appender.kafka.name = Kafka
appender.kafka.topic = test
appender.kafka.layout.type = JsonLayout
appender.kafka.layout.complete = false
appender.kafka.additional1.type = KeyValuePair
appender.kafka.additional1.key=app
appender.kafka.additional1.value=TEST
appender.kafka.additional2.type = KeyValuePair
appender.kafka.additional2.key=subsystem
appender.kafka.additional2.value=TEST
appender.kafka.additional3.type = KeyValuePair
appender.kafka.additional3.key=deployment
appender.kafka.additional3.value=TEST
appender.kafka.property.bootstrap.servers=***
rootLogger.appenderRef.console.ref = STDOUT
rootLogger.appenderRef.kafka.ref = Kafka
Im使用";flink:1.11-scala_2.11-java11〃;docker镜像,并验证所有log4j2依赖项都在类路径中。
我还试图从上面的基本映像创建一个新的docker映像,并添加缺失的依赖项,但什么都没发生。
我也受到了这个错误的困扰。这里的问题是,当任务管理器和作业管理器启动时,它们运行的是经过修改的类路径,而不是您通过构建系统构建的JAR。
请参阅flink源代码中的constructFlinkClassPath
。要证明这一点,请恢复JSON日志记录模式,并在启动时检查tm/jm日志中的类路径。您会注意到您的JAR不在类路径中。
要解决此问题,您需要向tm/jm节点中的lib
文件夹提供依赖项(在这种情况下,您需要jackson-core
jackson-annotations
和jackson-databind
((默认情况下,lib
文件夹包含在flink类路径中(。
如果使用docker,则可以在构建容器(RUN wget...
(时执行此操作。