Azure Databricks Logging配置问题

Question

我正在运行Spring Boot 2.7.4, logback 1.2.11和slf4j 1.7.36，我在我的logback- Spring .xml中有一个控制台附加程序:

<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.4</version>
<relativePath/>
</parent>

<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>log4j-over-slf4j</artifactId>
</dependency>

<dependency>
<groupId>com.databricks</groupId>
<artifactId>databricks-jdbc</artifactId>
<version>2.6.29</version>
</dependency>

<pattern>%mask(${CONSOLE_LOG_PATTERN})</pattern>
<pattern>%clr(%d{${LOG_DATEFORMAT_PATTERN:-yyyy-MM-dd HH:mm:ss.SSS}}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){magenta} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %mask(%m%n)</pattern>

我的整个应用程序都记录很好，但我需要添加Databricks JDBC驱动程序，当我从Spring JPA框架和JDBC模板执行SQL时，我在日志中得到这个:

ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [logger]
ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [msg]
ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [n]
ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [logger]
ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [msg]
ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [n]
ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.databricks.client.jdbc42.internal.io.netty.util.internal.ReflectionUtil (file:/C:/Users/[REDACTED]/com/databricks/databricks-jdbc/2.6.29/databricks-jdbc-2.6.29.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of com.databricks.client.jdbc42.internal.io.netty.util.internal.ReflectionUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

我发现这个问题，其他人有，它似乎是一些依赖带来的lo4j配置从某个地方，像一些。dat文件或log4j属性文件，但我找不到这些文件在任何地方的Databricks jar。在jar中有一些log4j文件，但不确定这些文件是否是导致问题的原因。

Log4j-charsets.properties
Log4j-config.xsd
Log4j-events.dtd
Log4j-events.xsd
Log4j-levels.xsd

这显然与我的日志模式相冲突，因为它无法理解"msg"的格式模式。或";thread"等。

当我调试我的代码并获得我的数据源和getParentLogger()时，它正在使用java.util.logging。Logger，就像我的其他数据源一样(使用Spring JPA/Hibernate)。

我已经尝试从依赖项中排除任何日志记录，但无法找出如何强制它在我的项目中使用日志记录:

<dependency>
<groupId>com.databricks</groupId>
<artifactId>databricks-jdbc</artifactId>
<version>2.6.29</version>
<exclusions>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty-buffer</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty-common</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>

我试着添加一些其他的依赖，像这样:

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jul-to-slf4j</artifactId>
</dependency>

这也不能解决问题。

我也尝试在我的logback-spring.xml中关闭StatusLogger的日志记录，但我仍然在日志中看到红色文本中的错误(ANSI是打开的)。

<logger name="org.apache.logging.log4j.status.StatusLogger" level="OFF" additivity="false">
<appender-ref ref="CONSOLE" />
</logger>
<logger name="com.databricks.*" level="OFF" additivity="false">
<appender-ref ref="CONSOLE" />
</logger>

当我使用数据源时，它只发生一次，hibernate正在打印SQL和一切都很好，所以我不知道我是否可以以某种方式抑制这一点，但理想情况下，我希望它使用我正在使用的日志框架，而不是试图覆盖我的日志模式。

Answer 1

这显然与Arrow序列化有关，可以通过JDBC URL禁用它来修复。

将;EnableArrow=0添加到URL的末尾。

警告:我不知道禁用此序列化的总体含义，但我知道这也将允许您在最新版本的Java中使用该驱动程序。