如何在应用程序关闭时滚动日志文件?

Question

我有一个市场数据Spring Boot应用程序，它每天运行，晚上自动关闭。应用程序关闭后，我想处理记录的市场数据消息。为了做到这一点，我正在寻找一种能够在关闭而不是启动时滚动日志文件的方法，以便我的后处理应用程序具有一致的方式来读取文件。

目前，我已经创建了一个自定义触发策略，并使用以下回日志配置来处理与市场数据相关的消息

@NoAutoStart
public class CustomTriggerPolicy<E> extends SizeAndTimeBasedFNATP<E> {
@Override
public boolean isTriggeringEvent(File activeFile, E event) {
boolean triggeringEvent = super.isTriggeringEvent(activeFile, event);
if(!triggeringEvent) {
elapsedPeriodsFileName = getCurrentPeriodsFileNameWithoutCompressionSuffix();
}
return triggeringEvent;
}
}

登录.xml

<configuration>
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener"/>
<shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook">
<delay>60000</delay>
</shutdownHook>
<appender name="MKT-DATA-FIX-LOG" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_HOME}/mkt-data-fix.log</file>
<append>false</append>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%m%n</Pattern>
</encoder>
<!-- rollover daily -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_HOME}/archived/foo/mkt-data-fix.%d{yyyy-MM-dd}.%i.gz</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="com.mkt.data.CustomTriggerPolicy">
<maxFileSize>100GB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
</configuration>

SpringBootApplication.class (Main class(

@PreDestroy
public void shutdown() {
manualLogRollover();
}
public void manualLogRollover() {
try {
Set<Appender<ILoggingEvent>> loggingAppenders = AppenderUtils.getLoggingAppenders();
AppenderUtils.rolloverAppenders(loggingAppenders);
} catch (Exception e) {
log.warn("Log rollover failed", e);
}
}

AppenderUtils.java

public class AppenderUtils {
public static Set<Appender<ILoggingEvent>> getLoggingAppenders() {
LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory();
Set<Appender<ILoggingEvent>> fileLogAppenders = new HashSet<>();
for(Logger logger : context.getLoggerList()) {
for(Iterator<Appender<ILoggingEvent>> index = logger.iteratorForAppenders(); index.hasNext();) {
Appender<ILoggingEvent> appender = index.next();
if(appender instanceof RollingFileAppender) {
if(appender.getName().equals("MKT-DATA-FIX-LOG") ||
appender.getName().equals("MKT-DATA-FIX-EVENT-LOG")) {
fileLogAppenders.add(appender);
}
}
}
}
return fileLogAppenders;
}
public static void rolloverAppenders(Set<Appender<ILoggingEvent>> appenders) {
for (Appender<ILoggingEvent> appender : appenders) {
if (appender instanceof RollingFileAppender) {
((RollingFileAppender<ILoggingEvent>) appender).rollover();
}
}
}
}

到目前为止，我已经做了以下事情;

1. 在没有 DelayingShutdownHook 的情况下尝试在关闭时手动翻转，最终会创建一个.tmp文件并且没有干净地完成翻转过程。
2. 添加一个延迟为 1 分钟的延迟关机钩子，但似乎 logback Compressor 的 asyncCompress 方法返回了一个 Future 在 30 秒后超时，因此我收到以下错误

18:16:58,872 |-ERROR in c.q.l.core.rolling.TimeBasedRollingPolicy@398887205 - Timeout while waiting for compression job to finish java.util.concurrent.TimeoutException
at java.util.concurrent.TimeoutException
at      at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at      at ch.qos.logback.core.rolling.TimeBasedRollingPolicy.waitForAsynchronousJobToStop(TimeBasedRollingPolicy.java:137)
at      at ch.qos.logback.core.rolling.TimeBasedRollingPolicy.stop(TimeBasedRollingPolicy.java:129)
at      at ch.qos.logback.core.rolling.RollingFileAppender.stop(RollingFileAppender.java:152)
at      at ch.qos.logback.core.spi.AppenderAttachableImpl.detachAndStopAllAppenders(AppenderAttachableImpl.java:107)
at      at ch.qos.logback.classic.Logger.detachAndStopAllAppenders(Logger.java:206)
at      at ch.qos.logback.classic.Logger.recursiveReset(Logger.java:331)iam
at      at ch.qos.logback.classic.Logger.recursiveReset(Logger.java:338)
at      at ch.qos.logback.classic.Logger.recursiveReset(Logger.java:338)
at      at ch.qos.logback.classic.Logger.recursiveReset(Logger.java:338)
at      at ch.qos.logback.classic.Logger.recursiveReset(Logger.java:338)
at      at ch.qos.logback.classic.LoggerContext.reset(LoggerContext.java:223)
at      at ch.qos.logback.classic.LoggerContext.stop(LoggerContext.java:348)
at      at ch.qos.logback.core.hook.ShutdownHookBase.stop(ShutdownHookBase.java:39)
at      at ch.qos.logback.core.hook.DelayingShutdownHook.run(DelayingShutdownHook.java:57)
at      at java.lang.Thread.run(Thread.java:748)

我别无选择。有谁知道如何实现我在这里寻找的东西？

Answer 1

我想我想通了。我必须定义自己的触发和滚动策略，以滚动追加器并同步压缩文件。

我不需要在我的主课上做翻转作为@PreDestroy的一部分。

@NoAutoStart
public class CustomTriggerPolicy<E> extends SizeAndTimeBasedFNATP<E> {
@Override
public boolean isTriggeringEvent(File activeFile, E event) {
boolean triggeringEvent = super.isTriggeringEvent(activeFile, event);
if(!triggeringEvent) {
elapsedPeriodsFileName = getCurrentPeriodsFileNameWithoutCompressionSuffix();
}
return triggeringEvent;
}
}

import ch.qos.logback.core.rolling.RolloverFailure;
import ch.qos.logback.core.rolling.TimeBasedRollingPolicy;
import ch.qos.logback.core.rolling.helper.*;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
/**
* Custom rolling policy where we trigger rollback at shutdown.
*/
public class MktDataRollingPolicy<E> extends TimeBasedRollingPolicy<E> {
private static final String LOGBACK_STATUS_FILE_NAME = "logback-status.log";
private Compressor compressor;
private RenameUtil renameUtil = new RenameUtil();
private String basePath;
private String logbackStatusFilePath;
FileNamePattern zipEntryFileNamePattern;
@Override
public void start() {
basePath = System.getProperty("logback.statusLog", "/home/master/logs/mkt-data-log");
logbackStatusFilePath = basePath + File.separator + LOGBACK_STATUS_FILE_NAME;
super.start();
// set the LR for our utility object
renameUtil.setContext(this.context);
compressor = new Compressor(compressionMode);
compressor.setContext(context);
if (compressionMode == CompressionMode.ZIP) {
String zipEntryFileNamePatternStr = transformFileNamePattern2ZipEntry(fileNamePatternStr);
zipEntryFileNamePattern = new FileNamePattern(zipEntryFileNamePatternStr, context);
}
Runtime.getRuntime().addShutdownHook(new Thread(new ShutdownHookRunnable()));
}
@Override
public void rollover() throws RolloverFailure {
String elapsedPeriodsFileName = getTimeBasedFileNamingAndTriggeringPolicy().getElapsedPeriodsFileName();
if (elapsedPeriodsFileName != null) {
if (compressionMode == CompressionMode.NONE) {
if (getParentsRawFileProperty() != null) {
renameUtil.rename(getParentsRawFileProperty(), elapsedPeriodsFileName);
} // else { nothing to do if CompressionMode == NONE and parentsRawFileProperty == null }
} else {
String elapsedPeriodStem = FileFilterUtil.afterLastSlash(elapsedPeriodsFileName);
if (getParentsRawFileProperty() == null) {
compressor.compress(elapsedPeriodsFileName, elapsedPeriodsFileName, elapsedPeriodStem);
} else {
if (isLogFileEmpty()) {
addInfo("Skipping compression for empty file: " + getParentsRawFileProperty());
return;
}
renameRawAndCompress(elapsedPeriodsFileName, elapsedPeriodStem);
}
}
} else {
String msg = String.format("ElapsedPeriodsFileName is null. Skipping rollover for file: %s",
getParentsRawFileProperty());
addWarn(msg);
File statusFile = new File(logbackStatusFilePath);
try (BufferedWriter bw = new BufferedWriter(new FileWriter(statusFile, true))) {
bw.append(msg);
bw.append("n");
} catch (IOException e) {
System.out.println("Failed to write to status file: " + logbackStatusFilePath);
e.printStackTrace();
}
}
}

private boolean isLogFileEmpty() {
File f = new File(getParentsRawFileProperty());
return f.exists() && f.length() == 0;
}
private String transformFileNamePattern2ZipEntry(String fileNamePatternStr) {
String slashified = FileFilterUtil.slashify(fileNamePatternStr);
return FileFilterUtil.afterLastSlash(slashified);
}
void renameRawAndCompress(String nameOfCompressedFile, String innerEntryName) throws RolloverFailure {
String parentsRawFile = getParentsRawFileProperty();
String tmpTarget = nameOfCompressedFile + System.nanoTime() + ".tmp";
renameUtil.rename(parentsRawFile, tmpTarget);
compressor.compress(tmpTarget, nameOfCompressedFile, innerEntryName);
}
class ShutdownHookRunnable implements Runnable {
@Override
public void run() {
try {
addInfo("Calling rollover inside shutdown hook");
rollover();
} catch (Exception e) {
addError("Failed to rollover", e);
}
}
}
}

登录.xml

<configuration>
<appender name="MKT-DATA-FIX-LOG" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_HOME}/mkt-data-fix.log</file>
<append>false</append>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%m%n</Pattern>
</encoder>
<!-- rollover daily -->
<rollingPolicy class="com.myapp.logback.MktDataRollingPolicy">
<fileNamePattern>${LOG_HOME}/archived/foo/mkt-data-fix.%d{yyyy-MM-dd}.%i.gz</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="com.mkt.data.CustomTriggerPolicy">
<maxFileSize>100GB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
</configuration>

Answer 2

也许它会帮助某人。

下面是一个示例，说明当 servlet 容器收到关机调用时，如何使用 Servlet 在应用程序上的登录中滚动日志文件。

我们从logback xml配置文件中获得的"ROLLING_FILES"。

<appender name="ROLLING_FILES" class="com.g5e.store.log.event.LogAppenderConfiguration">

之后，我们应该覆盖"翻转"方法。

public class LogAppenderConfiguration<E> extends RollingFileAppender<E>
{ 
@Override
public void rollover() {
super.rollover();
}
}

然后我们实现 ServletContextListener 来为 destroy 事件添加一个侦听器。

获取我们的追加器并在其上调用"滚动"方法。

@WebListener
public class LogServletContextListener implements ServletContextListener
{
@Override
public void contextDestroyed(ServletContextEvent arg0)
{
LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory();
LogAppenderConfiguration appender = (LogAppenderConfiguration) lc
.getLogger(Logger.ROOT_LOGGER_NAME)
.getAppender("ROLLING_FILES");
appender.rollover();
}
@Override
public void contextInitialized(ServletContextEvent arg0){}
}