如何在Java中创建自定义的日志监视作业，该作业提供了日志文件上发生异常消息的报告

Question

1. 应该能够处理较大的日志文件并提供异常消息报告
2. 完成日志分析后，将通知触发到特定的邮件ID。还请建议哪种框架是处理大文件的最佳框架。[例如：Spring Boot/batch]

Answer 1

我建议与麋鹿堆一样。将日志传输到弹性搜索并在Kibana中设置警报。

Answer 2

可以在系统上使用sendmail客户端并在该系统上运行脚本以在任何例外发送警报。

exception="Exception" # "Error", "HTTP 1.1 " 500", etc
ignoredException="ValidationException"
# log file to scan 
logFileToScan=/var/log/tomcat8/log/application.log
# file where we will keep log of this script
logFilePath=/home/ec2-user/exception.log
# a file where we store till what line the log file has been scanned
# initalize it with 0 
countPath=/home/ec2-user/lineCount
# subject with which you want to receive the mail regading Exception
subject="[ALERT] Exception"
# from whom do you want to send the mail regarding Exception
from="[e-mail]"
# to whom do you want to send the mail
to="[e-mail]"
# number of lines, before the line containing the word to be scanned, to be sent in the mail
linesBefore=1
# number of lines, before the line containing the word to be scanned, to be sent in the mail
linesAfter=4
# start line
fromLine=`cat $countPath`
# current line count in the file
toLine=`wc -l $logFileToScan | awk '{print $1}'`
#logs are rolling so if fromLine has a value greater than toLine then fromLine has to be set to 0
if [ "$fromLine" == "" ]; then
        fromLine=0
        echo `date` fromLine values was empty, set to 0 >> $logFilePath
elif [ $fromLine -gt $toLine ]; then
        echo `date` logfile was rolled, updating fromLine from $fromLine to 0 >> $logFilePath
        fromLine=0
fi
# if from n to lines are equal then no logs has been generated since last scan
if [ "$fromLine" == "$toLine" ]; then
        echo `date` no logs genetared after last scan >> $logFilePath
else
        echo `date` updating linecount to $toLine >> $logFilePath
        echo $toLine > $countPath
        logContent=`tail -n +"$fromLine" $logFileToScan | head -n "$((toLine - fromLine))" | grep -v $ignoredException | grep -A $linesAfter -B $linesBefore $exception`
        logContent=`echo $logContent | cut -c1-2000`
        if [ "$logContent" == "" ]; then
                echo `date` no exception found >> $logFilePath
                else
                /usr/sbin/sendmail $to <<EOF
subject: $subject
from: $from
logContent=$logContent
EOF
        fi
fi