格式化shell脚本日志并写入文件

Question

我需要将shell脚本中的所有输出/错误日志记录到一个标准格式的文件中。

例如script.sh文件:

echo "I want to be a json message!"
echo "I also want to be a json message!"

执行如下:

./script.sh >> jsonmessages.log

应该给我一个文件内容，如:

{timestamp: '2020-12-31 00:00:00', message: 'I want to be a json message!'}
{timestamp: '2020-12-31 00:00:00', message: 'I also want to be a json message!'}

Answer 1

可靠地将shell字符串转换为有效JSON字符串的最佳选择是使用JSON解析器/格式化器。最流行的是jq。

下面是一个使用jq的时间戳JSON消息记录器的实现:

#!/usr/bin/env bash
json_logger() {
while read -r msg || [ -n "$msg" ]; do
jq 
-nc 
--arg msg "$msg" 
'{ "timestamp": (now | strftime("%Y-%m-%d %H:%M:%S")), "message": $msg }'
done
}

----------

编辑:

正如KamilCuk所写:

我们可以做得更好没有缓慢的bash循环-它只是jq --raw-input '{ "timestamp": (now | strftime("%Y-%m-%d %H:%M:%S")), "message": . }'

所以这是一个改进的json_logger:

json_logger() {
jq 
--raw-input 
--compact-output 
'{ "timestamp": (now | strftime("%Y-%m-%d %H:%M:%S")), "message": . }'
}

---

附录:

Harshit Kushwaha写道:

例如，我需要从调用json_logger的位置打印json中的方法名。如何修改json_logger以及如何在echo中使用它?

下面是添加方法名的实现，如果作为参数提供给json_logger函数:

#!/usr/bin/env bash
IFS= read -r -d '' __JQ_LOGGER_SCRIPT <<'JQSCRIPT'
{
"timestamp": now | strftime("%Y-%m-%d %H:%M:%S"),
"message": .
} |
if ($name | length) != 0
then
. + { "method": $name }
else
.
end
JQSCRIPT
json_logger() {
jq 
--raw-input 
--compact-output 
--arg name "$1" 
"$__JQ_LOGGER_SCRIPT"
}
echo "I want to be a json message!" | json_logger
echo "I also want to be a json message!" | json_logger echo
printf %s $'I am a message with "quoted text" and some special characters: 't742\'; that can only be properly converted to JSON with a JSON formatter and parser.' | json_logger printf

生成以下JSON输出:

{"timestamp":"2021-01-29 14:02:46","message":"I want to be a json message!"}
{"timestamp":"2021-01-29 14:02:46","message":"I also want to be a json message!","method":"echo"}
{"timestamp":"2021-01-29 14:02:46","message":"I am a message with "quoted text" and some special characters: 'tu0007"\'; that can only be properly converted to JSON with a JSON formatter and parser.","method":"printf"}

Answer 2

感谢您的解决方案。

我能够将脚本执行的日志写入日志文件，实现如下:

1. 创建json记录器

   json_logger() {
   # Values for json
   current_timestamp="$(date +"%Y-%m-%dT%H:%M:%S")"
   param1="$param1value"
   param2="$param2value"
   log_level=$1
   script_name=$(basename $2 .sh)
   # Create json and re-direct to log file
   jq --raw-input --compact-output 
   '{
   "@timestamp": '$current_timestamp',
   "parameter1": "'$param1'",
   "parameter2": "'$param2'",
   "log-level": "'$log_level'",
   "logger": "'$script_name'",
   "message": .
   }' >> /var/log/app.log 2>&1 
   }
   

2. 然后执行如下脚本

   "script.sh" 2>&1 | json_logger "INFO" {job_name or script_name}
   

Answer 3

./script.sh | awk -v squot="'" '{ print "{"timestamp": ""strftime("%Y-%m-%d %H:%M:%S")"", "message": ""$0""}" }' mess | jq >> jsonmessages.log

使用GNU awk，使用awk的strftime函数以所需格式打印时间戳，以及双引号和所需格式的其他数据。通过管道连接到jq以确保有效的json