为fluentd创建正则表达式模式

Question

我使用Fluentd配置和regexp解析器来解析日志。

这是我在fluentd:中的原始日志

2022-09-22 18:15:09,633 [springHikariCP housekeeper    ] DEBUG HikariPool                     - springHikariCP - Fill pool skipped, pool is at sufficient level.
2022-09-22 18:15:14,968 [ringHikariCP connection closer] DEBUG PoolBase                       - springHikariCP - Closing connection com.mysql.cj.jdbc.ConnectionImpl@7f535ea4: (connection has passed maxLifetime)

我想用regexp为上面的日志创建json格式，如下所示：

{
"logtime": "2022-09-22 18:15:09,633",
"Logger Name": "[springHikariCP housekeeper    ]",
"Log level": "DEBUG",
"message": "HikariPool                     - springHikariCP - Fill pool skipped, pool is at sufficient level"
}
{
"logtime": "2022-09-22 18:15:09,633",
"Logger Name": "[ringHikariCP connection closer]",
"Log level": "DEBUG",
"message": "PoolBase                       - springHikariCP - Closing connectioncom.mysql.cj.jdbc.ConnectionImpl@7f535ea4: (connection has passed maxLifetime)"
}

有Ruby经验的人能帮我为以上的日志创建Ruby正则表达式吗

Answer 1

正则表达式应该很简单，例如

(?'logtime'd{4}-d{2}-d{2}sd{2}:d{2}:d{2},d{3})s(?'logname'[[^]]*])s(?'loglevel'w+)s(?'message'.*)

在线测试

然后，使用替换(gsub，见下面的示例(或将零件中的新字符串组合在一起(match(来重新组装零件：

re = /(?'logtime'd{4}-d{2}-d{2}sd{2}:d{2}:d{2},d{3})s(?'logname'[[^]]*])s(?'loglevel'w+)s(?'message'.*?$)/m
str = '2022-09-22 18:15:09,633 [springHikariCP housekeeper    ] DEBUG HikariPool                     - springHikariCP - Fill pool skipped, pool is at sufficient level.
2022-09-22 18:15:14,968 [ringHikariCP connection closer] DEBUG PoolBase                       - springHikariCP - Closing connection com.mysql.cj.jdbc.ConnectionImpl@7f535ea4: (connection has passed maxLifetime)'
str.gsub(re) do |match|
puts "{
"logtime": "#{$~[:logtime]}",
"Logger Name": "#{$~[:logname]}",
"Log level": "#{$~[:loglevel]}",
"message": "#{$~[:message]}"
}"
end