无法正确使用grok过滤器从我的日志文件中找到有用的信息，以便在弹性搜索中推送:logstash

Question

这是我第一次使用grok filter，我想从日志中提取有用的密钥。我无法创建grok模式。

我有access.log文件，我使用logstash将日志数据放在elsticsearch中。

样本日志

September 6th 2020, 10:30:37:759 am [f657a07c-8c62-46ee-78a3-e7c2a04328b0] info: authentication successful...
September 6th 2020, 10:30:38:179 am [f657a07c-8c62-46ee-78a3-e7c2a04328b0] info: external apiresponse time: 3 ms
September 6th 2020, 10:30:49:200 am [f657a07c-8c62-46ee-78a3-e7c2a04328b0] info: {"path":"/route1","originalUrl":"/media/auth/route1?id=601&userId=9e2681-a188-4978-94b0-04c1d50ad056","timestamp":1599368449200,"uuid":"f657a07c-8c62-46ee-88a3-e7c3a04328b0","responseCode":"200","responseTime":11455,"externalCalls":[{"type":"apis","status":"success","timeTaken":2167},{"type":"pdf-generate-time","status":"success","timeTaken":5302},{"type":"pdf-upload-s3","status":"success","timeTaken":1146}]}

我只想处理{"路径"：…….}这个项目到弹性搜索，并提取所有键作为弹性中的一个字段。

我的logstash conf文件-

{
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log-level} [%{DATA:class}]:%{GREEDYDATA:message}" }
}
}

Answer 1

如果日志格式始终相同，则可以使用以下模式：info: %{GREEDYDATA:text}$

在您的示例中，它将创建一个名为text的字段，其值如下：

authentication successful...
external apiresponse time: 3 ms
{"path":"/route1","originalUrl":"/media/auth/route1?id=601&userId=9e2681-a188-4978-94b0-04c1d50ad056","timestamp":1599368449200,"uuid":"f657a07c-8c62-46ee-88a3-e7c3a04328b0","responseCode":"200","responseTime":11455,"externalCalls":[{"type":"apis","status":"success","timeTaken":2167},{"type":"pdf-generate-time","status":"success","timeTaken":5302},{"type":"pdf-upload-s3","status":"success","timeTaken":1146}]}

然后您可以使用json过滤器来解析它