放弃建议https://blog.codecentric.de/en/2014/10/log-management-spring-boot-applications-logstash-elastichsearch-kibana/我设置了logstash编码器+logstash转发器,将所有内容推送到我的logstash deamon,并最终在ElasticSearch中索引所有内容。
这是我的配置:
logstash.xml
<included>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<property name="FILE_LOGSTASH" value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}/}spring.log}.json"/>
<appender name="LOGSTASH"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
<file>${FILE_LOGSTASH}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${FILE_LOGSTASH}.%i</fileNamePattern>
</rollingPolicy>
<triggeringPolicy
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<MaxFileSize>10MB</MaxFileSize>
</triggeringPolicy>
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<includeCallerInfo>true</includeCallerInfo>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="LOGSTASH"/>
</root>
</included>
logstash-fowarder.conf
{
"network": {
"servers": [
"logstash:5043"
],
"ssl certificate": "/etc/pki/tls/certs/logstash-forwarder/logstash-forwarder.crt",
"ssl key": "/etc/pki/tls/private/logstash-forwarder/logstash-forwarder.key",
"ssl ca": "/etc/pki/tls/certs/logstash-forwarder/logstash-forwarder.crt",
"timeout": 15
},
"files": [
{
"paths": [
"${ENV_SERVICE_LOG}/*.log.json"
],
"fields": {
"type": "${ENV_SERVICE_NAME}"
}
}
]
}
的logstash.conf
input {
lumberjack {
port => 5043
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder/logstash-forwarder.key"
}
}
output {
elasticsearch { host => "localhost" }
}
一切正常,日志保存在ElasticSearch中。
在这一点上,我希望能够指定ElasticSearch索引的其他字段,例如日志级别。在@message内容中搜索是否存在"错误"或"警告"并没有多大用处。
我该怎么做?我应该更改哪个配置以使级别在ElasticSearch中显示为索引字段?
您想要的是一个logstash过滤器,它将在索引器上用作输入和输出节的对等。
有很多过滤器(请参阅文档(,但您可以使用grok{}将regexp应用于消息字段并提取日志级别。
您没有包含示例消息,但是,给定一个类似"foo 123 bar"的字符串,该模式将把"123"提取到一个名为loglevel:的整数字段中
grok {
match => ["message", "foo %{NUMBER:loglevel:int} bar"]
}
关于在网络上写grok模式,有相当多的信息。试试这个。
logstash config file:
input {
file {
path => [ "/tmp/web.log" ]
}
}
filter {
grok {
match => [ "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:severity} %{GREEDYDATA:message}" ]
}
}
output {
elasticsearch {
host => "127.0.0.1"
index => "web-%{+YYYY.MM.dd}"
}
}
您可以使用"add_tag"或"add_field"指定特殊字段