如何为log4j消息配置logstash过滤器

Question

我有一个mylog.log文件，它有以下内容:

2014-07-02 20:52:39 DEBUG HelloExample:19 - This is debug : ishan
2014-07-02 20:52:39 INFO  HelloExample:23 - This is info : ishan
2014-07-02 20:52:39 WARN  HelloExample:26 - This is warn : ishan
2014-07-02 20:52:39 ERROR HelloExample:27 - This is error : ishan
2014-07-02 20:52:39 FATAL HelloExample:28 - This is fatal : ishan

对于logstash

，我有以下配置文件

input {
  file {
    path => "/home/ishan/sf_shared/log4jlogs/1-7-2016.txt"
    start_position => "beginning"
    codec => multiline {
      pattern => "(^d+serror)|(^.+Exception: .+)|(^s+at .+)|(^s+... d+ more)|(^s*Caused by:.+)"
      what => "previous"
    }
  }
}
output {
  elasticsearch => ["127.0.0.1:9200"]
  stdout { codec => rubydebug }
}

所以，当我用下面的命令运行logstash时/opt/logstash/bin/logstash -f log4j.conf

但是它在控制台上没有显示任何东西。那么，如何在主机上显示它呢?也添加后如何检查ES?并且需要什么样的过滤器来解析以下json格式的信息?

{
  "severity": DEBUG
  "class": HelloExample
  "message":This is debug : ishan
}

Answer 1

您需要查看grok模式。点击这个链接。这应该对你有用:

input {
  file {
    path => "/home/ishan/sf_shared/log4jlogs/1-7-2016.txt"
    start_position => "beginning"
  }
}
filter {
    grok {
        match => {
            "message" => '%{YEAR}[-]%{MONTHNUM}[-]%{MONTHDAY} %{TIME} %{LOGLEVEL:severity} %{GREEDYDATA:class}%{INT} - %{GREEDYDATA:mess}'
        }   
    }
}
output {
  elasticsearch => ["127.0.0.1:9200"]
  stdout { codec => rubydebug }
}