无法使用日志筛选器解析 XML 输入

Question

嗨，我正在尝试解析以下xml：

<msg time='2014-08-04T14:36:02.136+03:00' org_id='oracle' comp_id='rdbms'
 msg_id='opistr_real:953:3971575317' type='NOTIFICATION' group='startup'
 level='16' host_id='linux4_l' host_addr='127.0.0.1'
 pid='8986' version='1'>
 <txt>Starting ORACLE instance (normal)
 </txt>
</msg>

使用此配置：

 input {
   stdin {
    type => "stdin-type"
  }
  }
 filter { multiline {
                       pattern => "^s|</msg>|^[A-Za-z].*"
                        what => "previous"
                }
                xml {
                        store_xml => "false"
                        source => "message"
                        xpath => [
                                "/msg/@client_id", "msg_client_id",
                                "/msg/@host_id", "msg_host_id",
                                "/msg/@host_addr", "msg_host_addr",
                                "/msg/@level", "msg_level",
                                "/msg/@module", "msg_module",
                                "/msg/@msg_id", "msg_msg_id",
                                "/msg/@pid", "msg_pid",
                                "/msg/@org_id", "msg_org_id",
                                "/msg/@time", "msg_time",
                                "/msg/@level", "msg_level",
                                "/msg/txt/text()","msg_txt"
                        ]
               }
                date {
                        match => [ "msg_time", "ISO8601" ]
                }
                mutate {
                        add_tag => "%{type}"
                }
}
output { elasticsearch { host => localhost } stdout { codec => rubydebug } }

但是当我运行logstash时，我会得到以下错误：

{:timestamp=>"2014-09-04T17:28:39.428000+0300", :message=>"Exception in filterworker", "exception"=>#<NoMethodError: undefined method `split' for ["msg_level", "msg_level"]:Array>, "backtrace"=>["/opt/logstash/lib/logstash/util/accessors.rb:19:in `parse'", "/opt/logstash/lib/logstash/util/accessors.rb:15:in `get'", "/opt/logstash/lib/logstash/util/accessors.rb:59:in `store_path'", "/opt/logstash/lib/logstash/util/accessors.rb:55:in `lookup'", "/opt/logstash/lib/logstash/util/accessors.rb:34:in `get'", "/opt/logstash/lib/logstash/event.rb:127:in `[]'", "/opt/logstash/lib/logstash/filters/xml.rb:117:in `filter'"

"/opt/logstash/lib/logstash/ppipeline.rb:143:在`start_filters'"]中，：level=>：error｝｛：timestamp=>"2014-09-04T17:30:47805000+0300"，：message=>"收到中断。正在关闭管道。"，：level=>：warn｝

Answer 1

我发现了我的问题，我在xpath上重复了解析/msg@levelapper两次。

Answer 2

multiline编解码器不太适合这种类型的文件，但您可以使用以下内容：

multiline {
      pattern => '<msg'
      negate => true
      what => previous
}

它的问题是，文件中的最后一个事件直到下一个事件出现才熄灭（因此您最终会丢失文件中的上一个事件）。

Answer 3

它的问题是，文件中的最后一个事件直到下一个事件出现才熄灭（因此您最终会丢失文件中的上一个事件）。

最好匹配结束标记。

multiline {
    pattern => "</msg>$"
    negate => true
    what => next
}