logstash和json样式日志从log4net

Question

给定以下配置：

input {
    udp {
            type=> "genericJson2"
            port => 9994
    }
}
filter{
    if [type] == "genericJson2" {
            json {
                    source => "message"
            }
    }
}
output {
    elasticsearch{
    }
}

以下输入：

{"date":"2018-02-27T13:21:41.3387552-05:00","level":"INFO","appname":"listenercore","logger":"Main","thread":"1","message":"test"}

我得到以下结果：

{
  "_index": "logstash-2018.02.27",
  "_type": "doc",
  "_id": "AWHYfh_qDl_9h030IXjC",
  "_score": 1,
  "_source": {
    "type": "genericJson2",
    "@timestamp": "2018-02-27T18:19:59.747Z",
    "host": "10.120.4.5",
    "@version": "1",
    "date": "{"date":"2018-02-27T13:20:02.2113",
    "message": "{"da",
    "logger": "{"da",
    "thread": "{",
    "level": "{"da",
    "appname": "{"date":"201"
  },
  "fields": {
    "@timestamp": [
      "2018-02-27T18:19:59.747Z"
    ]
  }
}

我需要做什么才能正确解析我的JSON日志？

编辑

我深入挖了一点。从CMD线运行此

sudo bin/logstash -e "input{stdin{type=>stdin}} filter{json {source=>message}} output{ stdout{ codec=>rubydebug } }"

产生了所需的输出

{
    "@timestamp" => 2018-02-28T02:07:01.710Z,
          "host" => "Elastisearch01",
       "appname" => "listenercore",
        "logger" => "Main",
      "@version" => "1",
          "type" => "stdin",
          "date" => "2018-02-27T13:21:41.3387552-05:00",
         "level" => "INFO",
        "thread" => "1",
       "message" => "test"
}

所以我写了一台快速的python udp服务器来查看跨过电线的内容，这是我捕获的内容：

{ " d a t e " : " 2 0 1 8 - 0 2 - 2 7 T 2 1 : 0 6 : 0 4 . 7 4 6 1 3 4 6 - 0 5 : 0 0 " , " l e v e l " : " I N F O " , " a p p n a m e " : " l i s t e n e r c o r e " , " l o g g e r " : " M a i n " , " t h r e a d " : " 1 " ,   m e s s a g e " : " t e s t " }

每个字符之间都有额外的空间，我正在调查文本编码，但我不确定。

编辑

很好地证明了这是一个编码问题。如果我使用此python脚本捕获，解码和重新启动日志，它将解决问题：

import socket
UDP_IP_ADDRESS = "10.254.18.166"
UDP_PORT_NO = 9993
serverSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
serverSock.bind((UDP_IP_ADDRESS, UDP_PORT_NO))
clientSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
while True:
    data, addr = serverSock.recvfrom(1024)
    clientSock.sendto(data.decode('utf-16'), ("127.0.0.1", 9994))

如何获得logstash到达我的UTF-16输入？我已经尝试过，它不起作用：

bin/logstash -e "input{udp{port=>9994 type=>stdin codec=>plain{charset=>'UTF-16'}}} filter{json {source=>message}} output{ stdout{ codec=>rubydebug } }"

Answer 1

您可以使用UTF-16LE而不是UTF-16？