Grok 与日期'13 Oct 2015 21:30:26'的时间戳不匹配

Question

我已经尝试了几个小时了，但是grok就是不想正确解析时间戳:

消息:

Tue, 13 Oct 2015 21:30:26 GMT users_service Three users logged in.

. conf文件:

input { stdin { } }
filter {
  grok {
    match => { "message" => "%{DAY:day}, %{MONTHDAY:month_day} %{MONTH:month} %{YEAR:year} %{TIME:time} GMT %{WORD:service} %{GREEDYDATA:message_entry}" }
    add_field => [ "received_at", "%{@timestamp}" ]
    add_field => [ "received_from", "%{host}" ]
  }
  date {
    match => [ "timestamp", "dd MMM yyyy HH:mm:ss" ]
    locale => "en"
  }
}
output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

我得到的是:

{
          "message" => "Tue, 13 Oct 2015 21:30:26 GMT users_service Three users logged in.",
         "@version" => "1",
       "@timestamp" => "2015-10-13T23:09:58.738Z",
             "host" => "users_host",
              "day" => "Tue",
        "month_day" => "13",
            "month" => "Oct",
             "year" => "2015",
             "time" => "21:30:26",
          "service" => "users_service",
    "message_entry" => "Three users logged in.",
      "received_at" => "2015-10-13T23:09:58.738Z",
    "received_from" => "users_host"
}

我希望在那里有一个timestamp字段，但是没有。

Answer 1

可以看到，grok{}正在为您创建几个字段。Date{}需要一个单独的字段和该字段的格式，但是在配置中没有将它们组合成一个可以使用的单个'时间戳'字段。

你有两个选择:做一个不同的grok模式，把所有的日期/时间的东西放在一个字段，然后传递(与正确的模式信息)到date{}，或使用add_field从所有的片段，你有一个新的字段

Answer 2

您已配置的date过滤器期望记录中有"timestamp"字段。当前您的记录不包含具有该名称和预期内容的字段。

您可以使用您在消息和add_field设置中匹配的数据在grok过滤器中创建所需的字段，然后在日期过滤器成功时删除临时字段:

filter {
  grok {
    match => {
      "message" => "%{DAY:day}, %{MONTHDAY:month_day} %{MONTH:month} %{YEAR:year} %{TIME:time} GMT %{WORD:service} %{GREEDYDATA:message_entry}"
    }
    add_field => {
      "received_at" => "%{@timestamp}"
      "received_from" => "%{host}"
      "timestamp" => "%{month_day} %{month} %{year} %{time}"
    }
  }
  date {
    match => [ "timestamp", "dd MMM yyyy HH:mm:ss" ]
    locale => "en"
    remove_field => [ "timestamp" ]
  }
}