添加json转义字符的异常

Question

我在谷歌上搜索了这个问题的解决方案，并添加了Ruby代码。问题是，如果我使用gsub，我也会得到一个错误。

我需要一个解决方案来修复Ruby错误或另一种方法来添加转义字符的异常。

这是我的Logstash过滤器:

filter {
  if [type] == "apache" {
    json {
          source => "message"
        }
    ruby {
          code => "str=event['url'];  str=str.gsub('\','*'); event['url']=str;"
        }
 }
}

我得到我的日志从Apache json格式。下面是Logstash的错误消息:

 {:timestamp=>"2016-06-28T12:49:21.821000+0100", :message=>"Error
 parsing json", :source=>"message", :raw=>"{ "@version": "1",
 "@timestamp": "2016-06-28T12:49:16.000+0100", "message": "GET
 /yolo/q-ap%C3%Ablah/?search%5Bdist%5D=15&page=13 HTTP/1.1", "via":
 "192.168.220.100", "client-ip": "123.123.123.123",
 "remote-logname": "-", "remote-user": "-", "recv-time":
 "[28/Jun/2016:12:49:16 +0100]", "serve-time-microsec": "88613",
 "request": "GET /yolo/q-ap%C3%Ablah/?search%5Bdist%5D=15&page=13
 HTTP/1.1", "status": "200", "size": "184985", "referer":
 "-", "user-agent": "Mozilla/5.0 (compatible; Googlebot/2.1;
 +http://www.google.com/bot.html)", "url": "/yolo/q-ap\xc3\xablah/", "query":
 "?search%5Bdist%5D=15&page=13", "method": "GET", "protocol":
 "HTTP/1.1", "vhost": "www.site.com", "received-size": "1136"
 }", :exception=>#LogStash::Json::ParserError: Unrecognized character
 escape 'x' (code 120) at Source: [B@2ebf051e; line: 1, column: 599>,
 :level=>:warn}
 {:timestamp="2016-06-28T12:49:21.821000+0100", :message=>"Ruby
 exception occurred: undefined method `gsub' for nil:NilClass",
 :level=>:error}

Answer 1

多亏了logstash团队，我才能够解决我的问题。https://discuss.elastic.co/t/add-execption-for-special-character-from-json-log/54285/3

下面是正确的配置:

filter {
  if [type] == "apache" {
    ruby {
        code => " if event['message']
                  event['message'] = event['message'].gsub('x','Xx')
                  event['message'] = event['message'].gsub('\x','XXx')
                  end
                "
            }
    json {
          source => "message"
        }
 }
}

Answer 2

Logstash 5在使用event['message']方面有一个突破性的变化。https://github.com/logstash-plugins/logstash-filter-aggregate/issues/48 issuecomment - 258297534

为我工作的配置是:

ruby {
  code => " if event.get('message')
               event.set('message', event.get('message').gsub('x','Xx'))
               event.set('message', event.get('message').gsub('\x','XXx'))
            end
          "
}