获取elasticsearch-py的解析错误

Question

我试图在一个时间范围内搜索某个单词"tsbu"的整个elasticsearch数据。当我尝试运行这个，我得到一个SearchParseException和解析失败。

es = Elasticsearch()
doc = {
        "query": {
            "match" : { 
                "message" : "tsbu"
            }
        }, 
        "range" : { 
            "@timestamp" : { 
                "gte" : "2015-06-09T14:44:00.000Z", 
                "lte" : "2015-06-09T14:50:00.000Z"
            }
        }
    }
print es.search(index="_all", body=doc)

我得到的完整错误是:

Traceback (most recent call last):
  File "essearch.py", line 22, in <module>
print es.search(index="_all", body=doc)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 504, in search
params=params, body=body)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 307, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 89, in perform_request
self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[mPhuId4qSpa5osrqfeG5Tw][.kibana][0]: SearchParseException[[.kibana][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[.kibana][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][0]: SearchParseException[[logstash-2015.06.08][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][0]: SearchParseException[[logstash-2015.06.09][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][0]: SearchParseException[[logstash-2015.06.10][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][1]: SearchParseException[[logstash-2015.06.08][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][1]: SearchParseException[[logstash-2015.06.09][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][1]: SearchParseException[[logstash-2015.06.10][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][2]: SearchParseException[[logstash-2015.06.08][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][2]: SearchParseException[[logstash-2015.06.09][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][2]: SearchParseException[[logstash-2015.06.10][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][3]: SearchParseException[[logstash-2015.06.08][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][3]: SearchParseException[[logstash-2015.06.09][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][3]: SearchParseException[[logstash-2015.06.10][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][4]: SearchParseException[[logstash-2015.06.08][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][4]: SearchParseException[[logstash-2015.06.09][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][4]: SearchParseException[[logstash-2015.06.10][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }]')

Answer 1

你的查询基本正确。你得到的错误状态是...Parse Failure [Failed to parse source...，这基本上意味着你的查询是病态的，不符合查询DSL。

range查询需要与match查询组合(使用bool/must查询)，并且两者都需要嵌套在query中，像这样

doc = {
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "message": "tsbu"
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2015-06-09T14:44:00.000Z",
              "lte": "2015-06-09T14:50:00.000Z"
            }
          }
        }
      ]
    }
  }
}

最终结果是，您需要tsbu匹配message字段和， @timestamp在指定的日期范围内。

为了让message匹配tsbu或host匹配vcs，但仍然需要timestamp匹配，您可以这样做:

doc = {
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "message": "tsbu"
          }
        },
        {
          "match": {
            "host": "vcs"
          }
        }
      ],
      "must": [
        {
          "range": {
            "@timestamp": {
              "gte": "2015-06-09T14:44:00.000Z",
              "lte": "2015-06-09T14:50:00.000Z"
            }
          }
        }
      ]
    }
  }
}