Filebeat无法连接到Amazon Elasticsearch服务

Question

我们试图通过文件节拍连接到AWS ES，但是，出现以下错误

pipeline/output.go:100  Failed to connect to backoff(elasticsearch(https://xxx.us-east-1.es.amazonaws.com:443)): 
Connection marked as failed because the onConnect callback failed: 
cannot retrieve the elasticsearch license: unauthorized access, 
could not connect to the xpack endpoint, verify your credentials

配置文件为

filebeat.yml: |-
filebeat.config:
inputs:
enabled: true
path: ${path.config}/inputs.d/*.yml
reload.enabled: true
reload.period: 60s
modules:
enabled: true
path: ${path.config}/modules.d/*.yml
reload.enabled: true
reload.period: 60s
output.elasticsearch:
protocol: "https"
hosts: ["xxxx.us-east-1.es.amazonaws.com:443"]
username: "user_id"
password: "password"
index: "test-log--%{+yyyy.MM.dd}"
setup.template.name: "test-log"
setup.template.pattern: "test-log-*"
setup.ilm.enabled: false

filebeat docker图像为docker.elastic.co/beats/filebeat:7.13.1

亚马逊Elasticsearch服务的ES版本是7.10

有人能帮我们在这里找到无效的配置吗？

感谢

Answer 1

有两个问题

1. 对于节拍7.13

Beats可能没有将数据发送到ElasticSearchdit 的某些分发版

我们可以使用与aws es7.10.2相同的版本filebeat-oss:7.10.2

2. 需要禁用的X-Pack设置

setup.ilm.enabled: false
setup.pack.security.enabled: false
setup.xpack.graph.enabled: false
setup.xpack.watcher.enabled: false
setup.xpack.monitoring.enabled: false
setup.xpack.reporting.enabled: false

参考编号：https://aws.amazon.com/premiumsupport/knowledge-center/es-connect-filebeat-logstash-linux/