如何自动删除AWS中超过1个月的弹性搜索记录

Question

我有一个功能，我必须删除超过1个月的弹性搜索记录。我可以通过让cron作业在弹性搜索上运行删除查询来实现这一点，但我想自动做到这一点。

就像AWS中的S3文件一样，我可以设置TTL。类似地，在弹性搜索中寻找类似_ttl的东西>7.1

Answer 1

您可以使用专门为考虑这些用例而设计的索引生命周期管理选项。

它也是基本许可证的一部分，所以你不必购买它，请参阅弹性订阅了解更多详细信息。

Answer 2

对于AWS管理的ElasticSearch，您可以使用索引状态管理(ISM(：https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ism.html

Answer 3

Posting the complete answer:
1. Create the policy to rollover based on max_age and delete after 10 mins
policy = {
"policy": {
"phases": {
"hot": {
"actions": {
"rollover": {
"max_age": "11m"
}
}
},
"delete": {
"min_age": "10m",
"actions": {
"delete": {}
}
}
}
}
}
2. Insert the policy:
IlmClient.put_lifecycle(es, "datastream_policy", policy)
3. Create index template to apply policy to all rollover index being created so that they will be deleted after 10 mins:
template = {
"index_patterns": ["datastream-*"],
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0,
"index.lifecycle.name": "datastream_policy",
"index.lifecycle.rollover_alias": "datastream"
},
"mappings": {
"email": "keyword"
}
}
4. Put the index template in elastic search
es.indices.put_template(name="datastream_template", body=json.dumps(template))
5. Create a starter index:
indexd = {
"aliases": {
"datastream": {
"is_write_index": True
}
}
}
es.indices.create("datastream-000001", body=indexd)