ElasticSearch 定期删除过期索引

索引管理工具 curator

安装 curator

1
pip install elasticsearch-curator

curator 命令

1
curator [--config CONFIG.YML] [--dry-run] ACTION_FILE.YML

需要配置两个文件 CONFIG.YMLACTION_FILE.YML, 若果不设置 --config CONFIG.YML 则读取默认路径配置 ~/.curator/curator.yml--dry-run 为调试模式,模拟删除,不做真实处理

下面是两个配置文件的具体内容

config.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
client:
hosts:
- 127.0.0.1
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout:
master_only: True

logging:
loglevel: INFO
logfile:
logformat: default

action.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
---
actions:
1:
action: delete_indices
description: 'Base on index name, delete indices which prefix is logstash-android- and is older than 90 days. ref: https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html'
options:
ignore_empty_list: True
timeout_override: 3600
continue_if_exception: False
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: logstash-android-
exclude:
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 90
exclude:

定期任务工具 crontab

1.定期任务需要 crontab 命令,执行

1
crontab -e 

2.编写脚本任务 clean_expired_indices.sh

1
2
3
4
cd `dirname $0`
echo `pwd`
# curator --config config_file.yml --dry-run action_file.yml
curator --config config_file.yml action_file.yml
  1. 进入 vi 界面后,添加一行命令,每天凌晨 2 点执行清理脚本,保存退出
1
0 2 * * *  /bin/sh /path/to/clean_expired_indices.sh >> /var/log/clean_expired_indices.log
  1. 查看是否生效
1
crontab -l