用了Elasticsearch,一口气上5T

Elasticsearch关于unassigned shards修复

Elasticsearch | 作者 arterforyou | 发布于2017年12月23日 | | 阅读数:5850

ES 版本: 5.2.1

步骤:

  • curl localhost:9200/_cat/shards > shards
  • 跑脚本:nohup python recovery.py &

    注意:跑脚本过程会返回大量json,时间较长,请注意放入后台

  • 查看修复shard进度:curl 127.0.0.1:9200/_cat/recovery/你修复shard对应的索引
  • 结果: 找到索引对应的shard,看到existing_store done说明已经从本地修复
     index 19 268ms existing_store done n/a        n/a                    10.0.58.67 node_name
#!/usr/bin/env python
#name: recovery.py

import requests
import json
host = "http://localhost:9200/_cluster/allocation/explain"
s= requests.Session()
def reroute_shard(index,shard,node):
    data = {
    "commands" : [
        {
          "allocate_stale_primary" : {
              "index" : index, "shard" : shard, "node" : node, "accept_data_loss": True
          }
        }
    ]
   }
    print data
    url = "http://localhost:9200/_cluster/reroute"
    res = s.post(url,json=data)
    print res

def get_node(line):
    if "UNASSIGNED" in line:
        line = line.split()
        index = line[0]
        shard = line[1]
        if line[2] != "p":
            return
        body = {
           "index": index,
           "shard": shard,
           "primary": True
               }
        res = s.get(host, json = body)
        for store in res.json().get("node_allocation_decisions"):
            if store.get("store").get("allocation_id"):
               node_name = store.get("node_name")
               reroute_shard(index,shard,node_name)
    else:
        return

with open("shards", 'rb') as f:
    map(get_node,f)

相关文档:


[尊重社区原创,转载请保留或注明出处]
本文地址:http://elasticsearch.cn/article/431


2 个评论

对于磁盘上shard已经丢失的,是否可以将shard的删除?
https://www.elastic.co/guide/en/elasticsearch/reference/5.6/cluster-reroute.html 可以把某个shard指定到某个node,为空allocate_empty_primary

{
"commands" : [
{
"allocate_empty_primary" : {
"index" : "thz.api-2018.05.09", "shard" : 0,
"node" : "6ZFH1JX",
"accept_data_loss": true
}
}
]
}

curl -XPOST localhost:9200/_cluster/reroute -d @1.json

要回复文章请先登录注册