三人行必有我师

Elasticsearch 内存泄漏 oom 没有bulk输入

Elasticsearch | 作者 caomaocao | 发布于2017年12月22日 | 阅读数:6358

ES版本: 5.2.2
正常运行已久,今天oom,停止了索引数据,一启动该节点,就有在warn日志输出的,old区占用,heap 占用越来越大,最终oom生成 .hprof。
 
[2017-12-22T17:56:03,337][WARN ][o.e.m.j.JvmGcMonitorService] [node-244] [gc][old][1456][82] duration [43.3s], collections [1]/[43.4s], total [43.3s]/[41.4m], memory [15.9gb]->[15.9gb]/[15.9gb], all_pools {[young] [532.5mb]->[532.5mb]/[532.5mb]}{[survivor] [65.6mb]->[66.4mb]/[66.5mb]}{[old] [15.3gb]->[15.3gb]/[15.3gb]}
[2017-12-22T17:56:03,337][WARN ][o.e.m.j.JvmGcMonitorService] [node-244] [gc][1456] overhead, spent [43.3s] collecting in the last [43.4s]
[2017-12-22T17:56:03,337][INFO ][o.e.d.z.ZenDiscovery     ] [node-244] master_left [{node-243}{Q1tAoYFyRXeCwS9cKnawAw}{xcezi1icSU2ZY5ZmUo7_GA}{10.0.0.243}{10.0.0.243:9300}], reason [failed to ping, tried [3] times, each with  maximum [30s] timeout]
[2017-12-22T17:56:45,838][WARN ][o.e.m.j.JvmGcMonitorService] [node-244] [gc][old][1457][84] duration [1.2m], collections [2]/[1.2m], total [1.2m]/[42.7m], memory [15.9gb]->[15.9gb]/[15.9gb], all_pools {[young] [532.5mb]->[532.5mb]/[532.5mb]}{[survivor] [66.4mb]->[66.3mb]/[66.5mb]}{[old] [15.3gb]->[15.3gb]/[15.3gb]}
[2017-12-22T17:56:45,838][WARN ][o.e.m.j.JvmGcMonitorService] [node-244] [gc][1457] overhead, spent [1.2m] collecting in the last [1.2m]
 
 
参考了https://elasticsearch.cn/question/3183, https://elasticsearch.cn/question/3183 和wu大的一些解答:
 
TIM截图20171222180636.png

 
 

GET /_cat/nodes?v&h=id,port,v,m,fdp,mc,mcs,sc,sm,qcm,fm,im,siwm,svmm

TIM截图20171222180859.png

 
/_cat/allocation?v&h=shards,disk.indices,disk.used,disk.avail
TIM截图20171222181001.png
已邀请:

要回复问题请先登录注册