是时候用 ES 拯救发际线啦

[operation indices:monitor/stats failed]; nested: IndexNotFoundException[no such index]

匿名 | 发布于2017年08月31日 | 阅读数:12057

重启es发现大量这样的错误,索引一直没法恢复,求高手解惑[2017-08-31 18:20:31,988][DEBUG][action.admin.indices.stats] [es2-hot] [indices:monitor/stats] failed to execute operation for shard [[fy3c-lsfinfo-20170829][4], node[IX0iasV_QdGOPeJ1Hr70_Q], [P], v[24], s[STARTED], a[id=_TbrvfmOSFKqH-cPkrxQKQ]]
[fy3c-lsfinfo-20170829][[fy3c-lsfinfo-20170829][4]] BroadcastShardOperationFailedException[operation indices:monitor/stats failed]; nested: IndexNotFoundException[no such index];
        at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.onShardOperation(TransportBroadcastByNodeAction.java:399)
        at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:376)
        at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:365)
        at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:299)
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: [fy3c-lsfinfo-20170829] IndexNotFoundException[no such index]
        at org.elasticsearch.indices.IndicesService.indexServiceSafe(IndicesService.java:295)
        at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:97)
        at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:47)
        at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.onShardOperation(TransportBroadcastByNodeAction.java:395)
已邀请:

guopeng7216 - 90后运维

赞同来自:

请问你的问题解决了吗?我出先的问题和你类似,从5.4.1升级到6.1.0 现在是任何节点重启,都很难加入集群,一会加入一会会出来的,不知什么原因

[2018-03-06T17:06:26,222][INFO ][o.e.c.s.MasterService    ] [ES-node2-Prod-alidc-0gow] zen-disco-node-failed({ES-node4-Prod-alidc-0gow}{ALFiaxulQ9av0F4Ld-F4lw}{ECPEE8BtQROudn9Z2G_BNQ}{172.19.6.127}{172.19.6.127:9300}), reason(transport disconnected)[{ES-node4-Prod-alidc-0gow}{ALFiaxulQ9av0F4Ld-F4lw}{ECPEE8BtQROudn9Z2G_BNQ}{172.19.6.127}{172.19.6.127:9300} transport disconnected], reason: removed {{ES-node4-Prod-alidc-0gow}{ALFiaxulQ9av0F4Ld-F4lw}{ECPEE8BtQROudn9Z2G_BNQ}{172.19.6.127}{172.19.6.127:9300},}
[2018-03-06T17:06:26,222][DEBUG][o.e.a.a.i.s.TransportIndicesStatsAction] [ES-node2-Prod-alidc-0gow] failed to execute [indices:monitor/stats] on node [ALFiaxulQ9av0F4Ld-F4lw]
org.elasticsearch.transport.NodeNotConnectedException: [ES-node4-Prod-alidc-0gow][172.19.6.127:9300] Node not connected
at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:692) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.transport.TcpTransport.getConnection(TcpTransport.java:122) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.transport.TransportService.getConnection(TransportService.java:525) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:488) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction.sendNodeRequest(TransportBroadcastByNodeAction.java:322) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction.start(TransportBroadcastByNodeAction.java:311) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction.doExecute(TransportBroadcastByNodeAction.java:234) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction.doExecute(TransportBroadcastByNodeAction.java:79) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1253) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.stats(AbstractClient.java:1574) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.cluster.InternalClusterInfoService.updateIndicesStats(InternalClusterInfoService.java:268) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.cluster.InternalClusterInfoService.refresh(InternalClusterInfoService.java:319) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.cluster.InternalClusterInfoService.maybeRefresh(InternalClusterInfoService.java:275) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.cluster.InternalClusterInfoService.lambda$clusterChanged$1(InternalClusterInfoService.java:178) ~[elasticsearch-6.1.0.jar:6.1.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:568) [elasticsearch-6.1.0.jar:6.1.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
 

要回复问题请先登录注册