使用RestHighLevelClient获取Cluster setting信息BLOCK死锁【已解决】
Elasticsearch | 作者 yangjianxuan | 发布于2023年05月18日 | 阅读数:2956
大家好:
有个问题请教一下我使用elasticsearch-rest-high-level-client-6.8.17.jar中提供的RestHighLevelClient获取Cluster setting信息时,线程死锁。
跟踪了一下源码,毫无头绪,请问一下大家有没有什么排查思路可以借鉴。
调用代码如下:
Thread 204677: (state = BLOCKED)
- org.elasticsearch.action.admin.cluster.settings.ClusterGetSettingsResponse.lambda$static$1(org.elasticsearch.common.xcontent.XContentParser, java.lang.Void) @bci=1, line=62 (Interpreted frame)
- org.elasticsearch.action.admin.cluster.settings.ClusterGetSettingsResponse$$Lambda$785.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=5 (Interpreted frame)
- org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareObject$1(org.elasticsearch.common.xcontent.ContextParser, org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=3, line=146 (Interpreted frame)
- org.elasticsearch.common.xcontent.AbstractObjectParser$$Lambda$786.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=6 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser.lambda$declareField$1(java.util.function.BiConsumer, org.elasticsearch.common.xcontent.ContextParser, org.elasticsearch.common.xcontent.XContentParser, java.lang.Object, java.lang.Object) @bci=6, line=213 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser$$Lambda$788.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object, java.lang.Object) @bci=11 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser.parseValue(org.elasticsearch.common.xcontent.XContentParser, org.elasticsearch.common.xcontent.ObjectParser$FieldParser, java.lang.String, java.lang.Object, java.lang.Object) @bci=9, line=314 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser.parseSub(org.elasticsearch.common.xcontent.XContentParser, org.elasticsearch.common.xcontent.ObjectParser$FieldParser, java.lang.String, java.lang.Object, java.lang.Object) @bci=80, line=326 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object, java.lang.Object) @bci=237, line=168 (Interpreted frame)
- org.elasticsearch.common.xcontent.ConstructingObjectParser.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=16, line=169 (Interpreted frame)
- org.elasticsearch.common.xcontent.ConstructingObjectParser.apply(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=3, line=161 (Interpreted frame)
- org.elasticsearch.action.admin.cluster.settings.ClusterGetSettingsResponse.fromXContent(org.elasticsearch.common.xcontent.XContentParser) @bci=5, line=143 (Interpreted frame)
- org.elasticsearch.client.ClusterClient$$Lambda$775.apply(java.lang.Object) @bci=4 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.parseEntity(org.apache.http.HttpEntity, org.elasticsearch.common.CheckedFunction) @bci=164, line=2061 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$13(org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.Response) @bci=6, line=1698 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient$$Lambda$751.apply(java.lang.Object) @bci=12 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(java.lang.Object, org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.RequestOptions, org.elasticsearch.common.CheckedFunction, java.util.Set) @bci=93, line=1782 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.performRequest(org.elasticsearch.action.ActionRequest, org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.RequestOptions, org.elasticsearch.common.CheckedFunction, java.util.Set) @bci=35, line=1735 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(org.elasticsearch.action.ActionRequest, org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.RequestOptions, org.elasticsearch.common.CheckedFunction, java.util.Set) @bci=14, line=1697 (Interpreted frame)
- org.elasticsearch.client.ClusterClient.getSettings(org.elasticsearch.action.admin.cluster.settings.ClusterGetSettingsRequest, org.elasticsearch.client.RequestOptions) @bci=19, line=116 (Interpreted frame)
有个问题请教一下我使用elasticsearch-rest-high-level-client-6.8.17.jar中提供的RestHighLevelClient获取Cluster setting信息时,线程死锁。
跟踪了一下源码,毫无头绪,请问一下大家有没有什么排查思路可以借鉴。
调用代码如下:
return restHighLevelClient.cluster().getSettings(clusterGetSettingsRequest, RequestOptions.DEFAULT);
堆栈信息如下:Thread 204677: (state = BLOCKED)
- org.elasticsearch.action.admin.cluster.settings.ClusterGetSettingsResponse.lambda$static$1(org.elasticsearch.common.xcontent.XContentParser, java.lang.Void) @bci=1, line=62 (Interpreted frame)
- org.elasticsearch.action.admin.cluster.settings.ClusterGetSettingsResponse$$Lambda$785.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=5 (Interpreted frame)
- org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareObject$1(org.elasticsearch.common.xcontent.ContextParser, org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=3, line=146 (Interpreted frame)
- org.elasticsearch.common.xcontent.AbstractObjectParser$$Lambda$786.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=6 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser.lambda$declareField$1(java.util.function.BiConsumer, org.elasticsearch.common.xcontent.ContextParser, org.elasticsearch.common.xcontent.XContentParser, java.lang.Object, java.lang.Object) @bci=6, line=213 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser$$Lambda$788.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object, java.lang.Object) @bci=11 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser.parseValue(org.elasticsearch.common.xcontent.XContentParser, org.elasticsearch.common.xcontent.ObjectParser$FieldParser, java.lang.String, java.lang.Object, java.lang.Object) @bci=9, line=314 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser.parseSub(org.elasticsearch.common.xcontent.XContentParser, org.elasticsearch.common.xcontent.ObjectParser$FieldParser, java.lang.String, java.lang.Object, java.lang.Object) @bci=80, line=326 (Interpreted frame)
- org.elasticsearch.common.xcontent.ObjectParser.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object, java.lang.Object) @bci=237, line=168 (Interpreted frame)
- org.elasticsearch.common.xcontent.ConstructingObjectParser.parse(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=16, line=169 (Interpreted frame)
- org.elasticsearch.common.xcontent.ConstructingObjectParser.apply(org.elasticsearch.common.xcontent.XContentParser, java.lang.Object) @bci=3, line=161 (Interpreted frame)
- org.elasticsearch.action.admin.cluster.settings.ClusterGetSettingsResponse.fromXContent(org.elasticsearch.common.xcontent.XContentParser) @bci=5, line=143 (Interpreted frame)
- org.elasticsearch.client.ClusterClient$$Lambda$775.apply(java.lang.Object) @bci=4 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.parseEntity(org.apache.http.HttpEntity, org.elasticsearch.common.CheckedFunction) @bci=164, line=2061 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$13(org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.Response) @bci=6, line=1698 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient$$Lambda$751.apply(java.lang.Object) @bci=12 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(java.lang.Object, org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.RequestOptions, org.elasticsearch.common.CheckedFunction, java.util.Set) @bci=93, line=1782 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.performRequest(org.elasticsearch.action.ActionRequest, org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.RequestOptions, org.elasticsearch.common.CheckedFunction, java.util.Set) @bci=35, line=1735 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(org.elasticsearch.action.ActionRequest, org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.RequestOptions, org.elasticsearch.common.CheckedFunction, java.util.Set) @bci=14, line=1697 (Interpreted frame)
- org.elasticsearch.client.ClusterClient.getSettings(org.elasticsearch.action.admin.cluster.settings.ClusterGetSettingsRequest, org.elasticsearch.client.RequestOptions) @bci=19, line=116 (Interpreted frame)
6 个回复
yangjianxuan
赞同来自:
yangjianxuan
赞同来自:
yangjianxuan
赞同来自:
1、还是从堆栈信息入手:
org.elasticsearch.common.settings.Settings
org.elasticsearch.common.settings.Settings$Builder
这两个类在初始化时好像死锁了
Thread 18683: (state = BLOCKED)
- org.elasticsearch.common.settings.Settings.<clinit>() @bci=0, line=84 (Interpreted frame)
- org.elasticsearch.action.admin.indices.settings.get.GetSettingsResponse.parseSettingsField(org.elasticsearch.common.xcontent.XContentParser, java.lang.String, java.util.Map, java.util.Map) @bci=115, line=147 (Interpreted frame)
- org.elasticsearch.action.admin.indices.settings.get.GetSettingsResponse.parseIndexEntry(org.elasticsearch.common.xcontent.XContentParser, java.util.Map, java.util.Map) @bci=39, line=166 (Interpreted frame)
- org.elasticsearch.action.admin.indices.settings.get.GetSettingsResponse.fromXContent(org.elasticsearch.common.xcontent.XContentParser) @bci=86, line=182 (Interpreted frame)
- org.elasticsearch.client.IndicesClient$$Lambda$750.apply(java.lang.Object) @bci=4 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.parseEntity(org.apache.http.HttpEntity, org.elasticsearch.common.CheckedFunction) @bci=164, line=2061 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$13(org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.Response) @bci=6, line=1698 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient$$Lambda$751.apply(java.lang.Object) @bci=12 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(java.lang.Object, org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.RequestOptions, org.elasticsearch.common.CheckedFunction, java.util.Set) @bci=93, line=1782 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.performRequest(org.elasticsearch.action.ActionRequest, org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.RequestOptions, org.elasticsearch.common.CheckedFunction, java.util.Set) @bci=35, line=1735 (Interpreted frame)
- org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(org.elasticsearch.action.ActionRequest, org.elasticsearch.common.CheckedFunction, org.elasticsearch.client.RequestOptions, org.elasticsearch.common.CheckedFunction, java.util.Set) @bci=14, line=1697 (Interpreted frame)
- org.elasticsearch.client.IndicesClient.getSettings(org.elasticsearch.action.admin.indices.settings.get.GetSettingsRequest, org.elasticsearch.client.RequestOptions) @bci=19, line=858 (Interpreted frame)
Thread 18681: (state = BLOCKED)
- org.elasticsearch.common.settings.Settings$Builder.build() @bci=8, line=1242 (Interpreted frame)
- org.elasticsearch.common.settings.Settings$Builder.<clinit>() @bci=7, line=782 (Interpreted frame)
- org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest.<init>(java.lang.String) @bci=16, line=56 (Interpreted frame)
yangjianxuan
赞同来自:
Setings类初始化时会去new Settings.Builder()
build()方法会去new Settings
yangjianxuan
赞同来自:
4、UpdateSettingsRequest初始化Builder时将Builder类锁住,调用builder方法时去new Settings需要获取Settings类的锁,但是获取不到,因为已经被GetSettingsResponse的调用parseSettingsField时锁住
因此,形成死锁
yangjianxuan
赞同来自:
2、解决方法,将注释的两行代码打开,提前加载Builder类,破坏死锁条件
最后,虽然问题解决了,但是理论还是比较匮乏,希望有懂的大神指点一下,这个锁机制。
以及,有没有直观地定位问题的方法。因为看到堆栈信息,基本就是靠猜,感觉有点误打误撞了。