使用 dmesg 来查看一些硬件或驱动程序的信息或问题。

elasticsearch data too large异常

Elasticsearch | 作者 code4j | 发布于2019年05月16日 | 阅读数:4195



企业微信截图_1b650272-ceb5-451d-8da7-89fae5eb809d(2)(1)(2).png

 
es集群版本5.2.1  出现这个异常,网上有资料 es自带的熔断器,indices.breaker.fielddata.limit fielddata 超过这个阈值,就会抛异常。
 
我看了下当前的fielddata 似乎也没有那么大
 

企业微信截图_a99071c7-a2f3-468f-a445-5304a9f5fb8f.png

 
 
求解
已邀请:

xiaoke - http://blog.51cto.com/kexiaoke

赞同来自: code4j

哈哈,这个问题,我上上周也查了一天。https://elasticsearch.cn/question/7587

laoyang360 - 《一本书讲透Elasticsearch》作者,Elastic认证工程师 [死磕Elasitcsearch]知识星球地址:http://t.cn/RmwM3N9;微信公众号:铭毅天下; 博客:https://elastic.blog.csdn.net

赞同来自: code4j

GET _nodes/stats/breaker 查看一下;
Elasticsearch包含多个断路器,用于防止操作导致OutOfMemoryError。
整体父断路器的起始限制——默认为JVM堆的70%。 截图bug就是触发了父类断路器机制。

bellengao - 博客: https://www.jianshu.com/u/e0088e3e2127

赞同来自:

是触发了parent级别的的熔断,当前已经使用的总的内存占jvm heap超过70%, GET _nodes/stats 看一下breakers的内存统计

sailershen

赞同来自:

各位老师,我也碰到同样的问题,哪里能修改相关配置?
 
前提:我把truncate:maxHeight的值改为0以后出现以下这个错误提示。

Discover页面上显示:
Discover: [parent] Data too large, data for [<agg [2]>] would be [493370184/470.5mb], which is larger than the limit of [493030604/470.1mb], real usage: [493365064/470.5mb], new bytes reserved: [5120/5kb]

 
在安装Metricbeat的服务器上执行metricbeat setup --dashboards出错:
Loading dashboards (Kibana must be running and reachable)
Exiting: Failed to import dashboard: Failed to load directory /usr/share/metricbeat/kibana/7/dashboard:
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-aws-sqs-overview.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"[parent] Data too large, data for [<http_request>] would be [493043884/470.2mb], which is larger than the limit of [493030604/470.1mb], real usage: [493012912/470.1mb], new bytes reserved: [30972/30.2kb]: [circuit_breaking_exception] [parent] Data too large, data for [<http_request>] would be [493043884/470.2mb], which is larger than the limit of [493030604/470.1mb], real usage: [493012912/470.1mb], new bytes reserved: [30972/30.2kb], with { bytes_wanted=493043884 & bytes_limit=493030604 & durability=\"PERMANENT\" }"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-containers-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-coredns-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-docker-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-golang-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-haproxy-backend.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-haproxy-frontend.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-haproxy-http-backend.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-haproxy-http-frontend.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-haproxy-http-server.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-haproxy-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-haproxy-visualizations.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-host-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-kafka-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-kubernetes-apiserver.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-kubernetes-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-mongodb-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-mssql-performance.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-mssql-transaction_log.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-mysql-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-nats-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-rabbitmq-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-redis-keys.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-redis-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-system-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-uwsgi-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-zookeeper-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/metricbeat-nginx-overview.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
error loading /usr/share/metricbeat/kibana/7/dashboard/metricbeat-windows-service.json: returned 401 to import file: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}

以下是GET _nodes/stats的结果,breakers这里的limit_size是290m:
{
"_nodes" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"cluster_name" : "docker-cluster",
"nodes" : {
......
"breakers" : {
"request" : {
"limit_size_in_bytes" : 311387750,
"limit_size" : "296.9mb",
"estimated_size_in_bytes" : 0,
"estimated_size" : "0b",
"overhead" : 1.0,
"tripped" : 0
}


 
 
 
 

sailershen

赞同来自:

我把我把truncate:maxHeight的值改为默认值115以后,再执行metricbeat setup --dashboards的结果 :
Loading dashboards (Kibana must be running and reachable)
Exiting: Failed to import dashboard: Failed to load directory /usr/share/metricbeat/kibana/7/dashboard:
error loading /usr/share/metricbeat/kibana/7/dashboard/Metricbeat-mysql-overview.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"[parent] Data too large, data for [<http_request>] would be [498201506/475.1mb], which is larger than the limit of [493030604/470.1mb], real usage: [498174552/475mb], new bytes reserved: [26954/26.3kb]: [circuit_breaking_exception] [parent] Data too large, data for [<http_request>] would be [498201506/475.1mb], which is larger than the limit of [493030604/470.1mb], real usage: [498174552/475mb], new bytes reserved: [26954/26.3kb], with { bytes_wanted=498201506 & bytes_limit=493030604 & durability=\"PERMANENT\" }"}
明天继续想办法解决。
 
 
 
 

要回复问题请先登录注册