聚合查询返回的结果还未达到search.max_buckets的值，就报 超过search.max_buckets 的异常
Elasticsearch | 作者 zhangRock | 发布于2020年08月06日 | 阅读数：7464
es 版本 7.6
kibana查询语句：
GET dcp2020-07-28/_search?format=txt
{
"size":1,
"aggs": {
"testa":{
"terms":{
"field":"imsi",
"size":7000
}
}
}
}
报错信息：
{ "error": { "root_cause": [], "type": "search_phase_execution_exception", "reason": "", "phase": "fetch", "grouped": true, "failed_shards": [], "caused_by": { "type": "too_many_buckets_exception", "reason": "Trying to create too many buckets. Must be less than or equal to: [10000] but was [10001]. This limit can be set by changing the [search.max_buckets] cluster level setting.", "max_buckets": 10000 } }, "status": 503 }
根据官网对search.max_buckets 的解释
The maximum number of buckets allowed in a single response is limited by a dynamic cluster setting named search.max_buckets. It defaults to 10,000, requests that try to return more than the limit will fail with an exception.
我在查询的时候已经限制了size为7000了，低于10000， 还是报错
而设值size 为6600 ，是正常返回6600条。
6000也是。
现在不知道问题所在。官网描述是 一次请求的返回不超过 10000个 buckets。 而我查询 size限制返回buckets 是 7000条
kibana查询语句：
GET dcp2020-07-28/_search?format=txt
{
"size":1,
"aggs": {
"testa":{
"terms":{
"field":"imsi",
"size":7000
}
}
}
}
报错信息：
{ "error": { "root_cause": [], "type": "search_phase_execution_exception", "reason": "", "phase": "fetch", "grouped": true, "failed_shards": [], "caused_by": { "type": "too_many_buckets_exception", "reason": "Trying to create too many buckets. Must be less than or equal to: [10000] but was [10001]. This limit can be set by changing the [search.max_buckets] cluster level setting.", "max_buckets": 10000 } }, "status": 503 }
根据官网对search.max_buckets 的解释
The maximum number of buckets allowed in a single response is limited by a dynamic cluster setting named search.max_buckets. It defaults to 10,000, requests that try to return more than the limit will fail with an exception.
我在查询的时候已经限制了size为7000了，低于10000， 还是报错
而设值size 为6600 ，是正常返回6600条。
6000也是。
现在不知道问题所在。官网描述是 一次请求的返回不超过 10000个 buckets。 而我查询 size限制返回buckets 是 7000条
4 个回复
Charele - Cisco4321
赞同来自: viewsite 、zhangRock
前提: search.max_buckets: 3
我按上面朋友那样，插入了"A","B","C","D" 4条数据
要分两种情况来说明：
1 索引只有一个分片
这时候不管你设不设size，哪怕size=1都会报错！
原因就是我上面说的那个，因为它在这个分片上找到的条目超过了限定的3个（实际有4个）
2 如果这个索引有5个分片，数据分散在各个分片上面
index shard prirep state docs store ip node
a 1 p STARTED 1 3kb 127.0.0.1 es0
a 3 p STARTED 1 3kb 127.0.0.1 es0
a 4 p STARTED 1 3kb 127.0.0.1 es0
a 2 p STARTED 0 208b 127.0.0.1 es0
a 0 p STARTED 1 3kb 127.0.0.1 es0
这时如果你设size=2，就不会报错。
因为还没有等它找完全部分片，他就找到了你想要的2个，就返回了。
如果你不设size，或者设一个大值size=10，它就必须找完全部分片，
这时就会遇到限定3，就会报错
所以这和你的数据在分片上的分布有关，
解决办法还是加大search.max_buckets
viewsite
赞同来自: zhangRock
PUT /testBucket/_doc/1
{"term":"A"}
PUT /testBucket/_doc/2
{"term":"B"}
PUT /testBucket/_doc/3
{"term":"C"}
PUT /testBucket/_doc/4
{"term":"D"}
PUT /_cluster/settings?pretty
{"persistent": {"search.max_buckets": 3}}
POST /testBucket/_search?pretty
{"aggs":{"term":{"terms":{"field":"term","size":2}}}}
成功
POST /testBucket/_search?pretty
{"aggs":{"term":{"terms":{"field":"term","size":4}}}}
返回报错：
{
"error" : {
"root_cause" : [ ],
"type" : "search_phase_execution_exception",
"reason" : "",
"phase" : "fetch",
"grouped" : true,
"failed_shards" : [ ],
"caused_by" : {
"type" : "too_many_buckets_exception",
"reason" : "Trying to create too many buckets. Must be less than or equal to: [3] but was [4]. This limit can be set by changing the [search.max_buckets] cluster level setting.",
"max_buckets" : 3
}
},
"status" : 503
}
Charele - Cisco4321
赞同来自: byx313
（缺省就是数量最多的前7000个）
如果你库里imsi的类别超过缺省的1万的，就会报错。
解决办法就是加大search.max_buckets
thewind
赞同来自:
Control number of buckets created in an aggregation
Terms aggregation
size 的计算公式为
(1) 当search.maxBuckets = 10000时候，maxNumTermBuckets = 6660,
即size> 6660 都会报错