一个直接用 terms aggs ，一个用 terms aggs 的 painless script 写法。求大神解惑一下。我这个索引比较大，没有副本，5.16T
测试1
结果一 （每次都是10多秒，有同事执行100多万次这样的语句，es 集群所有节点很快 queue 1000 ，出现大量rejected ）
结果1 search profiler
测试2 使用 painless script 写法的terms aggs
测试1
GET /weibo/weibo/_search
{
"query": {
"bool": {
"filter": [{
"query_string": {
"fields": ["content_full"],
"query" : "\"@小米主题\""
}
}, {
"range": {
"published_at": {
"gte": "2016-05-01",
"lte": "2017-05-01"
}
}
}]
}
},
"size": 0,
"_source": {
"excludes":
},
"aggs": {
"sdf": {
"terms": {
"field": "weiboer_id"
}
}
}
}结果一 （每次都是10多秒，有同事执行100多万次这样的语句，es 集群所有节点很快 queue 1000 ，出现大量rejected ）
{
"took": 12715,
"timed_out": false,
"_shards": {
"total": 45,
"successful": 45,
"failed": 0
},
"hits": {
"total": 11043,
"max_score": 0,
"hits":
},
"aggregations": {
"sdf": {
"doc_count_error_upper_bound": 51,
"sum_other_doc_count": 10298,
"buckets": [
{
"key": "1657835230",
"doc_count": 140
},
{
"key": "1710824123",
"doc_count": 106
},
{
"key": "2117757773",
"doc_count": 90
},
{
"key": "1669734057",
"doc_count": 81
},
{
"key": "1787512884",
"doc_count": 75
},
{
"key": "1659333803",
"doc_count": 62
},
{
"key": "1727090064",
"doc_count": 59
},
{
"key": "1768333234",
"doc_count": 45
},
{
"key": "1747268970",
"doc_count": 44
},
{
"key": "1720297231",
"doc_count": 43
}
]
}
}
}结果1 search profiler
测试2 使用 painless script 写法的terms aggs
GET /weibo/weibo/_search
{
"query": {
"bool": {
"filter": [{
"query_string": {
"fields": ["content_full"],
"query" : "\"@小米主题\""
}
}, {
"range": {
"published_at": {
"gte": "2014-05-01",
"lte": "2017-05-01"
}
}
}]
}
},
"size": 0,
"_source": {
"excludes":
},
"aggs" : {
"clientip_top10" : {
"terms" : {
"script" : {
"lang" : "painless",
"inline" : "doc['weiboer_id'].value"
}
}
}
}
}{
"took": 64,
"timed_out": false,
"_shards": {
"total": 45,
"successful": 45,
"failed": 0
},
"hits": {
"total": 19317,
"max_score": 0,
"hits":
},
"aggregations": {
"clientip_top10": {
"doc_count_error_upper_bound": 81,
"sum_other_doc_count": 18041,
"buckets": [
{
"key": "1649471152",
"doc_count": 447
},
{
"key": "1657835230",
"doc_count": 179
},
{
"key": "1710824123",
"doc_count": 120
},
{
"key": "1659333803",
"doc_count": 91
},
{
"key": "2117757773",
"doc_count": 88
},
{
"key": "1704468187",
"doc_count": 81
},
{
"key": "1669734057",
"doc_count": 80
},
{
"key": "1787512884",
"doc_count": 69
},
{
"key": "1727090064",
"doc_count": 64
},
{
"key": "1747268970",
"doc_count": 57
}
]
}
}
}
5 个回复
kennywu76 - Wood
赞同来自: lunatictwo
具体写法:
pqy
赞同来自:
lz8086 - es小司机
赞同来自:
pqy
赞同来自:
Simple Query String Query 第一次查询差不多 400ms
fhyes123 - ES小白
赞同来自: