Hello,World

ES查询排序性能问题,使用From Size和Search After差距巨大

Elasticsearch | 作者 MarvinLiu | 发布于2021年09月06日 | 阅读数:301

几亿条数据,做match all查询,按照时间和id排序,使用From Size方式查询秒回,使用Search after方式查询,耗时十几二十秒,通过profile查看,时间耗在了next_doc: "searches":[
{
"query":[
{
"type":"BoostQuery",
"description":"(ConstantScore(DocValuesFieldExistsQuery [field=__time__]))^0.0",
"time_in_nanos":45722536283,
"breakdown":{
"set_min_competitive_score_count":0,
"match_count":0,
"shallow_advance_count":0,
"set_min_competitive_score":0,
"next_doc":45722241414,
"match":0,
"next_doc_count":410919487,
"score_count":0,
"compute_max_score_count":0,
已邀请:

yuechen323 - 晨儿哥

赞同来自: MarvinLiu

你是不是有自定义排序, query 发来看看

MarvinLiu

赞同来自:

使用的time和key进行排序,其中key是复制的_id,类型为keyword

{
    "query":{
        "bool":{
            "filter":[
                {
                    "query_string":{
                        "query":"*"
                    }
                },
                {
                    "range":{
                        "__time__":{
                            "gte":1324958207,
                            "lte":1724958207
                        }
                    }
                }
            ]
        }
    },
    "size":10,
    "sort":[
        {
            "__time__":{
                "order":"desc"
            }
        },
        {
            "__key__":{
                "order":"desc"
            }
        }
    ],
    "search_after":[
        1630594662000,
        "6130e666-2-67e9e3-f5-1"
    ],
    "profile":true
}

MarvinLiu

赞同来自:

profile的结果:
{
"searches":[
{
"query":[
{
"type":"BoostQuery",
"description":"(ConstantScore(DocValuesFieldExistsQuery [field=__time__]))^0.0",
"time_in_nanos":45722536283,
"breakdown":{
"set_min_competitive_score_count":0,
"match_count":0,
"shallow_advance_count":0,
"set_min_competitive_score":0,
"next_doc":45722241414,
"match":0,
"next_doc_count":410919487,
"score_count":0,
"compute_max_score_count":0,
"compute_max_score":0,
"advance":19517,
"advance_count":39,
"score":0,
"build_scorer_count":78,
"create_weight":14271,
"shallow_advance":0,
"create_weight_count":1,
"build_scorer":261081
},
"children":[
{
"type":"DocValuesFieldExistsQuery",
"description":"DocValuesFieldExistsQuery [field=__time__]",
"time_in_nanos":16571715415,
"breakdown":{
"set_min_competitive_score_count":0,
"match_count":0,
"shallow_advance_count":0,
"set_min_competitive_score":0,
"next_doc":16571493898,
"match":0,
"next_doc_count":410919487,
"score_count":0,
"compute_max_score_count":0,
"compute_max_score":0,
"advance":15074,
"advance_count":39,
"score":0,
"build_scorer_count":78,
"create_weight":517,
"shallow_advance":0,
"create_weight_count":1,
"build_scorer":205926
}
}
]
}
],
"rewrite_time":116538,
"collector":[
{
"name":"PagingFieldCollector",
"reason":"search_top_hits",
"time_in_nanos":30851166561
}
]
}
],
"aggregations":[

]
}
@yuechen323 感谢

wanghaiyu1997 - 95后

赞同来自:

from和size是浅分页 如果数据量很大 页数很多会变慢 建议了解下深分页

要回复问题请先登录注册