是时候用 ES 拯救发际线啦

关于Java highlevel rest client 的 slice 请问大佬们有实例参考吗?

Elasticsearch | 作者 Han_ | 发布于2019年12月10日 | 阅读数:1843

最近公司的项目需要在很短的时间内返回2w条左右的数据量,因此我用了scoll查询,2w条数据大约不到1秒的时间返回。又了解到slice采用多线程的方式,可以进一步提高查询性能,奈何本人水平有限,望大佬们不吝赐教,给我个demo之类的即可。以下是我的代码
@Autowired
private RestHighLevelClient restHighLevelClient;

@Override
public List<ResultDto> searchHighLevelClientToken( List<String> strList) {
Long start = System.currentTimeMillis();
//初始化scroll
final Scroll scroll = new Scroll(TimeValue.timeValueMinutes(1L)); //设定滚动时间间隔
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices(INDEX_NAME);
searchRequest.scroll(scroll);

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery().filter(QueryBuilders.boolQuery()
// .should(QueryBuilders.termQuery("last_visit.grid.key", gridKey))
// .should(QueryBuilders.termQuery("company.key", gridKey))
// .should(QueryBuilders.termQuery("home.key", gridKey)))

// .should(QueryBuilders.termsQuery("last_visit.grid.key", strList))
.should(QueryBuilders.termsQuery("company.key", strList))
.should(QueryBuilders.termsQuery("home.key", strList)))
.filter(QueryBuilders.existsQuery("token"))
;
searchSourceBuilder.query(queryBuilder);
searchSourceBuilder.size(10000); //设定每次返回多少条数据
searchSourceBuilder.sort("_doc", SortOrder.ASC);//如果对scroll取出的数据顺序没有要求的话,则可以对“_doc”进行排序,es对这种排序做了优化。
String[] includes = new String[] {"uid","token", "company.poi", "home.poi"};
String[] excludes = new String[] {};//{"uid", "main_location.*", "last_visit.*"};
searchSourceBuilder.fetchSource(includes, excludes);//ES里该index内存的文档,可能存了很多我们不关心的字段,全返回没必要,所以指定有用的字段
searchRequest.source(searchSourceBuilder);

SearchResponse searchResponse = null;
try {
searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
e.printStackTrace();
}
String scrollId = searchResponse.getScrollId();
SearchHit[] searchHits = searchResponse.getHits().getHits();
int a = searchHits.length;
System.out.println(a);
TimeValue took = searchResponse.getTook(); // 查询占用的时间
log.info("es查询耗时:{}", took);
List<ResultDto> resultList = new ArrayList<ResultDto>();
setSearchHitResult(resultList,searchHits);

//遍历搜索命中的数据,直到没有数据
while (searchHits != null && searchHits.length > 0) {
SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
scrollRequest.scroll(scroll);
try {
searchResponse = restHighLevelClient.scroll(scrollRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
e.printStackTrace();
}
scrollId = searchResponse.getScrollId();
searchHits = searchResponse.getHits().getHits();
TimeValue took1 = searchResponse.getTook(); // 查询占用的时间
log.info("循环中es查询耗时:{}", took1);
setSearchHitResult(resultList,searchHits);
}

System.out.println("查询总命中数:" + resultList.size());
//清除滚屏
ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
clearScrollRequest.addScrollId(scrollId);//也可以选择setScrollIds()将多个scrollId一起使用
ClearScrollResponse clearScrollResponse = null;
try {
clearScrollResponse = restHighLevelClient.clearScroll(clearScrollRequest,RequestOptions.DEFAULT);
} catch (IOException e) {
e.printStackTrace();
}
boolean succeeded = clearScrollResponse.isSucceeded();
System.out.println("succeeded:" + succeeded);
Long end = System.currentTimeMillis();
log.info("RestHighLevelClient search time consuming:{}ms", end - start);
return resultList;
}
List<ResultDto> setSearchHitResult(List<ResultDto> resultList, SearchHit[] searchHits)
{
if (searchHits != null && searchHits.length > 0) {
for (SearchHit searchHit : searchHits) {
GridTokenInfo gti = JSON.parseObject(searchHit.getSourceAsString(), GridTokenInfo.class);
ResultDto rd = new ResultDto();
if(StringUtils.isNotBlank(gti.getUid())) {
rd.setId(gti.getUid());
}
if(StringUtils.isNotBlank(gti.getToken())) {
rd.setTk(gti.getToken());
}
if(gti.getCompany() != null && StringUtils.isNotBlank(gti.getCompany().getPoi())) {
rd.setPoi(gti.getCompany().getPoi());
}
else if(gti.getHome() != null && StringUtils.isNotBlank(gti.getHome().getPoi())) {
rd.setPoi(gti.getHome().getPoi());
}
resultList.add(rd);
}
}
return resultList;
}
已邀请:

Charele - Cisco4321

赞同来自:

你在哪看到的啊?HLRC里面好像没有slice这概念吧

Han_

赞同来自:

Charele - Cisco4321

赞同来自:

你上面说的那些是定制ES搜索时的行为,
(在TransportSearchAction里面)
 
你要明白,这些不属于Java highlevel rest client里面的,
 
也就是说如果你用的是别的客户端,比如TransportClient,也可以用这些选项。

要回复问题请先登录注册