要不要也来分享分享一下啊

ES5.0 批量导入数据,设置导入结束后刷新不生效

匿名 | 发布于2016年12月05日 | 阅读数:8797

我想在批量导入结束后再让ES执行刷新操作,提高导入性能,
但是实际上我看到数据一直是刷新的,因为我在head插件的概览界面看到索引docs数量是一直增加的
这不就表示ES实际上已经再刷新了,并没有执行我的设置


elasticsearch.yml配置:

http.cors.enabled: true
http.cors.allow-origin: "*"
path.data: /esdata/elasticsearch/data
path.logs: /esdata/elasticsearch/logs
network.host: 192.168.24.84
http.port: 8201
transport.tcp.port: 8301
bootstrap.memory_lock: true

索引的Mapping设置:
{"state": "open",
"settings": {
"index": {
"refresh_interval": "-1",
"number_of_shards": "4",
"translog": {
"flush_threshold_size": "10240mb",
"sync_interval": "60m",
"durability": "async"
}... ...

JAVA客户端代码:
           
     client.admin()
                    .indices()
                    .prepareUpdateSettings(indexName)
                    .setSettings(Settings.builder()
                        .put("index.refresh_interval", "-1")
                        .put("index.translog.flush_threshold_size", "10240mb")
                        .put("index.translog.durability", "async")
                        )
                    .get();
                BulkProcessor bulkProcessor = BulkProcessor.builder(client, new BulkProcessor.Listener()
                {
                    public void beforeBulk(long executionId, BulkRequest request){
                    }
                    
                    @Override
                    public void afterBulk(long executionId, BulkRequest request, BulkResponse response){
                    }
                    
                    @Override
                    public void afterBulk(long executionId, BulkRequest request, Throwable failure){
                    }
                }).setBulkActions(5000).setBulkSize(new ByteSizeValue(1L, ByteSizeUnit.PB)).setFlushInterval(TimeValue.timeValueHours(1L)).setConcurrentRequests(10).build();
                IndexRequest iq = new IndexRequest(indexName, indexType).source(doc).setRefreshPolicy(RefreshPolicy.NONE);
                for (int i = 0; i < 1000000; i++)
                {
                    bulkProcessor.add(iq);
                }
                bulkProcessor.awaitClose(3, TimeUnit.MINUTES);
                client.admin().indices().prepareUpdateSettings(indexName).setSettings(Settings.builder().put("index.refresh_interval", "1s")).get();
 
已邀请:

240475588

赞同来自:

同样遇到一样的问题,bulk update 5000,用heap 搜索更新结果,发现是实时的,index.refresh_interval:-1 并没有效果

240475588

赞同来自:

问题解决了吗,我也是更新的时候
index.refresh_interval
不生效,感觉没更新一个文档都会刷新

Jaret

赞同来自:

同样的问题,但是es7.3可以生效

要回复问题请先登录注册