要不要也来分享分享一下啊

使用BulkProcessor批量导入时报错

Elasticsearch | 作者 zxasqw | 发布于2019年11月18日 | 阅读数:4750

在使用BulkProcessor进行批量添加时,有的会报连接超时,请问是为什么
代码是:
BulkProcessor.Listener listener = new BulkProcessor.Listener() {

@Override
public void beforeBulk(long executionId, BulkRequest request) {
// 执行之前调用
System.out.println("操作" + request.numberOfActions()+"条数据");
}

@Override
public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {
// 执行之后调用
System.out.println("成功" + request.numberOfActions()+"条数据,用时" + response.getTook());
}

@Override
public void afterBulk(long executionId, BulkRequest request, Throwable failure) {
// 失败时调用
System.out.println("失败" + request.numberOfActions()+"条数据");
System.out.println("失败" + failure);
}
};

BulkProcessor bulkProcessor = BulkProcessor.builder(
(request, bulkListener) -> client.bulkAsync(request, RequestOptions.DEFAULT, bulkListener),
listener).setBulkActions(-1)
.setBulkSize(new ByteSizeValue(100L, ByteSizeUnit.MB))
.setFlushInterval(TimeValue.timeValueSeconds(10))
.setConcurrentRequests(1)
// 设置退避, 100ms后执行, 最大请求3次
.setBackoffPolicy(BackoffPolicy.constantBackoff(TimeValue.timeValueSeconds(100),3))
.build();
for (int i = 0; i < 2000000; i++) {
bulkProcessor.add(new IndexRequest("device_datas").source(
XContentType.JSON,
"id" , i,
"type", "xxxx",
"fileName", "xxxxxxxxx",
"elapsedTime", Math.random()*100,
"count", Math.floor(Math.random()*10000),
"feeder001", "feeder001" + i,
"feeder002", "feeder002" + i,
"feeder003", "feeder003" + i,
"feeder004", "feeder004" + i,
"feeder005", "feeder005" + i,
"feeder006", "feeder006" + i,
"feeder007", "feeder007" + i,
"feeder008", "feeder008" + i,
"feeder009", "feeder009" + i,
"feeder010", "feeder010" + i,
"feeder011", "feeder011" + i,
"feeder012", "feeder012" + i,
"feeder013", "feeder013" + i,
"feeder014", "feeder014" + i,
"feeder015", "feeder015" + i,
"feeder016", "feeder016" + i,
"feeder017", "feeder017" + i,
"feeder018", "feeder018" + i,
"feeder019", "feeder019" + i,
"feeder020", "feeder020" + i,
"feeder021", "feeder021" + i,
"feeder022", "feeder022" + i,
"feeder023", "feeder023" + i,
"feeder024", "feeder024" + i,
"feeder025", "feeder025" + i,
"feeder026", "feeder026" + i,
"feeder027", "feeder027" + i,
"feeder028", "feeder028" + i,
"feeder029", "feeder029" + i,
"feeder030", "feeder030" + i,
"feeder031", "feeder031" + i,
"feeder032", "feeder032" + i,
"feeder033", "feeder033" + i,
"feeder034", "feeder034" + i,
"feeder035", "feeder035" + i
));
}
控制台打印的信息:

操作90268条数据
操作87523条数据
成功90268条数据,用时29.6s
操作87231条数据
失败87523条数据
失败java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]
操作61条数据
失败87231条数据
失败java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-1 [ACTIVE]
成功61条数据,用时341ms
已邀请:

要回复问题请先登录注册