提问要多花一点心思哦

如何保证采用bulk大批量的写入数据(多线程),同时对服务器的响应速度影响不大,有什么好的策略?

Elasticsearch | 作者 huan | 发布于2018年09月30日 | 阅读数:2151

bulk批量写入
已邀请:

laoyang360 - Elastic认证工程师 [死磕Elasitcsearch]知识星球地址:http://t.cn/RmwM3N9;微信公众号:铭毅天下; 博客:https://elastic.blog.csdn.net

赞同来自:

官网在写入部分,有明确的建议:
逐步增加bulk size大小的值。



Use bulk requestsedit Bulk requests will yield much better performance than single-document index requests. In order to know the optimal size of a bulk request, you should run a benchmark on a single node with a single shard. First try to index 100 documents at once, then 200, then 400, etc. doubling the number of documents in a bulk request in every benchmark run. When the indexing speed starts to plateau then you know you reached the optimal size of a bulk request for your data. In case of tie, it is better to err in the direction of too few rather than too many documents. Beware that too large bulk requests might put the cluster under memory pressure when many of them are sent concurrently, so it is advisable to avoid going beyond a couple tens of megabytes per request even if larger requests seem to perform better.


zqc0512 - andy zhou

赞同来自:

要测试的,这个BULK不是越大越好,是一个波值,测试的时候根据监控取个波峰值左右就行了,每个环境这个值都不一样的。
有个esrally可以测试下。

要回复问题请先登录注册