8台机器,一共15个data节点,3台master节点,5台(每台起三个data实例,共15个)
大神们,这是线上ES 2.4.2的一个节点的yml配置,1.6升级过来的,没做多大改动,集群是3台master+5台node(每台6T磁盘,12块盘,radi0,256G内存,配置3个节点,31G*3 heap,),每天BulkProcessor全量写入ES集群数据量2T多左右,比较慢,请问这个配置哪里可以优化的吗?
cluster.name: index_es
node.name: "es_node1-1"
transport.tcp.port: 9400
node.master: false
node.data: true
http.enabled: false
network.host: xx.xx.xx.xx
path.data: /data1/elasticsearch/data1,/data2/elasticsearch/data1,/data3/elasticsearch/data1,/data4/elasticsearch/data1
path.logs: /data1/elasticsearch/logs1
bootstrap.memory_lock: true
transport.tcp.compress: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.unicast.hosts: ["xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300"]
indices.store.throttle.max_bytes_per_sec : 800mb
indices.cache.filter.size: 50%
discovery.zen.ping.timeout: 120s
discovery.zen.fd.ping_timeout: 120s
discovery.zen.fd.ping_retries: 8
discovery.zen.fd.ping_interval: 30s
client.transport.ping_timeout: 120s
client.transport.nodes_sampler_interval: 30s
index.translog.flush_threshold_size: 2000mb
index.translog.flush_threshold_period: 120m
index.translog.interval: 60m
index.translog.flush_threshold_ops: 100000
index.cache.field.max_size: 50000
index.cache.field.expire: 10m
index.cache.field.type: soft
indices.breaker.fielddata.limit: 50%
indices.memory.index_buffer_size: 50%
indices.fielddata.cache.size: 50%
processors: 10
index.mapping.nested_fields.limit: 1001
大神们,这是线上ES 2.4.2的一个节点的yml配置,1.6升级过来的,没做多大改动,集群是3台master+5台node(每台6T磁盘,12块盘,radi0,256G内存,配置3个节点,31G*3 heap,),每天BulkProcessor全量写入ES集群数据量2T多左右,比较慢,请问这个配置哪里可以优化的吗?
cluster.name: index_es
node.name: "es_node1-1"
transport.tcp.port: 9400
node.master: false
node.data: true
http.enabled: false
network.host: xx.xx.xx.xx
path.data: /data1/elasticsearch/data1,/data2/elasticsearch/data1,/data3/elasticsearch/data1,/data4/elasticsearch/data1
path.logs: /data1/elasticsearch/logs1
bootstrap.memory_lock: true
transport.tcp.compress: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.unicast.hosts: ["xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300","xx.xx.xx.xx:9300"]
indices.store.throttle.max_bytes_per_sec : 800mb
indices.cache.filter.size: 50%
discovery.zen.ping.timeout: 120s
discovery.zen.fd.ping_timeout: 120s
discovery.zen.fd.ping_retries: 8
discovery.zen.fd.ping_interval: 30s
client.transport.ping_timeout: 120s
client.transport.nodes_sampler_interval: 30s
index.translog.flush_threshold_size: 2000mb
index.translog.flush_threshold_period: 120m
index.translog.interval: 60m
index.translog.flush_threshold_ops: 100000
index.cache.field.max_size: 50000
index.cache.field.expire: 10m
index.cache.field.type: soft
indices.breaker.fielddata.limit: 50%
indices.memory.index_buffer_size: 50%
indices.fielddata.cache.size: 50%
processors: 10
index.mapping.nested_fields.limit: 1001
1 个回复
白衬衣 - 金桥
赞同来自:
thread_pool.index.queue_size:
thread_pool.search.queue_size:
thread_pool.get.queue_size:
index.merge.scheduler.max_thread_count:
index.translog.durability: