使用 dmesg 来查看一些硬件或驱动程序的信息或问题。

logstash消费kafka问题

Logstash | 作者 banna147 | 发布于2019年11月06日 | 阅读数:2120

logstash消费kafka时,如果当前链接的那台kafka挂掉,logstash不会重试别的kafka节点直接就停止了。
kafka版本2.10
logstash版本6.5.4
logstash配置:
    kafka {
    bootstrap_servers => ["172.20.213.105:9092,172.20.213.106:9092,172.20.213.107:9092"]
    client_id => "stock-account-api"
    topics => ["stock-account-api"]
    codec => "plain"
    auto_offset_reset => "latest"
    consumer_threads => 1
    decorate_events => true
    type => "stock-account-api"
  }
不知道啥原因,求大神指导!
已邀请:

chencandong

赞同来自:

stock-account-api的partition配置为多少呢 只分配到1台?看你consumer_threads配置为1?

banna147 - 80后运维

赞同来自:

topic信息

chencandong

赞同来自:


bootstrap_serversedit
Value type is string
Default value is "localhost:9092"
A list of URLs of Kafka instances to use for establishing the initial connection to the cluster. This list should be in the form of host1:port1,host2:port2 These urls are just used for the initial connection to discover the full cluster membership (which may change dynamically) so this list need not contain the full set of servers (you may want more than one, though, in case a server is down).


 改成这个试试呢 bootstrap_servers => "172.20.213.105:9092,172.20.213.106:9092,172.20.213.107:9092"
6.5.4的logstash的bootstrap_servers的要求是string不是arry来的  不过看你跑好像也没有什么问题哦。。

 
具体文档https://www.elastic.co/cn/supp ... ugins
 

要回复问题请先登录注册