filebeat 5.6 如何定义多个index

filebeat 启用 module: nginx
filebeat.yml配置如下:
#-------------------------------- Nginx Module -------------------------------
- module: nginx
# Access logs
access:
enabled: true
var.paths:
- /usr/local/nginx/logs/products_a_*.log
- /usr/local/nginx/logs/products_b_*.log
 
收集到的日志直接发送到 es, 默认的index为: filebeat-%{+yyyy.MM.dd}"
 
现在想根据上面配置文件中的 var.paths: 里面两个不同项目的日志文件生成各自的index: 
filebeat-products_a-%{+yyyy.MM.dd}
filebeat-products_b-%{+yyyy.MM.dd}
 
即:
/usr/local/nginx/logs/products_a_*.log  日志内容--> filebeat-products_a-%{+yyyy.MM.dd}
/usr/local/nginx/logs/products_b_*.log  日志内容--> filebeat-products_b-%{+yyyy.MM.dd}
 
请教,filebeat上该怎么配置呢?
已邀请:

djx1996 - 90

赞同来自: kuanglin ghdong

你可以参考一下我的配置。
在filebeat配置
filebeat.prospectors:
- input_type: log

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
  - /var/lib/mysql/localhost-slow.log
  document_type: redhat.mysql.log
- input_type: log
  paths:
  - /var/log/yum.log
  document_type: redhat.yum.log

在logstash的配置
input {
        beats{
                port =>5044
        }
}
output {
        if [type] == "redhat.mysql.log"
        {
        elasticsearch {
                         hosts => "192.168.40.66:9200"
                         index => "testserver_mysql.log-%{+YYYY.MM.DD}"
                      }
        }

        if [type] ==  "redhat.yum.log"
        {
        elasticsearch {
                         hosts => "192.168.40.66:9200"
                         index => "testserver_yum.log-%{+YYYY.MM.DD}"
                      }
        }

    stdout { codec=> rubydebug }

}

 

pf

赞同来自:

filebaet直传到es,  不用logstash,  有没有解决方案呢?

ledefe

赞同来自:

用条件表达式

kkgace

赞同来自:

在output到es时使用source来判断就好,举个栗子:
indices:
    - index: "ela-%{+yyyy.MM}"
      when.contains:
         source: "elasticsearch"
    - index: "kibana-%{+yyyy.MM.dd}"
      when.contains:
         source: "kibana"
 

要回复问题请先登录注册