filebeat发送日志到kafka可以只发送原始日志吗，也就是message中的内容，
目前发送内容：{"@timestamp":"2018-03-29T07:06:19.606Z","beat":{"hostname":"BC-VM-e4e8928f82d14f49846d8679e70b5333","name":"BC-VM-e4e8928f82d14f49846d8679e70b5333","version":"5.6.0"},"input_type":"log","message":"2018-03-29 15:06:10,283 1141622185 [Timer-0] INFO c.x.e.logback.controller.LogHelper(44) run - [MESSAGE][DOMAINTYPE]B00.00.00.02_person [DIDATA]60 [CORRECTDATA]188 [ERRORDATA]47 *STATISTICS-DI*","offset":31535221,"source":"/opt/logbackdemo/logbackDemo.log","type":"log"}
想要发送内容：2018-03-29 15:06:10,283 1141622185 [Timer-0] INFO c.x.e.logback.controller.LogHelper(44) run - [MESSAGE][DOMAINTYPE]B00.00.00.02_person [DIDATA]60 [CORRECTDATA]188 [ERRORDATA]47 *STATISTICS-DI*
目前发送内容：{"@timestamp":"2018-03-29T07:06:19.606Z","beat":{"hostname":"BC-VM-e4e8928f82d14f49846d8679e70b5333","name":"BC-VM-e4e8928f82d14f49846d8679e70b5333","version":"5.6.0"},"input_type":"log","message":"2018-03-29 15:06:10,283 1141622185 [Timer-0] INFO c.x.e.logback.controller.LogHelper(44) run - [MESSAGE][DOMAINTYPE]B00.00.00.02_person [DIDATA]60 [CORRECTDATA]188 [ERRORDATA]47 *STATISTICS-DI*","offset":31535221,"source":"/opt/logbackdemo/logbackDemo.log","type":"log"}
想要发送内容：2018-03-29 15:06:10,283 1141622185 [Timer-0] INFO c.x.e.logback.controller.LogHelper(44) run - [MESSAGE][DOMAINTYPE]B00.00.00.02_person [DIDATA]60 [CORRECTDATA]188 [ERRORDATA]47 *STATISTICS-DI*
3 个回复
zcheneng
赞同来自: Jiehui Tang
1、filebeat.yml文件增加
############################# Processors ######################################
processors:
- drop_fields:
fields: ["prospector","source","input", "beat", "offset","tags","host"]
2、@version,@timestamp,tags在filebeat不知道怎么去掉，反正我是没成功，不过可以再logstash中的配置文件中去掉
filter {
json {
source => "message"
#target => "doc"
remove_field => ["@version","@timestamp","tags"]
}
}
yirujiwang
赞同来自:
typuc - 80后IT男，乒乓球爱好者
赞同来自: