filebeat 收集日志怎么配置可以做到实时采集
filebeat.yml配置如下:
filebeat:
spool_size: 1024
idle_timeout: "5s"
prospectors:
-
document_type: "module"
paths:
- /data/logs/module/log.log
multiline.pattern: "^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3} -"
multiline.negate: true
multiline.match: after
output:
kafka:
hosts: ["127.0.0.1:9092"]
topic: '%{[type]}'
partition.round_robin:
reachable_only: false
max_message_bytes: 1000000
运行命令./filebeat -c filebeat.yml -d publish,可以在console看到相关信息输出:
2017-06-13T17:00:25+08:00 DBG Publish: {
"@timestamp": "2017-06-13T09:00:24.871Z",
"beat": {
"hostname": "localhost",
"name": "localhost",
"version": "5.4.0"
},
"input_type": "log",
"message": "2017-06-13 17:00:19,701 - INFO - Thread-73 - "发送服务心跳命令成功",
"offset": 43997,
"source": "/data/logs/module/log.log",
"type": "module"
}
可以看到3个时间点1:publish time(2017-06-13T17:00:25+08:00);2:beats time(2017-06-13T09:00:24.871Z);3:app time(2017-06-13 17:00:19,701) 延迟会有10秒左右,请问这个怎么解决。。
filebeat.yml配置如下:
filebeat:
spool_size: 1024
idle_timeout: "5s"
prospectors:
-
document_type: "module"
paths:
- /data/logs/module/log.log
multiline.pattern: "^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3} -"
multiline.negate: true
multiline.match: after
output:
kafka:
hosts: ["127.0.0.1:9092"]
topic: '%{[type]}'
partition.round_robin:
reachable_only: false
max_message_bytes: 1000000
运行命令./filebeat -c filebeat.yml -d publish,可以在console看到相关信息输出:
2017-06-13T17:00:25+08:00 DBG Publish: {
"@timestamp": "2017-06-13T09:00:24.871Z",
"beat": {
"hostname": "localhost",
"name": "localhost",
"version": "5.4.0"
},
"input_type": "log",
"message": "2017-06-13 17:00:19,701 - INFO - Thread-73 - "发送服务心跳命令成功",
"offset": 43997,
"source": "/data/logs/module/log.log",
"type": "module"
}
可以看到3个时间点1:publish time(2017-06-13T17:00:25+08:00);2:beats time(2017-06-13T09:00:24.871Z);3:app time(2017-06-13 17:00:19,701) 延迟会有10秒左右,请问这个怎么解决。。
1 个回复
yiyezhiqiu425
赞同来自: