Q:有两个人掉到陷阱里了,死的人叫死人,活人叫什么?

Filebeat 遇到这种情况怎么处理

Beats | 作者 minisheep | 发布于2019年09月21日 | 阅读数:3223

固定格式为:一行Timestamp开头,一行Message开头,一行------------,现在需要过滤掉-------------这行,且Timestamp和Message合并成同行,一直写filebeat脚本都不行,求大神相助。
Timestamp: 2019-09-11 12:15:34,129
Message: TaskSchedulingThreadPool configured with max concurrency of 10 and TaskScheduler ThreadPoolTaskScheduler.
--------------------------------------------------------------------------
Timestamp: 2019-09-11 12:15:34,176
Message: Batch acquisition of 0 triggers
--------------------------------------------------------------------------
Timestamp: 2019-09-11 12:15:34,192
Message: Batch acquisition of 1 triggers
--------------------------------------------------------------------------
已邀请:

stone_xy

赞同来自:

用multiline匹配,再把--开头的行exclude就可以了。
- type: log

# Change to true to enable this input configuration.
enabled: true

# Paths that should be crawled and fetched. Glob based paths.
paths:
- /opt/es/tmp/filebeat/*.log
multiline.pattern: "^Message:"
multiline.negate: false
multiline.match: after
exclude_lines: "^--"
输出如下:
{
"@timestamp": "2019-09-21T03:29:41.434Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "7.3.1"
},
"host": {
"name": "ecs-test"
},
"agent": {
"hostname": "ecs-test",
"id": "3493a9d8-01ac-4fb8-ab76-f8f9cae51b3e",
"version": "7.3.1",
"type": "filebeat",
"ephemeral_id": "a3a2382f-50c0-4a28-b998-d1addb42db3d"
},
"log": {
"offset": 0,
"file": {
"path": "/opt/es/tmp/filebeat/test.log"
},
"flags": [
"multiline"
]
},
"message": "Timestamp: 2019-09-11 12:15:34,129\nMessage: TaskSchedulingThreadPool configured with max concurrency of 10 and TaskScheduler ThreadPoolTaskScheduler.",
"input": {
"type": "log"
},
"ecs": {
"version": "1.0.1"
}
}
{
"@timestamp": "2019-09-21T03:29:41.434Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "7.3.1"
},
"ecs": {
"version": "1.0.1"
},
"host": {
"name": "ecs-test"
},
"agent": {
"version": "7.3.1",
"type": "filebeat",
"ephemeral_id": "a3a2382f-50c0-4a28-b998-d1addb42db3d",
"hostname": "ecs-test",
"id": "3493a9d8-01ac-4fb8-ab76-f8f9cae51b3e"
},
"log": {
"offset": 225,
"file": {
"path": "/opt/es/tmp/filebeat/test.log"
},
"flags": [
"multiline"
]
},
"message": "Timestamp: 2019-09-11 12:15:34,176\nMessage: Batch acquisition of 0 triggers",
"input": {
"type": "log"
}
}
{
"@timestamp": "2019-09-21T03:29:41.434Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "7.3.1"
},
"log": {
"file": {
"path": "/opt/es/tmp/filebeat/test.log"
},
"flags": [
"multiline"
],
"offset": 376
},
"message": "Timestamp: 2019-09-11 12:15:34,192\nMessage: Batch acquisition of 1 triggers",
"input": {
"type": "log"
},
"ecs": {
"version": "1.0.1"
},
"host": {
"name": "ecs-test"
},
"agent": {
"type": "filebeat",
"ephemeral_id": "a3a2382f-50c0-4a28-b998-d1addb42db3d",
"hostname": "ecs-test",
"id": "3493a9d8-01ac-4fb8-ab76-f8f9cae51b3e",
"version": "7.3.1"
}
}

 
 
 

zqc0512 - andy zhou

赞同来自:

只合到一行不拆分子段么?
 

要回复问题请先登录注册