有个人长的像洋葱,走着走着就哭了…….

filebeat decode_json 的 max_depth 选项不生效

Beats | 作者 qadet123 | 发布于2019年11月21日 | 阅读数:2818

我有一些嵌套了很多层的 json log,filebeat 在做采集的时候如果不使用 decode_json_fields,拿到的结果长这样
{
"@timestamp": "2019-11-21T08:51:23.441Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "7.4.2"
},
"agent": {
"id": "3eb2ee6c-e426-4c1c-b863-91071de38ce8",
"version": "7.4.2",
"type": "filebeat",
"ephemeral_id": "66a3013c-d245-4d30-93f5-ecb34dca0ca5",
"hostname": "testo-master-022"
},
"log": {
"offset": 12258583,
"file": {
"path": "/var/log/audit/audit.log"
}
},
"message": "{\"kind\":\"Event\",\"apiVersion\":\"audit.k8s.io/v1beta1\",\"metadata\":{\"creationTimestamp\":\"2019-11-21T08:41:21Z\"},\"level\":\"Metadata\",\"timestamp\":\"2019-11-21T08:41:21Z\",\"auditID\":\"cd6a0bc7-78ab-4f6a-8d61-a744f4847882\",\"stage\":\"ResponseComplete\",\"requestURI\":\"/api/v1/namespaces/kube-system/endpoints/kube-controller-manager\",\"verb\":\"get\",\"user\":{\"username\":\"system:kube-controller-manager\",\"groups\":[\"system:authenticated\"]},\"sourceIPs\":[\"192.168.50.22\"],\"objectRef\":{\"resource\":\"endpoints\",\"namespace\":\"kube-system\",\"name\":\"kube-controller-manager\",\"apiVersion\":\"v1\"},\"responseStatus\":{\"metadata\":{},\"code\":200},\"requestReceivedTimestamp\":\"2019-11-21T08:41:21.034575Z\",\"stageTimestamp\":\"2019-11-21T08:41:21.036580Z\"}",
"input": {
"type": "log"
},
"ecs": {
"version": "1.1.0"
},
"host": {
"name": "testo-master-022"
}
}
我希望 fileabeat 对我的 json 做一层解析,只做一层,配置如下
filebeat.registry.path: /var/log/audit.pos

filebeat.inputs:

- type: log
enabled: true
paths:
- /var/log/audit/audit.log

processors:
- decode_json_fields:
fields: ["message"]
process_array: false
max_depth: 1

output.console:
pretty: true
我得到的结果,json 被完整的解析了,并没有如预期那样只解析第一层
{
"@timestamp": "2019-11-21T09:21:37.140Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "7.4.2"
},
"log": {
"offset": 12059832,
"file": {
"path": "/var/log/audit/audit.log"
}
},
"message": {
"kind": "Event",
"metadata": {
"creationTimestamp": "2019-11-21T08:40:53Z"
},
"verb": "list",
"sourceIPs": [
"127.0.0.1"
],
"requestReceivedTimestamp": "2019-11-21T08:40:53.358938Z",
"stage": "RequestReceived",
"auditID": "02ca2680-ca33-4050-b267-6a9280f717e8",
"requestURI": "/apis/admissionregistration.k8s.io/v1alpha1/initializerconfigurations",
"user": {
"groups": [
"system:masters"
],
"username": "system:apiserver",
"uid": "f9502c83-4631-498e-9620-f7824fa727a2"
},
"objectRef": {
"resource": "initializerconfigurations",
"apiGroup": "admissionregistration.k8s.io",
"apiVersion": "v1alpha1"
},
"stageTimestamp": "2019-11-21T08:40:53.358938Z",
"apiVersion": "audit.k8s.io/v1beta1",
"level": "Metadata",
"timestamp": "2019-11-21T08:40:53Z"
},
"input": {
"type": "log"
},
"host": {
"name": "testo-master-022"
},
"agent": {
"type": "filebeat",
"ephemeral_id": "e34fe610-1972-4273-a9ee-014852357300",
"hostname": "testo-master-022",
"id": "3eb2ee6c-e426-4c1c-b863-91071de38ce8",
"version": "7.4.2"
},
"ecs": {
"version": "1.1.0"
}
}
请问是我的配置哪里有问题吗
已邀请:

jessehuang

赞同来自:

大佬 你这个问题解决了吗? 我也被这个困扰很久,有偿寻找解决方案

要回复问题请先登录注册