用了Elasticsearch,一口气上5T

filebeat pipline处理timestamp 报错,求助

匿名 | 发布于2019年11月30日 | 阅读数:2659

不想使用logstash 来解析字段所以想直接使用filebeat
es 里面的时间想使用原始日志的时间,现在使用pipline 报错,:
019-11-30T13:36:04.545+0800 DEBUG [elasticsearch] elasticsearch/client.go:337 PublishEvents: 1 events have been published to elasticsearch in 1.332163ms.
2019-11-30T13:36:04.545+0800 DEBUG [elasticsearch] elasticsearch/client.go:544 Bulk item insert failed (i=0, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: unable to parse date [2019-11-25 10:02:27:901]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: unable to parse date [2019-11-25 10:02:27:901]","caused_by":{"type":"illegal_argument_exception","reason":"unable to parse date [2019-11-25 10:02:27:901]","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: \"2019-11-25 10:02:27:901\" is malformed at \" 10:02:27:901\""}}},"header":{"processor_type":"date"}}
2019-11-30T13:36:04.889+0800 ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure
 
 
 
pipline
PUT /_ingest/pipeline/test-pipeline/
{
  "description" : "test-pipeline",
  "processors" : [
    {
      "grok" :{
        "field" : "message",
        "patterns" : ["%{TIMESTAMP_ISO8601:time}"]
      },
    "remove": {
      "field": "@timestamp"
    }
  },
    {
      "date" : {
        "field" : "time",
        "target_field" : "@timestamp",
        "formats" : ["ISO8601"],
        "timezone" : "UTC"
      }
    }
  ]
}
已邀请:

要回复问题请先登录注册