问题原因找到!!! logstash 采用日志中time 解析到@timestamp 时,match中的解析格式被拆解, 出现匹配异常!

Logstash | 作者 sun_changlong | 发布于2019年01月15日 | 阅读数:312

日志中time字段格式:
version=1.0 time="2019-01-15 18:20:05" dev="WAF01.PUB.BEIJING-B" pri="0"

match 中解析:
date {
#"time" => "2019-01-15 18:20:05"
match => ["time", "yyyy-MM-dd HH:mm:ss"]
target => "@timestamp"
locale => "en"
}

我是按照time格式依次对应的,但是报的异常信息只显示格式为 时分秒,
[WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"test", :_type=>"udp_test", :_routing=>nil}, #<LogStash::Event:0x7a64786c>], :response=>{"index"=>{"_index"=>"test", "_type"=>"udp_test", "_id"=>"19qPVWgBL1XXUvJfF182", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [time]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"2017-06-05 15:20:05\" is malformed at \" 15:20:05\""}}}}}

正常格式在这被认为是
\" 18:20:05\"

有人知道这是出于什么原因被拆解了么?
 
已邀请:

zqc0512 - andy zhou

赞同来自:

=号的原因吧,用kv解析下再试试。

ccsy

赞同来自:

这么试一下
date {
#"time" => "2019-01-15 18:20:05"
match => ["time", "yyyy-MM-dd HH:mm:ss","ISO8601"]
target => "@timestamp"
}

ccsy

赞同来自:

 
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:time}" }
}
date{
match => ["time", "yyyy-MM-dd HH:mm:ss", "ISO8601"]
target => "@timestamp"
}
mutate{
remove_field => ["time"]
}

sun_changlong

赞同来自:

找到问题的原因了,是因为原日志数据中包含type字段,造成解析冲突,但是不知道会报异常在date字段上

要回复问题请先登录注册