橡皮、老虎皮、狮子皮哪一个最不好?

filebeat收集日志问题转义符

Beats | 作者 imsa | 发布于2018年04月16日 | 阅读数:7379

原始日志
101.90.254.100 - - [16/Apr/2018:18:01:11 +0800] "GET /new/api/site_login_ijf.htm?referer=&user_fingerprint=&app_key=adhu5487540&callback=getSiteLoginInfo15238728&v=9663 HTTP/1.1" 200 490 "https://display.test.com/site_ ... ot%3B "Mozilla/5.0 (Linux; U; Android 7.0; zh-CN; KNT-AL20 Build/HUAWEIKNT-AL20) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/40.0.2214.89 UCBrowser/11.6.4.950 UWS/2.11.0.46 Mobile Safari/537.36 UCBS/2.11.0.46_180226200455 NebulaSDK/1.8.100112 Nebula AlipayDefined(nt:4G,ws:360|0|4.0) AliApp(AP/10.1.18.708) AlipayClient/10.1.18.708 Language/zh-Hans useStatusBar/true" "-" "display.test.com" 127.0.0.1:17290 200 - "text/javascript; charset=utf-8" 0.028 > 0.028
保存到kafka中的
{"@timestamp":"2018-04-16T10:01:15.120Z","beat":{"hostname":"sh-103250","name":"sh-103250","version":"5.4.0"},"fields":{"type":"nginx_log"},"input_type":"log","message":"101.90.254.100 - - [16/Apr/2018:18:01:11 +0800] \"GET /new/api/site_login_ijf.htm?referer=\u0026user_fingerprint=\u0026app_key=adhu5487540dc\u0026callback=getSiteLoginInfo1523872870\u0026v=9663 HTTP/1.1\" 200 490 \"https://display.test.com/site_ ... 540dc\" \"Mozilla/5.0 (Linux; U; Android 7.0; zh-CN; KNT-AL20 Build/HUAWEIKNT-AL20) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/40.0.2214.89 UCBrowser/11.6.4.950 UWS/2.11.0.46 Mobile Safari/537.36 UCBS/2.11.0.46_180226200455 NebulaSDK/1.8.100112 Nebula AlipayDefined(nt:4G,ws:360|0|4.0) AliApp(AP/10.1.18.708) AlipayClient/10.1.18.708 Language/zh-Hans useStatusBar/true\" \"-\" \"display.test.com\" 127.0.0.1:17290 200 - \"text/javascript; charset=utf-8\" 0.028 \u003e 0.028","offset":19006739,"source":"/usr/local/nginx/logs/display.test.com.log","tags":["nginx_log"],"type":"log"}

多了不少转义符,另外logstash从kafka中读取会更多转义符,请解决
已邀请:

ribbel - linux运维

赞同来自:

表示遇到了同样的问题。。。

football025

赞同来自:

我也遇到同样的问题  & 号 变成了\u0026

football025

赞同来自:

有找到处理办法吗,我找了几天都搞不定这个转义符问题

a505100745

赞同来自:

这个到es里就没了吧

football025

赞同来自:

终于找到了一个参考https://discuss.elastic.co/t/f ... 827/3
 
input的时候要codec => json
output 要 codec => line { format => "%{message}"}

要回复问题请先登录注册