kafka数据：{
{"cluster":"qy_api_v2_pool","body_bytes_sent":"8579","http_versioncode":"Android_32"}\n
{"cluster":"qy_api_v2_pool","body_bytes_sent":"8579","http_versioncode":"Android_33"}\n
{"cluster":"qy_api_v2_pool","body_bytes_sent":"8579","http_versioncode":"Android_34"}\n
....
}
kafka团队因考虑性能问题，将原始日志多条合并一条发送（每一条用换行符分割），这样我读的kafka就必须拆成一条一条的写入到ES，不然数据就不准确了，请问这种需求该如何处理呢？
{"cluster":"qy_api_v2_pool","body_bytes_sent":"8579","http_versioncode":"Android_32"}\n
{"cluster":"qy_api_v2_pool","body_bytes_sent":"8579","http_versioncode":"Android_33"}\n
{"cluster":"qy_api_v2_pool","body_bytes_sent":"8579","http_versioncode":"Android_34"}\n
....
}
kafka团队因考虑性能问题，将原始日志多条合并一条发送（每一条用换行符分割），这样我读的kafka就必须拆成一条一条的写入到ES，不然数据就不准确了，请问这种需求该如何处理呢？
3 个回复
zhangyufu - 学习使我快乐
赞同来自:
kennywu76 - Wood
赞同来自:
https://www.elastic.co/guide/e ... .html
zhangxing - 一个背不好的锅的运维
赞同来自:
filter {
mutate {
split=>["message","
"]
}
还有一个小问题split中terminator默认是\n,但是我如下写法为什么切割不成功呢，不写terminator是可以的
filter {
split {
field => "message"
terminator => "\n"
}