流程: filebeat读取java日志->es中配置了pipeline预处理->根据模板写入es
问题: filebeat做了multiline配置,可以多行合并,但是经过pipeline的grok预处理以后就只能读取第一行了.如果清空pipeline的processors配置则一切正常.
这是我filebeat的配置
这是我pipeline
这是我template
这是我的demo文件
问题: filebeat做了multiline配置,可以多行合并,但是经过pipeline的grok预处理以后就只能读取第一行了.如果清空pipeline的processors配置则一切正常.
这是我filebeat的配置
filebeat.inputs:
- type: log
enabled: true
paths:
- /Users/liule/Desktop/demo.log
multiline:
pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
negate: true
match: after
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
setup.ilm.enabled: false
output.elasticsearch:
hosts: ["localhost:9200"]
pipeline: "java_pipeline"
index: "iotrta-%{+yyyy.MM.dd}"
setup.template.name: "log-java"
setup.template.pattern: "log-java-*"
queue.mem:
events: 256
flush.min_events: 128
这是我pipeline
这是我template
这是我的demo文件
2019-10-17 11:29:45.011 INFO 12365 --- [main] c.d.AbnormalInterruptTaskFixRunner : dddddddddddddd
2019-10-17 11:30:45.011 INFO 12365 --- [main] c.d.AbnormalInterruptTaskFixRunner : test
2019-10-17 11:31:34.243 ERROR 12365 --- [XNIO-2 I/O-10] io.undertow.request.io : UT005090: Unexpected failure
java.dddddd.dddddddd: dddddddddddddd
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
2019-10-17 11:31:34.243 ERROR 12365 --- [XNIO-2 I/O-11] io.undertow.request.io : UT005090: Unexpected failure
java.lang.NoClassDefFoundError: Could not initialize class io.undertow.UndertowMessages
at io.undertow.server.DefaultByteBufferPool$DefaultPooledBuffer.getBuffer(DefaultByteBufferPool.java:260) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
at io.undertow.protocols.ssl.SslConduit.doUnwrap(SslConduit.java:833) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
at io.undertow.protocols.ssl.SslConduit.doHandshake(SslConduit.java:648) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
at io.undertow.protocols.ssl.SslConduit.access$900(SslConduit.java:63) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
at io.undertow.protocols.ssl.SslConduit$SslReadReadyHandler.readReady(SslConduit.java:1127) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:88) [xnio-nio-3.3.8.Final.jar!/:3.3.8.Final]
at org.xnio.nio.WorkerThread.run(WorkerThread.java:561) [xnio-nio-3.3.8.Final.jar!/:3.3.8.Final]
1 个回复
CononYc
赞同来自:
参阅elasticsearch 官方文档 Ingest node ->grok-processor
表达式参阅:github.com/kkos/oniguruma/blob/master/doc/RE
匹配多行文本的表达式为(.m:.*)
在pipeline下配置
使用