看,灰机...

filebeat每天下午自动关闭问题

Beats | 作者 Kuange | 发布于2019年11月27日 | 阅读数:6724

我用的filebeat7.3解压版本,在两台主机上部署两个beat,每天下午6点钟左右,这两个filebeat几乎同时关闭(前后相差10毫秒左右),日志中也看不到异常,下面是日志:
2019-11-25T18:04:09.341+0800 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":790,"time":{"ms":14}},"total":{"ticks":2660,"time":{"ms":36},"value":2660},"user":{"ticks":1870,"time":{"ms":22}}},"handles":{"limit":{"hard":65536,"soft":65536},"open":6},"info":{"ephemeral_id":"9b083e50-ee97-4b0b-879b-43a6c4a6b312","uptime":{"ms":900142}},"memstats":{"gc_next":5010656,"memory_alloc":2529576,"memory_total":54489744},"runtime":{"goroutines":23}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":4}},"system":{"load":{"1":0.07,"15":0.4,"5":0.29,"norm":{"1":0.0088,"15":0.05,"5":0.0363}}}}}}
2019-11-25T18:04:10.278+0800 INFO beater/filebeat.go:443 Stopping filebeat
2019-11-25T18:04:10.279+0800 INFO crawler/crawler.go:139 Stopping Crawler
2019-11-25T18:04:10.279+0800 INFO crawler/crawler.go:149 Stopping 1 inputs
2019-11-25T18:04:10.280+0800 INFO cfgfile/reload.go:229 Dynamic config reloader stopped
2019-11-25T18:04:10.282+0800 INFO input/input.go:149 input ticker stopped
2019-11-25T18:04:10.284+0800 INFO input/input.go:167 Stopping Input: 16752264264616077088
2019-11-25T18:04:10.285+0800 INFO crawler/crawler.go:165 Crawler stopped
2019-11-25T18:04:10.285+0800 INFO registrar/registrar.go:367 Stopping Registrar
2019-11-25T18:04:10.285+0800 INFO registrar/registrar.go:293 Ending Registrar
2019-11-25T18:04:10.306+0800 INFO [monitoring] log/log.go:153 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":800,"time":{"ms":807}},"total":{"ticks":2680,"time":{"ms":2693},"value":2680},"user":{"ticks":1880,"time":{"ms":1886}}},"handles":{"limit":{"hard":65536,"soft":65536},"open":6},"info":{"ephemeral_id":"9b083e50-ee97-4b0b-879b-43a6c4a6b312","uptime":{"ms":901100}},"memstats":{"gc_next":5010656,"memory_alloc":2835768,"memory_total":54795936,"rss":36265984},"runtime":{"goroutines":13}},"filebeat":{"events":{"added":4,"done":4},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"type":"logstash"},"pipeline":{"clients":0,"events":{"active":0,"filtered":4,"total":4}}},"registrar":{"states":{"current":4,"update":4},"writes":{"success":5,"total":5}},"system":{"cpu":{"cores":8},"load":{"1":0.07,"15":0.4,"5":0.29,"norm":{"1":0.0088,"15":0.05,"5":0.0363}}}}}}
2019-11-25T18:04:10.307+0800 INFO [monitoring] log/log.go:154 Uptime: 15m1.115983263s
2019-11-25T18:04:10.307+0800 INFO [monitoring] log/log.go:131 Stopping metrics logging.
2019-11-25T18:04:10.307+0800 INFO instance/beat.go:431 filebeat stopped.
已邀请:

Kuange - 90后IT男

赞同来自: a953546597

安装的是tar压缩版本,我是通过nohup方式启动的,有人知道这个问题怎么解决吗

imwesley

赞同来自: a953546597

我也遇到了这个问题,看了下退出日志如下:
 
2019-12-13T18:41:30.741+0800    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":930,"time":{"ms":2}},"total":{"ticks":2620,"time":{"ms":4},"value":2620},"user":{"ticks":1690,"time":{"ms":2}}},"handles":{"limit":{"hard":131072,"soft":65536},"open":8},"info":{"ephemeral_id":"3ceef3ed-ff4c-49f8-b20e-c8ef157d191b","uptime":{"ms":11400164}},"memstats":{"gc_next":8144336,"memory_alloc":5222496,"memory_total":345942544},"runtime":{"goroutines":26}},"filebeat":{"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":10}},"system":{"load":{"1":0.09,"15":0.17,"5":0.14,"norm":{"1":0.0225,"15":0.0425,"5":0.035}}}}}}
2019-12-13T18:42:00.741+0800 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":930,"time":{"ms":1}},"total":{"ticks":2630,"time":{"ms":6},"value":2630},"user":{"ticks":1700,"time":{"ms":5}}},"handles":{"limit":{"hard":131072,"soft":65536},"open":8},"info":{"ephemeral_id":"3ceef3ed-ff4c-49f8-b20e-c8ef157d191b","uptime":{"ms":11430163}},"memstats":{"gc_next":5097280,"memory_alloc":2605536,"memory_total":346301744,"rss":-888832},"runtime":{"goroutines":26}},"filebeat":{"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":10}},"system":{"load":{"1":0.11,"15":0.17,"5":0.14,"norm":{"1":0.0275,"15":0.0425,"5":0.035}}}}}}
2019-12-13T18:42:28.303+0800 INFO beater/filebeat.go:443 Stopping filebeat
2019-12-13T18:42:28.304+0800 INFO crawler/crawler.go:139 Stopping Crawler
2019-12-13T18:42:28.304+0800 INFO crawler/crawler.go:149 Stopping 1 inputs
2019-12-13T18:42:28.304+0800 INFO input/input.go:149 input ticker stopped
2019-12-13T18:42:28.304+0800 INFO input/input.go:167 Stopping Input: 10479034256437976383
2019-12-13T18:42:28.306+0800 INFO cfgfile/reload.go:229 Dynamic config reloader stopped
2019-12-13T18:42:28.306+0800 INFO log/harvester.go:274 Reader was closed: /opt/ebiz/webapps/ebiz-claim-center/logs/application.log. Closing.
2019-12-13T18:42:28.306+0800 INFO crawler/crawler.go:165 Crawler stopped
2019-12-13T18:42:28.306+0800 INFO registrar/registrar.go:367 Stopping Registrar
2019-12-13T18:42:28.306+0800 INFO registrar/registrar.go:293 Ending Registrar
2019-12-13T18:42:28.321+0800 INFO [monitoring] log/log.go:153 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":930,"time":{"ms":938}},"total":{"ticks":2630,"time":{"ms":2644},"value":2630},"user":{"ticks":1700,"time":{"ms":1706}}},"handles":{"limit":{"hard":131072,"soft":65536},"open":7},"info":{"ephemeral_id":"3ceef3ed-ff4c-49f8-b20e-c8ef157d191b","uptime":{"ms":11457743}},"memstats":{"gc_next":5097280,"memory_alloc":2989608,"memory_total":346685816,"rss":15691776},"runtime":{"goroutines":12}},"filebeat":{"events":{"added":6248,"done":6248},"harvester":{"closed":8,"open_files":0,"running":0,"started":8},"input":{"log":{"files":{"renamed":1}}}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"events":{"acked":6223,"batches":98,"failed":652,"total":6875},"read":{"bytes":498},"type":"logstash","write":{"bytes":719476,"errors":20}},"pipeline":{"clients":0,"events":{"active":0,"filtered":25,"published":6223,"retry":3352,"total":6248},"queue":{"acked":6223}}},"registrar":{"states":{"current":10,"update":6248},"writes":{"success":98,"total":98}},"system":{"cpu":{"cores":4},"load":{"1":0.12,"15":0.17,"5":0.15,"norm":{"1":0.03,"15":0.0425,"5":0.0375}}}}}}
2019-12-13T18:42:28.321+0800 INFO [monitoring] log/log.go:154 Uptime: 3h10m57.744547981s
2019-12-13T18:42:28.321+0800 INFO [monitoring] log/log.go:131 Stopping metrics logging.
2019-12-13T18:42:28.321+0800 INFO instance/beat.go:432 filebeat stopped.

pzxsheng

赞同来自:

请教下,解决了吗?怎样解决的?谢谢

Florence - 待我光头之日

赞同来自:

请问解决了吗 

henryzhang

赞同来自:

如果使用root权限启动的filebeat,关闭Shell窗口后,进程会挂掉

要回复问题请先登录注册