要不要也来分享分享一下啊

修改filebeat索引名,在kibana里创建索引时没有索引到相关索引,好绝望啊~ 求大神指点迷津

Beats | 作者 MiaoRenFeng | 发布于2019年08月10日 | 阅读数:8859

这是我的配置文件

filebeat.inputs:
- type: log
enabled: true
paths:
-/var/log/test/*.log

filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false

setup.template.settings:
index.number_of_shards: 1

setup.kibana:
host: "127.0.0.1:5601"

setup.template.fields: "/home/filebeat-7.2.0-linux-x86_64/fields.yml"
setup.template.enabled: true
output.elasticsearch.index: "testgame01_log-%{+yyyy.MM.dd}"

setup.ilm.enabled: false

setup.template.overwrite: true
setup.template.name: "testgame01_log"
setup.template.pattern: "testgame01_log-*"


output.elasticsearch:
hosts: ["127.0.0.1:9200"]

processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
es的部分输出:
[2019-08-09T20:35:18,983][INFO ][o.e.c.m.MetaDataIndexTemplateService] [william01] adding template [.management-beats] for index patterns [.management-beats]
[2019-08-09T20:52:12,178][INFO ][o.e.c.m.MetaDataIndexTemplateService] [william01] adding template [testgame01_log] for index patterns [testgame01_log-*]
[2019-08-09T21:04:28,849][INFO ][o.e.c.m.MetaDataIndexTemplateService] [william01] adding template [.management-beats] for index patterns [.management-beats]
 
已经显示加载了,通过:GET /_cat/indices也能看到加载的索引名,但kibana创建索引模式里却找不到该索引,启动filebeat也没有显示已经连接到es ,下面时启动filebeat的输出部分输出:

 
log/elasticsearch/*_index_search_slowlog.json /var/log/elasticsearch/*_index_indexing_slowlog.json]
2019-08-09T21:07:00.611+0400 INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2019-08-09T21:07:00.611+0400 INFO cfgfile/reload.go:172 Config reloader started
2019-08-09T21:07:00.614+0400 INFO log/input.go:148 Configured paths: [/var/log/elasticsearch/*.log /var/log/elasticsearch/*_server.json]
2019-08-09T21:07:00.615+0400 INFO log/input.go:148 Configured paths: [/var/log/elasticsearch/*_index_search_slowlog.log /var/log/elasticsearch/*_index_indexing_slowlog.log /var/log/elasticsearch/*_index_search_slowlog.json /var/log/elasticsearch/*_index_indexing_slowlog.json]
2019-08-09T21:07:00.615+0400 INFO log/input.go:148 Configured paths: [/var/log/elasticsearch/*_access.log /var/log/elasticsearch/*_audit.log /var/log/elasticsearch/*_audit.json]
2019-08-09T21:07:00.615+0400 INFO log/input.go:148 Configured paths: [/var/log/elasticsearch/*_deprecation.log /var/log/elasticsearch/*_deprecation.json]
2019-08-09T21:07:00.615+0400 INFO log/input.go:148 Configured paths: [/var/log/elasticsearch/gc.log.[0-9]* /var/log/elasticsearch/gc.log]
2019-08-09T21:07:00.616+0400 INFO elasticsearch/client.go:166 Elasticsearch url: http://127.0.0.1:9200
2019-08-09T21:07:00.620+0400 INFO elasticsearch/client.go:735 Attempting to connect to Elasticsearch version 7.2.0
2019-08-09T21:07:00.663+0400 INFO input/input.go:114 Starting input of type: log; ID: 14951128314695463370
2019-08-09T21:07:00.663+0400 INFO input/input.go:114 Starting input of type: log; ID: 1799075452601998653
2019-08-09T21:07:00.663+0400 INFO input/input.go:114 Starting input of type: log; ID: 15011281060050230607
2019-08-09T21:07:00.663+0400 INFO input/input.go:114 Starting input of type: log; ID: 12232882340116258706
2019-08-09T21:07:00.663+0400 INFO input/input.go:114 Starting input of type: log; ID: 641225316547766288
2019-08-09T21:07:00.663+0400 INFO cfgfile/reload.go:227 Loading of config files completed.
2019-08-09T21:07:03.603+0400 INFO add_cloud_metadata/add_cloud_metadata.go:347 add_cloud_metadata: hosting provider type not detected.
2019-08-09T21:07:30.609+0400 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":53}},"total":{"ticks":80,"time":{"ms":83},"value":80},"user":{"ticks":30,"time":{"ms":30}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":6},"info":{"ephemeral_id":"59310862-d8f3-4242-ad3c-61f14e155a58","uptime":{"ms":30021}},"memstats":{"gc_next":4226128,"memory_alloc":2871288,"memory_total":10006864,"rss":29028352},"runtime":{"goroutines":76}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"type":"elasticsearch"},"pipeline":{"clients":11,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":8},"load":{"1":0.14,"15":0.1,"5":0.12,"norm":{"1":0.0175,"15":0.0125,"5":0.015}}}}}}
 
在filebeat的日志里也没有看到有错误的输出,根据官方文档,我尝试过修改各种属性值,实在是找不出问题所在了,好绝望啊,这问题已经搞了很久了,求大神指点秘境啊
已邀请:

bellengao - 博客: https://www.jianshu.com/u/e0088e3e2127

赞同来自: MiaoRenFeng

修改默认索引名称需要在output里配置index参数;看filebeat日志是没有采集到日志,所以没有创建索引,连接es没有问题

要回复问题请先登录注册