metricbeat 无法连续发送数据

作者 Jiehui Tang | 发布于2018年09月06日 | 阅读数:257

metricbeat 版本:5.4.0
安装平台: linux x86_64;
在metricbeat.yml中配置了下面的processor后,metricbeat仅仅发送第一次采集到的数据,后面的数据都不发送了.
metricbeat.yml 中processor配置如下:
processors:
- drop_event:
when:
not:
regexp:
system.socket.remote.ip: "10.240.120.*"

对应的metricbeat日志内容如下:
2018-09-06T19:19:50+08:00 INFO Home path: [/home/bcacspusr/filebeatAgent/metricbeat-5.4.0-linux-x86_64] Config path: [/home/bcacspusr/filebeatAgent/metricbeat-5.4.0-linux-x86_64] Data path: [/home/bcacspusr/filebeatAgent/metricbeat-5.4.0-linux-x86_64/data] Logs path: [/home/bcacspusr/filebeatAgent/metricbeat-5.4.0-linux-x86_64/logs]
2018-09-06T19:19:50+08:00 INFO Setup Beat: metricbeat; Version: 5.4.0
2018-09-06T19:19:50+08:00 INFO Activated console as output plugin.
2018-09-06T19:19:50+08:00 INFO Publisher name: 10.230.165.183
2018-09-06T19:19:50+08:00 INFO Flush Interval set to: 1s
2018-09-06T19:19:50+08:00 INFO Max Bulk Size set to: 2048
2018-09-06T19:19:50+08:00 INFO Register [ModuleFactory:[docker, mongodb, mysql, postgresql, system], MetricSetFactory:[apache/status, ceph/cluster_disk, ceph/cluster_health, ceph/monitor_health, ceph/pool_disk, couchbase/bucket, couchbase/cluster, couchbase/node, docker/container, docker/cpu, docker/diskio, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, haproxy/info, haproxy/stat, jolokia/jmx, kafka/consumergroup, kafka/partition, mongodb/dbstats, mongodb/status, mysql/status, nginx/stubstatus, php_fpm/pool, postgresql/activity, postgresql/bgwriter, postgresql/database, prometheus/collector, prometheus/stats, redis/info, redis/keyspace, system/core, system/cpu, system/diskio, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/process, system/socket, zookeeper/mntr]]
2018-09-06T19:19:50+08:00 INFO socket process info will only be available for metricbeat because the process is running as a non-root user
2018-09-06T19:19:50+08:00 INFO metricbeat start running.
2018-09-06T19:20:20+08:00 INFO Non-zero metrics in the last 30s: fetches.system-process.events=1 fetches.system-process.success=1 fetches.system-socket.events=191 fetches.system-socket.success=1 libbeat.publisher.messages_in_worker_queues=6 libbeat.publisher.published_events=6
2018-09-06T19:20:50+08:00 INFO Non-zero metrics in the last 30s: fetches.system-process.events=1 fetches.system-process.success=1 fetches.system-socket.events=4 fetches.system-socket.success=1
2018-09-06T19:21:20+08:00 INFO Non-zero metrics in the last 30s: fetches.system-process.events=1 fetches.system-process.success=1 fetches.system-socket.events=3 fetches.system-socket.success=1
2018-09-06T19:21:50+08:00 INFO Non-zero metrics in the last 30s: fetches.system-process.events=1 fetches.system-process.success=1 fetches.system-socket.events=5 fetches.system-socket.success=1
2018-09-06T19:22:20+08:00 INFO Non-zero metrics in the last 30s: fetches.system-process.events=1 fetches.system-process.success=1 fetches.system-socket.events=3 fetches.system-socket.success=1
2018-09-06T19:22:24+08:00 INFO Total non-zero values: fetches.system-process.events=6 fetches.system-process.success=6 fetches.system-socket.events=212 fetches.system-socket.success=6 libbeat.publisher.messages_in_worker_queues=6 libbeat.publisher.published_events=6
2018-09-06T19:22:24+08:00 INFO Uptime: 2m34.580274917s
2018-09-06T19:22:24+08:00 INFO metricbeat stopped.


已邀请:

Jiehui Tang

赞同来自:

processor配置换成下面这个,就可以连续发送数据了。
processors:- drop_event:
    when:
      equals:
        system.socket.direction: "listening"

zqc0512 - andy zhou

赞同来自:

2018-09-06T19:19:50+08:00 INFO socket process info will only be available for metricbeat because the process is running as a non-root user
try root user.
 

Jiehui Tang

赞同来自:

已经解决!
来自于 MetricBeat 5.4 官方文档里
https://www.elastic.co/guide/e ... .html
The system socket metricset reports an event for each new TCP socket that it sees. 
每次metricbeat只会采集到新增的TCP连接,只有第一次会采集系统内所有的TCP连接并处理,此后,只关注新增的TCP连接。

要回复问题请先登录注册