ELK收集网络设备日志
分步走：
1、新建一台Linux虚拟机做为rsyslog服务器接收网络设备日志
2、在上述虚拟机上部署filebeat，将接收到的网络日志发送到logstash上
3、最后通过logstash针对性配置将日志发送给es并在kibana上显示。
1、部署rsyslog服务器
1、新建rsyslog配置文件 [root@localhost ~]# vi /etc/rsyslog.d/10-network-output.conf # 定义模板：按来源IP和日期分割日志文件 # 格式：/var/log/network-devices/<源IP>/<年-月-日>.log template(name="NetworkDeviceLog" type="string" string="/home/network-devices/%fromhost-ip%/%$YEAR%-%$MONTH%-%$DAY%.log" ) # 规则：将所有来自网络的日志（排除本地 localhost）写入上述模板 # 注意：这里假设网络设备日志没有特定的 facility 限制，如果有，可改为 local0.* 等 :fromhost-ip, !isequal, "127.0.0.1" ?NetworkDeviceLog # 确保目录存在并设置权限 # mkdir -p /var/log/network-devices # chown -R syslog:adm /var/log/network-devices (CentOS可能是 root:root 或 syslogs:syslogs)2、启动rsyslog服务，并检查端口514是否存在
[root@localhost ~]# systemctl enable rsyslog [root@localhost ~]# systemctl start rsyslog [root@localhost ~]# systemctl status rsyslog ● rsyslog.service - System Logging Service Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2026-06-24 14:22:55 CST; 1h 45min ago Docs: man:rsyslogd(8) http://www.rsyslog.com/doc/ Main PID: 28876 (rsyslogd) CGroup: /system.slice/rsyslog.service └─28876 /usr/sbin/rsyslogd -n Jun 24 14:22:55 localhost.localdomain systemd[1]: Starting System Logging Service... Jun 24 14:22:55 localhost.localdomain rsyslogd[28876]: [origin software="rsyslogd" swVersion="8.24.0-55.el7" x-pid="28876" x-info="http://www.rsyslog.com"] start Jun 24 14:22:55 localhost.localdomain systemd[1]: Started System Logging Service. [root@localhost ~]# netstat -lntup |grep 514 tcp 0 0 0.0.0.0:514 0.0.0.0:* LISTEN 28876/rsyslogd tcp6 0 0 :::514 :::* LISTEN 28876/rsyslogd udp 0 0 0.0.0.0:514 0.0.0.0:* 28876/rsyslogd udp6 0 0 :::514 :::* 28876/rsyslogd3、网络设备上设置策略将选定日志发送到rsyslog的514端口(可由网工配置)
4、检查是否有日志接收到
[root@localhost home]# ll /home/network-devices/10.10.254.10/ total 32512 -rw-------. 1 root root 20922019 Jun 24 16:11 2026-06-24.log 有日志写入5、在rsyslog上部署filebeat并修改配置文件
部署过程忽略 [root@localhost home]# vim /opt/filebeat/filebeat.yml filebeat.inputs: - type: log id: logs-app-network-devices #ID自定义 enabled: true paths: - /home/network-devices/*/*.log #日志路径自定义 #tags: ["network-device"] encoding: utf-8 fields: log_source: "network-devices" #系统-必填 区分索引 log_source_env: "prod" #环境 fields_under_root: false # multiline: # pattern: '^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{3}' # negate: true # match: after # max_lines: 1000 scan_frequency: 10s close_inactive: 1m output.logstash: hosts: ["10.10.200.35:8888"] compression_level: 3 loadbalance: true bulk_max_size: 2048 logging.level: info logging.to_files: true logging.files: path: /opt/filebeat/logs #目录自定义 name: filebeat.log keepfiles: 76、转到logstash服务器，新建专门接收网络日志的conf
[root@elk-lo-node03 pipeline]# pwd /opt/logstash/config/pipeline [root@elk-lo-node03 pipeline]# vim network-device-log.conf input { beats { port => 8888 #与filebeat配置的发送端口一致 # 可选：限制只接受来自特定 IP 的连接，增强安全性 # host => "0.0.0.0" } } filter { # 1. 识别来源：根据 Filebeat 中定义的 fields.log_source 进行判断 if [fields][log_source] == "network-devices" { # 2. 添加标签：便于在 Kibana 中快速筛选或创建专用视图 mutate { add_tag => ["network_device_log"] add_field => { "[@metadata][target_index]" => "logs-app-network-devices" } } } # 其他来源的日志可以在这里添加 else if 分支处理 } output { # 调试输出：确认数据结构和字段是否正确（生产环境稳定后可注释掉） stdout { codec => rubydebug } # 输出到 Elasticsearch elasticsearch { hosts => ["https://10.10.200.31:9200"] # 动态索引名称： # 最终生成的索引名为: logs-app-network-2026.06.24 index => "%{[@metadata][target_index]}-%{+yyyy.MM.dd}" # 认证信息 user => "elastic" password => "JcJv*N7rUT6fE6fik4oY" # SSL 配置（根据你的 ES 集群实际情况调整） ssl_certificate_verification => false # 模板管理（可选）：如果希望自动应用映射模板 # manage_template => true # template_name => "network-log-template" } }7、因为pipeline包含多个conf，需要额外修改pipelines.yml注明
[root@elk-lo-node03 config]# ll total 48 -rw-r--r-- 1 root root 2924 Apr 1 17:49 jvm.options -rw-r--r-- 1 root root 8680 Apr 1 17:49 log4j2.properties -rw-r--r-- 1 root root 502 Jun 4 14:37 logstash.conf -rw-r--r-- 1 root root 342 Apr 1 17:49 logstash-sample.conf -rw-r--r-- 1 root root 15745 Apr 1 17:49 logstash.yml drwxr-xr-x 2 root root 98 Jun 24 16:21 pipeline -rw-r--r-- 1 root root 837 Jun 16 16:54 pipelines.yml -rw-r--r-- 1 root root 1696 Apr 1 17:49 startup.options [root@elk-lo-node03 config]# pwd /opt/logstash/config [root@elk-lo-node03 config]# vim pipelines.yml - pipeline.id: beats-elk-log path.config: "/opt/logstash/config/pipeline/beats-elk-log.conf" pipeline.workers: 2 pipeline.batch.size: 125 pipeline.batch.delay: 50 queue.type: persisted queue.max_bytes: 1gb queue.checkpoint.acks: 1024 queue.drain: false - pipeline.id: windows-winlogbeat-log path.config: "/opt/logstash/config/pipeline/windows-winlogbeat-log.conf" pipeline.workers: 2 pipeline.batch.size: 125 pipeline.batch.delay: 50 queue.type: persisted queue.max_bytes: 1gb queue.checkpoint.acks: 1024 queue.drain: false - pipeline.id: network-device-log path.config: "/opt/logstash/config/pipeline/network-device-log.conf" pipeline.workers: 2 pipeline.batch.size: 125 pipeline.batch.delay: 50 queue.type: persisted queue.max_bytes: 1gb queue.checkpoint.acks: 1024 queue.drain: false8、重启logstash，并新开窗口检查是否有接收到网络设备日志
[root@elk-lo-node03 config]# systemctl restart logstash [root@elk-lo-node03 config]# journalctl -u logstash.service -f Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "message" => "Jun 24 16:23:29 10.10.254.10 5823434235018048(root) 43240501 HillstoneNetworks#Event@NET: ARP entry is created, 10.10.254.65, 8840.33e8.c0d7, trust-vr", Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "@timestamp" => 2026-06-24T08:16:47.271Z, Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "host" => { Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "name" => "localhost.localdomain" Jun 24 16:23:29 elk-lo-node03 logstash[193222]: }, Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "input" => { Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "type" => "log" Jun 24 16:23:29 elk-lo-node03 logstash[193222]: }, Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "ecs" => { Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "version" => "8.0.0" Jun 24 16:23:29 elk-lo-node03 logstash[193222]: }, Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "log" => { Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "offset" => 21896295, Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "file" => { Jun 24 16:23:29 elk-lo-node03 logstash[193222]: "path" => "/home/network-devices/10.10.254.10/2026-06-24.log" 检查看到日志被logstash成功接收到9、kibana上检查索引管理上是否有自动创建索引(若没有，则说明需要修改logstash上的对应conf文件)
10、新建对应网络设备日志的数据视图
11、进入discover检查日志
以上，完成~！