当前位置：首页 > news >正文

【架构实战】Kubernetes监控体系：Prometheus + Grafana

news 2026/4/18 15:22:14

一、Kubernetes监控概述

Kubernetes监控是保障集群稳定性的关键：

监控层次：

基础设施层（Node、Pod）
应用层（业务指标）
集群层（K8s组件）

二、Prometheus架构

1. 架构图

┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Exporter │ │ Exporter │ │ Exporter │ │ (Node) │ │ (Kube) │ │ (App) │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ └───────────────────┼───────────────────┘ │ ┌──────┴──────┐ │ Prometheus │ │ Server │ └──────┬──────┘ │ ┌─────────────────┼─────────────────┐ │ │ │ ┌──────┴──────┐ ┌──────┴──────┐ ┌──────┴──────┐ │ Alertmanager│ │ Grafana │ │ Storage │ └─────────────┘ └─────────────┘ └─────────────┘

2. 核心概念

指标类型：

Counter：单调递增
Gauge：可增可减
Histogram：直方图
Summary：分位数

三、部署Prometheus

1. Operator部署

# 安装Prometheus Operatorkubectl create namespace monitoring helm repoaddprometheus-community https://prometheus-community.github.io/helm-charts helminstallprometheus prometheus-community/kube-prometheus-stack

2. ServiceMonitor

apiVersion:monitoring.coreos.com/v1kind:ServiceMonitormetadata:name:myapp-monitornamespace:monitoringspec:selector:matchLabels:app:myappendpoints:-port:httppath:/metricsinterval:15s

3. PrometheusRule

apiVersion:monitoring.coreos.com/v1kind:PrometheusRulemetadata:name:myapp-alertsspec:groups:-name:myapprules:-alert:HighMemoryUsageexpr:container_memory_usage_bytes>1e9for:5mlabels:severity:warningannotations:summary:"内存使用率过高"

四、Grafana配置

1. 数据源配置

apiVersion:v1kind:ConfigMapmetadata:name:grafana-datasourcesnamespace:monitoringdata:datasources.yaml:|apiVersion: 1 datasources: - name: Prometheus type: prometheus access: proxy url: http://prometheus-server.monitoring.svc:9090 isDefault: true

2. Dashboard

{"dashboard":{"title":"Kubernetes Cluster","panels":[{"title":"CPU Usage","type":"graph","targets":[{"expr":"rate(container_cpu_usage_seconds_total[5m])"}]}]}}

3. 常用查询

# Node资源使用 node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes * 100 # Pod资源使用 sum(rate(container_cpu_usage_seconds_total{pod!=""}[5m])) by (pod) # K8s组件状态 up{job="kube-apiserver"} # 存储使用 kubelet_volume_stats_used_bytes / kubelet_volume_stats_capacity_bytes

五、告警配置

1. AlertManager配置

apiVersion:monitoring.coreos.com/v1alpha1kind:AlertmanagerConfigmetadata:name:alert-configspec:route:groupBy:['namespace']groupWait:30sgroupInterval:5mrepeatInterval:4hreceiver:'default'receivers:-name:'default'webhookConfigs:-url:'http://notification-service:8080/webhook'

2. 告警规则

groups:-name:k8srules:-alert:PodNotReadyexpr:kube_pod_status_ready{condition="true"}== 0for:5mlabels:severity:criticalannotations:summary:"Pod {{ $labels.namespace }}/{{ $labels.pod }} not ready"-alert:HighMemoryexpr:sum(container_memory_working_set_bytes) by (pod) / 1024/1024>1024for:5mlabels:severity:warning