117. 如何在Rancher监控中测试 AlertManager

This guide demonstrates how to test Alertmanager and PrometheusRule configuration, to validate that alerts are sent successfully by Alertmanager.
本指南演示如何测试 AlertManager 和 PrometheusRule 配置，以验证 AlertManager 是否成功发送了警报。

With this objective in mind, and for this test to be self-contained, a webhook receiver is configured in Alertmanager. A webhook-receiver pod is deployed to receive these webhook alert requests and print them to stdout, such that they are visible in the Pod logs for verification. All of these resources are created in the cattle-monitoring-system.
基于这一目标，并使测试自成一体，在 Alertmanager 中配置了一个 webhook 接收器。部署一个 webhook-receiver pod 接收这些 webhook 警报请求并打印至 stdout，使其在 Pod 日志中可见以便验证。所有这些资源均在牛群监控系统中创建。

1. Navigate to a Rancher-managed cluster with rancher-monitoring installed.
   导航到安装了牧场监控的牧场主管理集群。
2. Apply the following YAMLs:
   应用以下 YAML：
   1. ConfigMap: 配置地图：

      <span style="color:#000000"><span style="background-color:#ffffff"><span style="background-color:#efefef"><code>apiVersion: v1 kind: ConfigMap metadata: name: webhook-receiver-configmap-script namespace: cattle-monitoring-system data: <a>Pod: 播客：
      <span style="color:#000000"><span style="background-color:#ffffff"><span style="background-color:#efefef"><code>apiVersion: v1 kind: Pod metadata: name: webhook-receiver namespace: cattle-monitoring-system labels: app: webhook-receiver spec: containers: - name: receiver-container image: rancherlabs/swiss-army-knife:latest command: ["/bin/bash", "/script/<a>Service: 服务：
      <span style="color:#000000"><span style="background-color:#ffffff"><span style="background-color:#efefef"><code>apiVersion: v1 kind: Service metadata: name: webhook-receiver-service namespace: cattle-monitoring-system spec: selector: app: webhook-receiver ports: - protocol: TCP port: 80 targetPort: 8080 type: ClusterIP </code></span></span></span>

3. Ensure that the pod is up and tail the log, you should see a couple of lines stating that the netcat listener is ready and waiting for a connection. The Alertmanager alert configured below will be visible in these logs.
确保 Pod 已上线并跟踪日志，你应该会看到几行显示 netcat 监听器已准备好并等待连接。下面配置的 Alertmanager 警报会在这些日志中显示。
4. Apply the following AlertmanagerConfig to configure Alertmanager to send any alerts with the label "severity=critical" to the webhook-receiver pod (the Alertmanager configuration documentation can be found here). Note that the URL used is that of the service created above:
   应用以下 AlertmanagerConfig 来配置 AlertManager，将任何标签为“severity=critical”的警报发送到 webhook-receiver pod（AlertManager 配置文件文档可在此处找到）。请注意，所使用的 URL 是上述创建服务的地址：

   <span style="color:#000000"><span style="background-color:#ffffff"><span style="background-color:#efefef"><code>apiVersion: <a>Create a PrometheusRule with an alert expression. This example uses vector(1) as the expression, such that its value will be always "1" and the alert will be trigged continuously:
   创建一个带有警报表达式的 Prometheus 规则。本示例使用向量（1）作为表达式，使其值始终为“1”，警报将持续触发：
   <span style="color:#000000"><span style="background-color:#ffffff"><span style="background-color:#efefef"><code>apiVersion: <a>
   

5. Wait for the alert to appear in the Alertmanager Alerts UI.
等待警报出现在警报管理器的警报界面中。
6. Check the log of the webhook-receiver pod and observe that the test-rule alert is received, similar to the following:
   检查 webhook-receiver pod 的日志，观察测试规则警报是否已接收，类似于以下内容：

   <span style="color:#000000"><span style="background-color:#ffffff"><span style="background-color:#efefef"><code>Starting nc listener with 1 second timeout... Waiting for connection... --- RECEIVED FULL REQUEST --- POST / HTTP/1.1 Host: webhook-receiver-service User-Agent: Alertmanager/0.28.1 Content-Length: 1214 Content-Type: application/json {"receiver":"cattle-monitoring-system/webhook-receiver-am-config/webhook-receiver-pod","status":"firing","alerts":[{"status":"firing","labels":{"alertname":"test-alert","namespace":"cattle-monitoring-system","prometheus":"cattle-monitoring-system/rancher-monitoring-prometheus","severity":"critical"},"annotations":{},"startsAt":"2025-10-14T09:04:11.437Z","endsAt":"0001-01-01T00:00:00Z","generatorURL":"<a>Following this method, it is possible to test Alertmanager and PrometheusRule configurations without needing a third party app or configuring an external receiver. This is useful to see if the alerts arrive as expected or if they are not being sent. If you are struggling to correctly apply an AlertmanagerConfig, you can check the rancher-monitoring-operator pod logs, in order to check that the syntax is correct and was accepted; the Alertmanager pod logs; as well as the value of the PrometheusRule expression, using the Prometheus Query UI, to confirm whether the alert should currently trigger.
   按照这种方法，可以在无需第三方应用或配置外部接收器的情况下测试 Alertmanager 和 PrometheusRule 的配置。这有助于判断警报是否按预期到达，或者是否未发送。如果你在正确应用 AlertmanagerConfig 时遇到困难，可以检查 rancher-monitoring-operator 的 pod 日志，以确认语法正确且已被接受;Alertmanager Pod 日志;以及使用 Prometheus 查询界面的 PrometheusRule 表达式值，以确认警报当前是否应触发。
   

A Kubernetes cluster managed by Rancher v2.6+ with rancher-monitoring installed
由 Rancher v2.6+ 管理的 Kubernetes 集群，安装了 rancher-monitoring（农场监控功能）