当前位置：首页 > news >正文

Kubernetes多集群管理策略：统一管理多个K8s集群

news 2026/5/24 23:21:58

Kubernetes多集群管理策略：统一管理多个K8s集群

一、多集群管理概述

Kubernetes多集群管理是指在企业环境中管理多个独立的Kubernetes集群，实现统一的部署、监控和运维。

1.1 多集群场景

场景	说明	示例
地域隔离	不同区域部署独立集群	北京、上海、广州各一个集群
环境隔离	开发、测试、生产分离	dev、staging、prod集群
租户隔离	多租户共享基础设施	每个租户独立集群
混合云	公有云+私有云混合部署	AWS+本地IDC集群

1.2 多集群架构

┌─────────────────────────┐ │ 统一管理平面 │ │ (Cluster Management) │ └───────────┬─────────────┘ │ ┌───────────────────────┼───────────────────────┐ │ │ │ ▼ ▼ ▼ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ 集群A │ │ 集群B │ │ 集群C │ │ (Production) │ │ (Staging) │ │ (Development)│ └───────────────┘ └───────────────┘ └───────────────┘

二、多集群管理工具

2.1 Rancher配置

apiVersion: rancher.cattle.io/v3 kind: Cluster metadata: name: production spec: rkeConfig: machinePools: - name: worker count: 3 machineConfigRef: apiVersion: rke-machine-config.cattle.io/v1 kind: DigitalOceanConfig name: do-worker

2.2 Fleet配置

apiVersion: fleet.cattle.io/v1alpha1 kind: GitRepo metadata: name: my-apps namespace: fleet-default spec: repo: https://github.com/example/fleet-repo branch: main targets: - name: production clusterSelector: matchLabels: env: prod - name: staging clusterSelector: matchLabels: env: staging

2.3 Cluster API配置

apiVersion: cluster.x-k8s.io/v1beta1 kind: Cluster metadata: name: my-cluster spec: topology: class: quick-start version: v1.27.3 workers: machineDeployments: - class: default-worker replicas: 3

三、多集群网络策略

3.1 集群间通信

apiVersion: v1 kind: Service metadata: name: cross-cluster-service spec: type: ExternalName externalName: service.other-cluster.svc.cluster.local

3.2 统一入口管理

apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: global-ingress annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - host: app.example.com http: paths: - path: /api pathType: Prefix backend: service: name: api-service port: number: 80 - host: app-staging.example.com http: paths: - path: /api pathType: Prefix backend: service: name: api-service-staging port: number: 80

四、多集群资源同步

4.1 配置同步

apiVersion: configsync.gke.io/v1beta1 kind: RootSync metadata: name: cluster-config spec: sourceFormat: unstructured git: repo: https://github.com/example/cluster-config branch: main policyDir: configs/ auth: token secretRef: name: git-creds

4.2 资源分发策略

apiVersion: distribution.k8s.io/v1alpha1 kind: ClusterResourceSet metadata: name: common-config spec: clusterSelector: matchLabels: environment: shared resources: - name: common-configmap kind: ConfigMap - name: common-secret kind: Secret

五、多集群监控

5.1 Prometheus联邦

apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: remote-cluster namespace: monitoring spec: endpoints: - honorLabels: true interval: 30s path: /federate params: match[]: - '{__name__=~"job:.*"}' port: http selector: matchLabels: app: prometheus

5.2 统一告警规则

apiVersion: monitoring.coreos.com/v1 kind: PrometheusRule metadata: name: cluster-alerts namespace: monitoring spec: groups: - name: cluster.rules rules: - alert: HighCPUUsage expr: avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) < 0.2 for: 10m labels: severity: critical annotations: summary: "High CPU usage detected"

六、多集群日志管理

6.1 Loki分布式日志

apiVersion: loki.grafana.com/v1 kind: LokiStack metadata: name: loki namespace: monitoring spec: size: 1x.extra-small storage: schemas: - version: v13 effectiveDate: "2024-01-01" secret: name: loki-storage

6.2 日志收集配置

apiVersion: v1 kind: ConfigMap metadata: name: fluentd-config namespace: logging data: fluent.conf: | <source> @type tail path /var/log/containers/*.log pos_file /var/log/fluentd-containers.log.pos tag kubernetes.* read_from_head true </source> <match kubernetes.**> @type loki url https://loki.example.com auth_user admin auth_password secret </match>

七、多集群安全策略

7.1 统一RBAC管理

apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: cluster-admin rules: - apiGroups: ["*"] resources: ["*"] verbs: ["*"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin-user subjects: - kind: User name: admin@example.com apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: cluster-admin apiGroup: rbac.authorization.k8s.io

7.2 证书管理

apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-prod spec: acme: server: https://acme-v02.api.letsencrypt.org/directory email: admin@example.com privateKeySecretRef: name: letsencrypt-prod solvers: - http01: ingress: class: nginx

八、多集群成本管理

8.1 资源使用监控

apiVersion: v1 kind: ConfigMap metadata: name: cost-exporter-config namespace: monitoring data: config.yaml: | exporters: - name: cloud-cost type: prometheus params: endpoint: http://prometheus:9090 query: | sum(node_cpu_hours_total) * 0.05 + sum(node_memory_hours_total) * 0.02

8.2 资源配额管理

apiVersion: v1 kind: ResourceQuota metadata: name: cluster-quota spec: hard: pods: "1000" requests.cpu: "100" requests.memory: 200Gi limits.cpu: "200" limits.memory: 400Gi

九、多集群故障恢复

9.1 灾难恢复策略

apiVersion: velero.io/v1 kind: Schedule metadata: name: daily-backup spec: schedule: "0 2 * * *" template: includedNamespaces: - default - kube-system storageLocation: name: s3-backup volumeSnapshotLocations: - name: aws-ebs

9.2 跨集群迁移

apiVersion: apps/v1 kind: Deployment metadata: name: migration-app spec: replicas: 0 selector: matchLabels: app: migration-app template: metadata: labels: app: migration-app spec: containers: - name: app image: migration-tool:latest env: - name: SOURCE_CLUSTER value: "https://source-cluster:6443" - name: TARGET_CLUSTER value: "https://target-cluster:6443"