Kubernetes部署实践:从入门到生产级配置
Kubernetes部署实践:从入门到生产级配置
一、Kubernetes基础概念
1.1 Kubernetes架构概览
Kubernetes(简称K8s)是一个用于自动化部署、扩展和管理容器化应用的开源平台。其核心架构包括:
控制平面(Control Plane):管理集群的决策中心
- API Server:集群的核心管理入口
- ETCD:分布式键值存储,保存集群状态
- Scheduler:调度Pod到节点
- Controller Manager:维护期望状态
节点(Node):运行应用的工作节点
- Kubelet:管理节点上的Pod
- Kube-proxy:网络代理和负载均衡
- Container Runtime:容器运行时(Docker/containerd)
1.2 核心资源对象
| 对象 | 描述 | 作用 |
|---|---|---|
| Pod | 最小部署单元,可包含多个容器 | 运行应用实例 |
| Service | 定义Pod的访问方式 | 服务发现和负载均衡 |
| Deployment | 声明式管理Pod副本 | 部署、升级、回滚 |
| StatefulSet | 有状态应用的管理 | 数据库、缓存等有状态服务 |
| ConfigMap | 配置文件管理 | 管理应用配置 |
| Secret | 敏感信息管理 | 存储密码、密钥等 |
二、Deployment配置详解
2.1 基础Deployment配置
apiVersion: apps/v1 kind: Deployment metadata: name: spring-boot-app labels: app: spring-boot-app version: v1.0.0 spec: replicas: 3 selector: matchLabels: app: spring-boot-app strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: labels: app: spring-boot-app spec: containers: - name: app image: registry.example.com/myapp:1.0.0 ports: - containerPort: 8080 resources: requests: cpu: "100m" memory: "256Mi" limits: cpu: "500m" memory: "512Mi" livenessProbe: httpGet: path: /actuator/health/liveness port: 8080 initialDelaySeconds: 30 periodSeconds: 10 timeoutSeconds: 5 readinessProbe: httpGet: path: /actuator/health/readiness port: 8080 initialDelaySeconds: 20 periodSeconds: 5 timeoutSeconds: 3 env: - name: SPRING_PROFILES_ACTIVE value: "prod" - name: DB_HOST valueFrom: configMapKeyRef: name: app-config key: db.host2.2 滚动更新策略
strategy: type: RollingUpdate rollingUpdate: maxSurge: 25% # 最大额外Pod数 maxUnavailable: 0 # 最大不可用Pod数 minReadySeconds: 30 # Pod就绪后等待时间 revisionHistoryLimit: 10 # 保留历史版本数2.3 回滚操作
# 查看历史版本 kubectl rollout history deployment/spring-boot-app # 回滚到上一个版本 kubectl rollout undo deployment/spring-boot-app # 回滚到指定版本 kubectl rollout undo deployment/spring-boot-app --to-revision=3三、Service配置与网络
3.1 Service类型对比
| 类型 | 描述 | 适用场景 |
|---|---|---|
| ClusterIP | 集群内部访问 | 内部服务通信 |
| NodePort | 节点端口暴露 | 开发测试环境 |
| LoadBalancer | 云负载均衡器 | 生产环境对外服务 |
| ExternalName | 外部服务别名 | 访问外部服务 |
3.2 Service配置示例
apiVersion: v1 kind: Service metadata: name: spring-boot-service annotations: service.beta.kubernetes.io/aws-load-balancer-type: "nlb" spec: type: LoadBalancer selector: app: spring-boot-app ports: - protocol: TCP port: 80 targetPort: 8080 nodePort: 30080 sessionAffinity: ClientIP sessionAffinityConfig: clientIP: timeoutSeconds: 3003.3 Ingress配置
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: app-ingress annotations: nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/use-regex: "true" nginx.ingress.kubernetes.io/rewrite-target: /$1 spec: tls: - hosts: - api.example.com secretName: example-tls rules: - host: api.example.com http: paths: - path: /api/(.*) pathType: Prefix backend: service: name: spring-boot-service port: number: 80四、配置管理
4.1 ConfigMap配置
apiVersion: v1 kind: ConfigMap metadata: name: app-config data: db.host: "mysql-service" db.port: "3306" app.env: "production" log.level: "INFO" --- apiVersion: v1 kind: ConfigMap metadata: name: app-config-files data: application.yml: | server: port: 8080 spring: datasource: url: jdbc:mysql://${DB_HOST}:${DB_PORT}/example_db4.2 Secret配置
apiVersion: v1 kind: Secret metadata: name: app-secret type: Opaque data: db.username: base64-encoded-username db.password: base64-encoded-password api.key: base64-encoded-api-key4.3 在Pod中使用配置
spec: containers: - name: app image: myapp:1.0.0 env: # 从ConfigMap获取 - name: DB_HOST valueFrom: configMapKeyRef: name: app-config key: db.host # 从Secret获取 - name: DB_PASSWORD valueFrom: secretKeyRef: name: app-secret key: db.password volumeMounts: - name: config-volume mountPath: /app/config volumes: - name: config-volume configMap: name: app-config-files五、资源管理与调度
5.1 资源请求与限制
resources: requests: cpu: "200m" # 0.2 CPU核心 memory: "512Mi" # 512MB内存 limits: cpu: "1" # 1 CPU核心 memory: "1Gi" # 1GB内存5.2 节点亲和性
affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: cloud.google.com/gke-nodepool operator: In values: - prod-pool preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: topology.kubernetes.io/zone operator: In values: - us-central1-a5.3 Pod亲和性与反亲和性
affinity: podAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - database topologyKey: topology.kubernetes.io/zone podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 podAffinityTerm: labelSelector: matchExpressions: - key: app operator: In values: - spring-boot-app topologyKey: kubernetes.io/hostname六、水平自动伸缩(HPA)
6.1 基于CPU的自动伸缩
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: spring-boot-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: spring-boot-app minReplicas: 3 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 706.2 基于自定义指标的自动伸缩
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: spring-boot-hpa-custom spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: spring-boot-app minReplicas: 3 maxReplicas: 15 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Pods pods: metric: name: requests-per-second target: type: AverageValue averageValue: 100m6.3 基于外部指标的自动伸缩
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: spring-boot-hpa-external spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: spring-boot-app minReplicas: 3 maxReplicas: 20 metrics: - type: External external: metric: name: queue_length selector: matchLabels: queue: orders target: type: AverageValue averageValue: 50七、持久化存储
7.1 PersistentVolumeClaim
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: mysql-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi storageClassName: standard-rwo7.2 StatefulSet配置
apiVersion: apps/v1 kind: StatefulSet metadata: name: mysql spec: serviceName: mysql replicas: 3 selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - name: mysql image: mysql:8.0 ports: - containerPort: 3306 volumeMounts: - name: data mountPath: /var/lib/mysql volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 10Gi八、安全配置
8.1 Pod安全策略
apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: restricted spec: privileged: false allowPrivilegeEscalation: false requiredDropCapabilities: - ALL volumes: - 'configMap' - 'emptyDir' - 'projected' - 'secret' - 'downwardAPI' - 'persistentVolumeClaim' runAsUser: rule: 'MustRunAsNonRoot' seLinux: rule: 'RunAsAny' supplementalGroups: rule: 'MustRunAs' ranges: - min: 1 max: 65535 fsGroup: rule: 'MustRunAs' ranges: - min: 1 max: 655358.2 ServiceAccount配置
apiVersion: v1 kind: ServiceAccount metadata: name: app-service-account annotations: eks.amazonaws.com/role-arn: arn:aws:iam::1234567890:role/k8s-app-role --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: app-role rules: - apiGroups: [""] resources: ["pods", "services"] verbs: ["get", "list", "watch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: app-role-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: app-role subjects: - kind: ServiceAccount name: app-service-account九、日志与监控
9.1 日志采集配置
apiVersion: v1 kind: ConfigMap metadata: name: fluentd-config data: fluent.conf: | <source> @type tail path /var/log/containers/*.log pos_file /var/log/fluentd-containers.log.pos tag kubernetes.* read_from_head true <parse> @type json time_key time time_format %Y-%m-%dT%H:%M:%S.%NZ </parse> </source> <filter kubernetes.**> @type kubernetes_metadata </filter> <match **> @type elasticsearch host elasticsearch-service port 9200 logstash_format true logstash_prefix k8s-logs </match>9.2 Prometheus监控配置
apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: spring-boot-monitor labels: release: prometheus spec: selector: matchLabels: app: spring-boot-app endpoints: - port: http path: /actuator/prometheus interval: 30s scrapeTimeout: 10s十、CI/CD集成
10.1 Jenkins流水线
pipeline { agent any environment { DOCKER_REGISTRY = 'registry.example.com' IMAGE_NAME = 'myapp' KUBECONFIG = credentials('k8s-config') } stages { stage('Build') { steps { sh 'mvn clean package -DskipTests' } } stage('Build Docker Image') { steps { script { docker.build("${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}") } } } stage('Push Docker Image') { steps { script { docker.withRegistry("https://${DOCKER_REGISTRY}") { docker.image("${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}").push() } } } } stage('Deploy to Kubernetes') { steps { sh 'kubectl --kubeconfig=$KUBECONFIG set image deployment/spring-boot-app app=${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}' sh 'kubectl --kubeconfig=$KUBECONFIG rollout status deployment/spring-boot-app' } } } }10.2 Argo Rollouts配置
apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: spring-boot-rollout spec: replicas: 5 selector: matchLabels: app: spring-boot-app template: metadata: labels: app: spring-boot-app spec: containers: - name: app image: registry.example.com/myapp:1.0.0 ports: - containerPort: 8080 strategy: canary: steps: - setWeight: 20 - pause: {duration: 10m} - setWeight: 50 - pause: {duration: 10m} - setWeight: 100 analysis: templates: - name: success-rate args: - name: service-name value: spring-boot-service十一、生产环境最佳实践
11.1 健康检查配置
livenessProbe: httpGet: path: /actuator/health/liveness port: 8080 initialDelaySeconds: 60 periodSeconds: 15 timeoutSeconds: 5 failureThreshold: 3 readinessProbe: httpGet: path: /actuator/health/readiness port: 8080 initialDelaySeconds: 30 periodSeconds: 10 timeoutSeconds: 3 failureThreshold: 3 startupProbe: httpGet: path: /actuator/health/liveness port: 8080 failureThreshold: 30 periodSeconds: 1011.2 优雅关闭配置
spec: terminationGracePeriodSeconds: 60 containers: - name: app image: myapp:1.0.0 lifecycle: preStop: exec: command: ["/bin/sh", "-c", "sleep 10"]11.3 Pod拓扑分布约束
topologySpreadConstraints: - maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: DoNotSchedule labelSelector: matchLabels: app: spring-boot-app - maxSkew: 1 topologyKey: kubernetes.io/hostname whenUnsatisfiable: DoNotSchedule labelSelector: matchLabels: app: spring-boot-app十二、故障排查指南
12.1 常用诊断命令
# 查看Pod状态 kubectl get pods -o wide # 查看Pod日志 kubectl logs <pod-name> kubectl logs <pod-name> -f # 查看Pod详细信息 kubectl describe pod <pod-name> # 查看事件 kubectl get events # 进入Pod kubectl exec -it <pod-name> -- /bin/bash # 检查资源使用 kubectl top pods kubectl top nodes12.2 常见问题排查
| 问题 | 排查方向 | 解决方案 |
|---|---|---|
| Pod处于Pending状态 | 资源不足、调度约束 | 检查节点资源、调整亲和性 |
| Pod处于CrashLoopBackOff | 应用启动失败 | 查看日志、检查配置 |
| Pod处于ImagePullBackOff | 镜像拉取失败 | 检查镜像地址、权限 |
| Service无法访问 | 网络策略、标签选择器 | 检查网络策略、确认标签匹配 |
十三、总结
Kubernetes部署是现代云原生应用的核心基础设施,掌握其配置和最佳实践对于构建稳定、可扩展的生产环境至关重要。本文涵盖了从基础概念到高级配置的完整内容,包括:
- Deployment配置:滚动更新、健康检查、资源管理
- 网络配置:Service、Ingress、网络策略
- 配置管理:ConfigMap、Secret、环境变量
- 自动伸缩:HPA基于CPU、自定义和外部指标
- 持久化存储:PVC、StatefulSet
- 安全配置:Pod安全策略、RBAC
- 监控日志:Prometheus、Fluentd集成
- CI/CD集成:Jenkins、Argo Rollouts
通过合理配置和实践,可以构建高可用、高性能的Kubernetes生产环境。
