当前位置：首页 > news >正文

02-大模型部署之Kubernetes+vLLM安装大模型和容器调度

news 2026/3/26 17:01:21

02-大模型部署之Kubernetes+vLLM安装大模型和容器调度

1. Kubernetes基础与vLLM集成概述

1.1 为什么使用Kubernetes部署vLLM

Kubernetes提供了企业级的容器编排能力，特别适合vLLM部署的以下场景：

弹性伸缩：根据负载自动调整vLLM实例数量
高可用性：自动故障恢复和负载均衡
资源管理：精细化的GPU资源分配和调度
多租户隔离：不同模型或用户之间的资源隔离
版本管理：无缝的模型版本升级和回滚

1.2 Kubernetes与vLLM架构

┌─────────────────────────────────────────────────────────────┐ │ Kubernetes集群 │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ Master节点 │ │ Worker节点1 │ │ Worker节点2 │ │ │ │ │ │ │ │ │ │ │ │ API Server │ │ vLLM Pod 1 │ │ vLLM Pod 2 │ │ │ │ Scheduler │ │ (GPU 0,1) │ │ (GPU 2,3) │ │ │ │ Controller │ │ │ │ │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ └─────────────────────────────────────────────────────────────┘

2. 环境准备与依赖安装

2.1 Kubernetes集群要求

硬件要求

Master节点：至少2 CPU，4GB内存
Worker节点：至少4 CPU，16GB内存，1-2张NVIDIA GPU
网络：节点间万兆网络推荐

软件要求

Kubernetes 1.24+
NVIDIA GPU Operator
Container Runtime (containerd或Docker)
kubectl命令行工具

2.2 NVIDIA GPU Operator安装

# 添加NVIDIA Helm仓库helm repoaddnvidia https://nvidia.github.io/gpu-operator helm repo update# 安装GPU Operatorhelminstallgpu-operator nvidia/gpu-operator\--namespacegpu-operator\--create-namespace\--setdriver.enabled=true\--settoolkit.enabled=true\--setdevicePlugin.enabled=true

2.3 验证GPU资源

# 检查GPU节点kubectl get nodes-lgpu=true# 查看GPU资源kubectl describenode<worker-node-name>|grep-igpu# 验证NVIDIA设备插件kubectl get pods-ngpu-operator

3. vLLM容器镜像构建

3.1 基础Dockerfile

FROM nvidia/cuda:11.8-devel-ubuntu20.04 # 设置环境变量 ENV DEBIAN_FRONTEND=noninteractive ENV PYTHONUNBUFFERED=1 # 安装系统依赖 RUN apt-get update && apt-get install -y \ python3.9 \ python3.9-pip \ python3.9-dev \ git \ wget \ && rm -rf /var/lib/apt/lists/* # 创建软链接 RUN ln -s /usr/bin/python3.9 /usr/bin/python && \ ln -s /usr/bin/pip3 /usr/bin/pip # 升级pip RUN pip install --upgrade pip # 安装vLLM RUN pip install vllm==0.2.5 torch==2.0.1 # 创建应用目录 WORKDIR /app # 复制启动脚本 COPY start_vllm.sh /app/ RUN chmod +x /app/start_vllm.sh # 暴露端口 EXPOSE 8000 # 启动命令 CMD ["/app/start_vllm.sh"]

3.2 启动脚本 (start_vllm.sh)

#!/bin/bash# 设置模型路径MODEL_PATH=${MODEL_PATH:-"meta-llama/Llama-2-7b-chat-hf"}# 设置GPU内存使用率GPU_MEMORY_UTILIZATION=${GPU_MEMORY_UTILIZATION:-0.9}# 设置张量并行大小TENSOR_PARALLEL_SIZE=${TENSOR_PARALLEL_SIZE:-1}# 启动vLLM API服务器python-mvllm.entrypoints.api_server\--model${MODEL_PATH}\--host0.0.0.0\--port8000\--gpu-memory-utilization${GPU_MEMORY_UTILIZATION}\--tensor-parallel-size${TENSOR_PARALLEL_SIZE}\--max-num-batched-tokens8192\--max-num-seqs256

3.3 构建和推送镜像

# 构建镜像dockerbuild-tyour-registry/vllm-server:latest.# 推送到镜像仓库dockerpush your-registry/vllm-server:latest

4. Kubernetes资源配置

4.1 Namespace创建

# namespace.yamlapiVersion:v1kind:Namespacemetadata:name:vllm

4.2 ConfigMap配置

# configmap.yamlapiVersion:v1kind:ConfigMapmetadata:name:vllm-confignamespace:vllmdata:MODEL_PATH:"meta-llama/Llama-2-7b-chat-hf"GPU_MEMORY_UTILIZATION:"0.85"TENSOR_PARALLEL_SIZE:"1"MAX_NUM_BATCHED_TOKENS:"8192"MAX_NUM_SEQS:"256"

4.3 Secret配置（用于模型访问）

# secret.yamlapiVersion:v1kind:Secretmetadata:name:huggingface-secretnamespace:vllmtype:Opaquedata:# echo -n "your-huggingface-token" | base64HF_TOKEN:eW91ci1odWdnaW5nZmFjZS10b2tlbg==

4.4 PVC配置（用于模型缓存）

# pvc.yamlapiVersion:v1kind:PersistentVolumeClaimmetadata:name:vllm-model-cachenamespace:vllmspec:accessModes:-ReadWriteOnceresources:requests:storage:100GistorageClassName:fast-ssd

4.5 Deployment配置

# deployment.yamlapiVersion:apps/v1kind:Deploymentmetadata:name:vllm-deploymentnamespace:vllmspec:replicas:2selector:matchLabels:app:vllmtemplate:metadata:labels:app:vllmspec:nodeSelector:gpu:"true"containers:-name:vllm-containerimage:your-registry/vllm-server:latestimagePullPolicy:Alwaysports:-containerPort:8000env:-name:MODEL_PATHvalueFrom:configMapKeyRef:name:vllm-configkey:MODEL_PATH-name:GPU_MEMORY_UTILIZATIONvalueFrom:configMapKeyRef:name:vllm-configkey:GPU_MEMORY_UTILIZATION-name:TENSOR_PARALLEL_SIZEvalueFrom:configMapKeyRef:name:vllm-configkey:TENSOR_PARALLEL_SIZE-name:HF_TOKENvalueFrom:secretKeyRef:name:huggingface-secretkey:HF_TOKENresources:requests:nvidia.com/gpu:1memory:"16Gi"cpu:"4"limits:nvidia.com/gpu:1memory:"32Gi"cpu:"8"volumeMounts:-name:model-cachemountPath:/root/.cache/huggingfacereadinessProbe:httpGet:path:/healthport:8000initialDelaySeconds:60periodSeconds:10livenessProbe:httpGet:path:/healthport:8000initialDelaySeconds:120periodSeconds:30volumes:-name:model-cachepersistentVolumeClaim:claimName:vllm-model-cachetolerations:-key:nvidia.com/gpuoperator:Existseffect:NoSchedule

4.6 Service配置

# service.yamlapiVersion:v1kind:Servicemetadata:name:vllm-servicenamespace:vllmspec:selector:app:vllmports:-port:80targetPort:8000protocol:TCPtype:ClusterIP

4.7 Ingress配置

# ingress.yamlapiVersion:networking.k8s.io/v1kind:Ingressmetadata:name:vllm-ingressnamespace:vllmannotations:nginx.ingress.kubernetes.io/rewrite-target:/nginx.ingress.kubernetes.io/proxy-body-size:"100m"spec:rules:-host:vllm.example.comhttp:paths:-path:/pathType:Prefixbackend:service:name:vllm-serviceport:number:80

5. 部署与管理

5.1 部署vLLM服务

# 创建命名空间kubectl apply-fnamespace.yaml# 应用配置kubectl apply-fconfigmap.yaml kubectl apply-fsecret.yaml kubectl apply-fpvc.yaml# 部署应用kubectl apply-fdeployment.yaml kubectl apply-fservice.yaml kubectl apply-fingress.yaml# 检查部署状态kubectl get pods-nvllm kubectl get services-nvllm

5.2 扩缩容操作

手动扩缩容

# 扩容到3个副本kubectl scale deployment vllm-deployment--replicas=3-nvllm# 缩容到1个副本kubectl scale deployment vllm-deployment--replicas=1-nvllm

自动扩缩容配置

# hpa.yamlapiVersion:autoscaling/v2kind:HorizontalPodAutoscalermetadata:name:vllm-hpanamespace:vllmspec:scaleTargetRef:apiVersion:apps/v1kind:Deploymentname:vllm-deploymentminReplicas:1maxReplicas:10metrics:-type:Resourceresource:name:cputarget:type:UtilizationaverageUtilization:70-type:Resourceresource:name:memorytarget:type:UtilizationaverageUtilization:80

5.3 滚动更新

# 更新镜像版本kubectlsetimage deployment/vllm-deployment\vllm-container=your-registry/vllm-server:v2.0-nvllm# 查看更新状态kubectl rollout status deployment/vllm-deployment-nvllm# 回滚到上一版本kubectl rollout undo deployment/vllm-deployment-nvllm

6. 监控与日志

6.1 Prometheus监控配置

# servicemonitor.yamlapiVersion:monitoring.coreos.com/v1kind:ServiceMonitormetadata:name:vllm-monitornamespace:vllmspec:selector:matchLabels:app:vllmendpoints:-port:httppath:/metricsinterval:30s

6.2 日志收集

# fluentd-config.yamlapiVersion:v1kind:ConfigMapmetadata:name:fluentd-confignamespace:vllmdata:fluent.conf:|<source> @type tail path /var/log/containers/*vllm*.log pos_file /var/log/fluentd-vllm.log.pos tag kubernetes.* format json </source><match kubernetes.**>@type elasticsearch host elasticsearch.logging.svc.cluster.local port 9200 index_name vllm-logs </match>

6.3 健康检查端点

# 在vLLM容器中添加健康检查端点fromflaskimportFlask,jsonifyimportpsutil app=Flask(__name__)@app.route('/health')defhealth_check():"""健康检查端点"""try:# 检查GPU状态importtorch gpu_available=torch.cuda.is_available()gpu_memory_used=torch.cuda.memory_allocated()/1024**3ifgpu_availableelse0# 检查系统资源cpu_percent=psutil.cpu_percent()memory_percent=psutil.virtual_memory().percentreturnjsonify({"status":"healthy","gpu_available":gpu_available,"gpu_memory_used_gb":gpu_memory_used,"cpu_percent":cpu_percent,"memory_percent":memory_percent})exceptExceptionase:returnjsonify({"status":"unhealthy","error":str(e)}),500if__name__=='__main__':app.run(host='0.0.0.0',port=8001)

7. 故障排查

7.1 常见问题

GPU资源不足

# 检查GPU资源分配kubectl describe pod<pod-name>-nvllm# 检查GPU节点资源kubectltopnodes kubectl describenode<node-name>

模型加载失败

# 查看Pod日志kubectl logs<pod-name>-nvllm# 进入容器调试kubectlexec-it<pod-name>-nvllm -- /bin/bash

网络连接问题

# 测试服务连通性kubectl run test-pod--image=busybox--rm-it-- /bin/sh# 在测试Pod中执行wget-qO- http://vllm-service.vllm.svc.cluster.local/health

7.2 性能调优

资源限制优化

resources:requests:nvidia.com/gpu:1memory:"24Gi"# 根据模型大小调整cpu:"6"limits:nvidia.com/gpu:1memory:"48Gi"# 避免OOMcpu:"12"

调度策略优化

# 使用节点亲和性spec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:-matchExpressions:-key:gpu-typeoperator:Invalues:["A100","V100"]