基于龙蜥anolis在gpu上通过docker手动编译部署ollama

一、安装docker

yum install docker

二、下载docker镜像

docker pull registry.openanolis.cn/openanolis/anolisos:23.4

三、编写Docker file

vim Dockerfile

FROM registry.openanolis.cn/openanolis/anolisos:23.4 MAINTAINER wangchaodong<wang479961270@vip.qq.com> ARG ARCH=x86_64 ARG VERSION=v0.17.7 ENV CUDA_HOME=/usr/local/cuda ENV PATH=${CUDA_HOME}/bin:${PATH} ENV LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${LD_LIBRARY_PATH} WORKDIR /opt/ RUN yum install gzip tar gcc which xz zstd wget kmod findutils pciutils go git g++ cmake -y && wget https://developer.download.nvidia.com/compute/cuda/12.8.0/local_installers/cuda_12.8.0_570.86.10_linux.run && mkdir /opt/tmp && chmod 600 /opt/tmp && bash cuda_12.8.0_570.86.10_linux.run -tmpdir /opt/tmp --silent --toolkit --no-man-page --no-drm --override && wget https://developer.download.nvidia.cn/compute/cudnn/redist/cudnn/linux-x86_64/cudnn-linux-x86_64-9.12.0.46_cuda12-archive.tar.xz && tar xf cudnn-linux-x86_64-9.12.0.46_cuda12-archive.tar.xz && cp cudnn-linux-x86_64-9.12.0.46_cuda12-archive/include/cudnn*.h /usr/local/cuda-12.8/include/. && cp cudnn-linux-x86_64-9.12.0.46_cuda12-archive/lib/libcudnn*.so* /usr/local/cuda-12.8/lib64/. && git clone https://gitee.com/wangchaodong/ollama.git -b ${VERSION} && cd /opt/ollama && export GOPROXY=https://goproxy.cn && cmake --preset 'CUDA 12' && cmake --build --parallel $(nproc) --preset 'CUDA 12' && cmake --install build --component CUDA --strip && mkdir -p /lib/ollama/ && cp -rf dist/lib/ollama/* /lib/ollama/. && cp -rf build/lib/ollama/* /lib/ollama/. && go build -trimpath -ldflags "-X github.com/ollama/ollama/version.Version=v0.17.7 -X github.com/ollama/ollama/server.mode=release" -o /bin/ollama . && yum remove wget go git g++ -y && yum clean all && rm -rf /opt/* EXPOSE 11434 ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility ENV NVIDIA_VISIBLE_DEVICES=all ENV OLLAMA_HOST 0.0.0.0 ENTRYPOINT ["/bin/ollama"] CMD ["serve"]

四、生成ollama镜像

docker build -f Dockerfile -t cr.delcare.cn/gpu/ollama:v0.17.7 .

五、创建并运行ollama容器

docker run -itd --name ollama --privileged --gpus all cr.delcare.cn/gpu/ollama:v0.17.7

六、验证ollama服务是否正常运行

1）查看ip

docker inspect ollama|grep 172.17

2）curl 172.17.0.4:11434

3）查看日志（启动时使用gpu）