当前位置：首页 > news >正文

极物科技正点原子 RK3588 部署 Qwen2-VL Qwen2-VL-2B-Instruct，提供模型和npu驱动0.9.8的kernel

news 2026/5/22 0:03:39

1. 前言

Qwen2-VL是一个基于视觉-语言预训练的多模态模型，支持图像和文本的联合输入，输出是文本形式。将在正点原子RK3588板卡上部署Qwen2-VL-2B-Instruct模型，对输入图像进行描述。

2. 版本

2.1 kernel版本

本次部署的kernel版本

root@ATK-DLRK3588-Ubuntu:~# uname -a Linux ATK-DLRK3588-Ubuntu 5.10.160 #2 SMP Mon Apr 14 21:43:53 CST 2025 aarch64 aarch64 aarch64 GNU/Linux

2.2 ubuntu 版本

root@ATK-DLRK3588-Ubuntu:~# cat /etc/issue Ubuntu 20.04.6 LTS \n \l

2.3 npu

2.3.1 npu 驱动版本

root@ATK-DLRK3588-Ubuntu:~# cat /sys/kernel/debug/rknpu/version RKNPUdriver:v0.9.8

2.3.2 编译0.9.8驱动

如何编译最新的驱动，以及解决编译过程中遇到的两个问题，可以参考这篇博文rk3588 npu升级修订

编译正点原子的kernel，参考正点原子的官方文档
替换npu驱动，并从新编译

2.3.3 升级kernel

将更新最新rknpu驱动的kernel升级。RK可以单独升级kernel。截图如下

2.3.4 rknn-llm

rknn-llm 提供大模型转换和和板端推理功能。本次PC端和板端使用的版本一致，版本如下

2.3.5 rknn-toolkit2

rknn-toolkit2提供通用的模型转换和板端推理功能。本次使用的版本如下

2.4 说明

最好保证版本的一致，除了kernel需要编译最新版本的npu驱动 0.9.8之外，其他的全部使用正点原子原厂提供的即可

3. 模型转换

3.1 已经转换好的模型

通过网盘分享的文件：Qwen2-VL-2B链接:https://pan.baidu.com/s/1CBEoRM2bW5zoTsXWNRk1dw?pwd=ij5d提取码:ij5d

3.2 模型转换

模型转换有非常多得参考文档，主要参考一下如下：
RKLLM 部署文档
Qwen2-VL 模型转换文档

本人也是参考以上两篇文档完成RKLLM部署和模型转换的。

4. Qwen2-VL 板端推理

4.1 模型拷贝

将完成转换的模型拷贝到设备

4.2 rknn-llm 推理代码修订

下载 rknn-llm到设备，修订build-linux.sh文件

rm -rf build mkdir build && cd build cmake .. -DCMAKE_CXX_COMPILER=/usr/bin/aarch64-linux-gnu-g++ \ -DCMAKE_C_COMPILER=/usr/bin/aarch64-linux-gnu-gcc \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_SYSTEM_NAME=Linux \ -DCMAKE_SYSTEM_PROCESSOR=aarch64 \ make -j8 make install

4.3 rknn-llm 推理代码编译

修改编译脚本之后，进入到examples/Qwen2-VL-2B_Demo/deploy 路径之后，执行

root@ATK-DLRK3588-Ubuntu:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy# ./build-linux.sh -- The C compiler identification is GNU 9.4.0 -- The CXX compiler identification is GNU 9.4.0 -- Check for working Ccompiler:/usr/bin/aarch64-linux-gnu-gcc -- Check for working Ccompiler:/usr/bin/aarch64-linux-gnu-gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXXcompiler:/usr/bin/aarch64-linux-gnu-g++ -- Check for working CXXcompiler:/usr/bin/aarch64-linux-gnu-g++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- FoundOpenCV:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/3rdparty/opencv-linux-aarch64(found version"3.4.5")-- Configuring done -- Generating done -- Build files have been writtento:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/build Scanning dependencies of target demo Scanning dependencies of target llm Scanning dependencies of target imgenc [ 12%] Building CXX object CMakeFiles/llm.dir/src/llm.cpp.occ1plus:warning:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/src/image_enc.h:not a directory [ 25%] Building CXX object CMakeFiles/demo.dir/src/image_enc.cc.o [ 37%] Building CXX object CMakeFiles/imgenc.dir/src/image_enc.cc.o [ 62%] Building CXX object CMakeFiles/imgenc.dir/src/img_encoder.cpp.o [ 62%] Building CXX object CMakeFiles/demo.dir/src/main.cpp.occ1plus:warning:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/src/image_enc.h:not a directorycc1plus:warning:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/src/image_enc.h:not a directorycc1plus:warning:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/src/image_enc.h:not a directorycc1plus:warning:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/src/image_enc.h:not a directory [ 75%] Linking CXX executable llm [ 75%] Built target llm [ 87%] Linking CXX executable imgenc [100%] Linking CXX executable demo [100%] Built target demo [100%] Built target imgenc [ 37%] Built target demo [ 62%] Built target llm [100%] Built target imgenc Install the project... -- Installconfiguration:"Release"--Installing:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64/./imgenc -- Set runtime path of"/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64/./imgenc"to""--Installing:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64/./llm -- Set runtime path of"/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64/./llm"to""--Installing:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64/./demo -- Set runtime path of"/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64/./demo"to""--Up-to-date:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64/lib/librknnrt.so --Up-to-date:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64/lib/librkllmrt.so --Up-to-date:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64/./demo.jpg root@ATK-DLRK3588-Ubuntu:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy#

4.4 验证

4.4.1 注意事项

注意：需要导出 librkllmrt.so 库的路径，否则会提示错误

./demo:error while loading sharedlibraries:librkllmrt.so:cannot open shared objectfile:No such file or directory

导出库

export LD_LIBRARY_PATH=./lib

4.4.2 验证

执行代码

root@ATK-DLRK3588-Ubuntu:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64# export LD_LIBRARY_PATH=./lib root@ATK-DLRK3588-Ubuntu:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64# ./demo demo.jpg /work/qianwen/qwen2_vl_2b_vision_rk3588.rknn /work/qianwen/Qwen2-VL-2B-Instruct.rkllm 128 512

验证结果

root@ATK-DLRK3588-Ubuntu:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64# export LD_LIBRARY_PATH=./lib root@ATK-DLRK3588-Ubuntu:/work/rknn-llm/examples/Qwen2-VL-2B_Demo/deploy/install/demo_Linux_aarch64# ./demo demo.jpg /work/qianwen/qwen2_vl_2b_vision_rk3588.rknn /work/qianwen/Qwen2-VL-2B-Instruct.rkllm 128 512 Irkllm:rkllm-runtimeversion:1.2.0,rknpu driverversion:0.9.8,platform:RK3588 Irkllm:loading rkllm model from /work/qianwen/Qwen2-VL-2B-Instruct.rkllm Irkllm:rkllm-toolkitversion:1.2.0,max_context_limit:4096,npu_core_num:3,target_platform:RK3588 Irkllm:Enabledcpus:[4,5,6,7] Irkllm:Enabled cpusnum:4 Irkllm:system_prompt:<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n Irkllm:prompt_prefix:<|im_start|>user\n Irkllm:prompt_postfix:<|im_end|>\n<|im_start|>assistant\n rkllm init successmain:LLM Model loaded in 8587.09 ms model inputnum:1,outputnum:1 inputtensors:index=0,name=onnx::Expand_0,n_dims=4,dims=[1,392,392,3],n_elems=460992,size=921984,fmt=NHWC,type=FP16,qnt_type=AFFINE,zp=0,scale=1.000000 outputtensors:index=0,name=6460,n_dims=2,dims=[196,1536,0,0],n_elems=301056,size=602112,fmt=UNDEFINED,type=FP16,qnt_type=AFFINE,zp=0,scale=1.000000 model input height=392,width=392,channel=3main:ImgEnc Model loaded in 6957.11 ms Irkllm:reset chattemplate:Irkllm:system_prompt:<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n Irkllm:prompt_prefix:<|im_start|>user\n Irkllm:prompt_postfix:<|im_end|>\n<|im_start|>assistant\n **********************可输入以下问题对应序号获取回答/或自定义输入******************** [0] <image>What is in the image? [1] <image>这张图片中有什么？ *************************************************************************user:1 <image>这张图片中有什么？robot:这张图片展示了一位穿着宇航服的宇航员，他坐在月球表面的一块岩石上。背景是地球和星空，显得非常科幻和未来感。宇航员手中拿着一个绿色的瓶子，似乎在庆祝或享受某种活动。整体画面充满了神秘和探索的感觉。user: