当前位置：首页 > news >正文

Qwen2.5-VL 零样本目标检测实战：提示词定位与自定义数据集测试

news 2026/7/10 7:24:50

Qwen2.5-VL 零样本目标检测实战：提示词定位与自定义数据集测试

这篇教程是我根据 Qwen2.5-VL 多模态定位能力和零样本目标检测复现过程整理出来的。重点演示如何用自然语言 prompt 让模型输出坐标，再用 supervision 转换为检测框可视化。

Qwen2.5-VL 可以根据文本描述定位图像中的目标。本文保留原始英文 prompt、模型 ID 和推理参数，避免改变模型输出。

本文会重点跑通以下流程：

加载 Qwen2.5-VL 并构造视觉问答输入
用英文 prompt 控制检测目标和空间约束
将 Qwen 输出解析为supervision.Detections
在示例图片和自定义数据集上验证零样本检测效果

如果你正在系统学习计算机视觉、多模态模型或项目复现，建议收藏本文；配套 notebook、示例图片和运行环境说明后续会继续整理。如果环境配置卡住，可以在评论区说明具体报错。

📚 文章目录

Qwen2.5-VL 零样本目标检测实战：提示词定位与自定义数据集测试
- ⚙️ 环境准备
- 🧠 加载 Qwen2.5-VL 模型
- 🛠️ 推理与可视化工具函数
- 🪑 示例 1：检测 each chair
- 👤 示例 2：检测坐着人的椅子
- 📌 示例 3：多类别检测
- 🥤 示例 4：检测玻璃杯
- ➡️ 示例 5：检测最右侧玻璃杯
- 🥢 示例 6：检测吸管
- 🧂 示例 7：检测 pepper 和 salt
- 📦 在自定义数据集上测试
- 📌 小结
- 📚 同系列教程汇总

⚙️ 环境准备

配置 HuggingFace Token 和数据集后台 API Key，检查 GPU，安装 maestro 与 supervision。

importosfromgoogle.colabimportuserdata os.environ["HF_TOKEN"]=userdata.get("HF_TOKEN")

!nvidia-smi

!pip install-q"maestro[qwen_2_5_vl]==1.1.0rc2"!pip install-q supervision==0.26.0

# 后台可私信获得下载数据集，上传到当前目录，或替换为对应资源链接。

🧠 加载 Qwen2.5-VL 模型

使用 maestro 封装加载 Qwen2.5-VL-7B-Instruct。

frommaestro.trainer.models.qwen_2_5_vl.checkpointsimportload_model,OptimizationStrategy MODEL_ID_OR_PATH="Qwen/Qwen2.5-VL-7B-Instruct"MIN_PIXELS=512*28*28MAX_PIXELS=2048*28*28processor,model=load_model(model_id_or_path=MODEL_ID_OR_PATH,optimization_strategy=OptimizationStrategy.NONE,min_pixels=MIN_PIXELS,max_pixels=MAX_PIXELS)

🛠️ 推理与可视化工具函数

定义推理函数和图片标注函数，后续所有示例复用这套流程。

fromPILimportImagefromtypingimportOptional,Tuple,Unionfrommaestro.trainer.models.qwen_2_5_vl.inferenceimportpredict_with_inputsfrommaestro.trainer.models.qwen_2_5_vl.loadersimportformat_conversationfrommaestro.trainer.common.utils.deviceimportparse_device_specfromqwen_vl_utilsimportprocess_vision_infodefrun_qwen_2_5_vl_inference(model,processor,image:Image.Image,prompt:str,system_message:Optional[str]=None,device:str="auto",max_new_tokens:int=1024,)->Tuple[str,Tuple[int,int]]:device=parse_device_spec(device)conversation=format_conversation(image=image,prefix=prompt,system_message=system_message)text=processor.apply_chat_template(conversation,tokenize=False,add_generation_prompt=True)image_inputs,_=process_vision_info(conversation)inputs=processor(text=text,images=image_inputs,return_tensors="pt",)input_h=inputs['image_grid_thw'][0][1]*14input_w=inputs['image_grid_thw'][0][2]*14response=predict_with_inputs(**inputs,model=model,processor=processor,device=device,max_new_tokens=max_new_tokens)[0]returnresponse,(int(input_w),int(input_h))

importsupervisionassvfromPILimportImagedefannotate_image(image:Image,detections:sv.Detections)->Image:text_scale=sv.calculate_optimal_text_scale(resolution_wh=image.size)thickness=sv.calculate_optimal_line_thickness(resolution_wh=image.size)ifdetections.maskisnotNone:mask_annotator=sv.MaskAnnotator(color_lookup=sv.ColorLookup.INDEX)image=mask_annotator.annotate(image,detections)else:box_annotator=sv.BoxAnnotator(color_lookup=sv.ColorLookup.INDEX,thickness=thickness)image=box_annotator.annotate(image,detections)label_annotator=sv.LabelAnnotator(color_lookup=sv.ColorLookup.INDEX,text_color=sv.Color.BLACK,text_scale=text_scale,text_thickness=thickness-1,smart_position=True)image=label_annotator.annotate(image,detections)returnimage

🪑 示例 1：检测 each chair

用 prompt 定位图中每一把椅子。

IMAGE_PATH="/content/dog-3.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of each chair and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

👤 示例 2：检测坐着人的椅子

通过更具体的描述约束目标范围。

IMAGE_PATH="/content/dog-3.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of chair with man sitting on it and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

📌 示例 3：多类别检测

一次给出多个类别，观察模型对复杂 prompt 的处理。

IMAGE_PATH="/content/dog-3.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of chair, dog, table, shoe, light bulb, coffee, hat, glasses, car, tail, umbrella and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

IMAGE_PATH="/content/dog-3.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of chair, dog, table, shoe, coffee, hat, glasses, car, tail, umbrella and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

🥤 示例 4：检测玻璃杯

在另一张图片上测试单类别定位。

IMAGE_PATH="/content/dog-2.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of glass and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

➡️ 示例 5：检测最右侧玻璃杯

用空间关系约束目标。

IMAGE_PATH="/content/dog-2.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of glass most to the right and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

🥢 示例 6：检测吸管

继续测试细长目标定位。

IMAGE_PATH="/content/dog-2.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of straw and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

🧂 示例 7：检测 pepper 和 salt

测试两个相近小物体的定位。

IMAGE_PATH="/content/dog-2.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of pepper, salt and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

📦 在自定义数据集上测试

下载安全背心数据集，并对测试集图片运行零样本检测。

# 如需额外的数据集下载依赖，请按数据集后台说明安装。

fromtypesimportSimpleNamespace# 从数据集后台下载并解压数据集后，修改 DATASET_DIR 指向数据集目录。DATASET_DIR="/content/dataset"# 修改为数据集后台导出的数据集目录dataset=SimpleNamespace(location=DATASET_DIR)

importsupervisionassv ds=sv.DetectionDataset.from_coco(images_directory_path=f"{dataset.location}/test",annotations_path=f"{dataset.location}/test/_annotations.coco.json",)

image_path,_,annotations=ds[0]SYSTEM_MESSAGE=NonePROMPT="Outline the position of helmet and output all the coordinates in JSON format."image=Image.open(image_path)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image