当前位置：首页 > news >正文

Phi-3.5-Mini-Instruct快速上手：CLI命令行模式调用与API服务封装方法

news 2026/4/24 18:49:21

Phi-3.5-Mini-Instruct快速上手：CLI命令行模式调用与API服务封装方法

1. 项目简介

Phi-3.5-Mini-Instruct是微软推出的轻量级旗舰小模型，具备出色的逻辑推理、代码生成和问答能力。本文将带您快速掌握如何在命令行模式下调用该模型，以及如何将其封装为API服务，实现更灵活的部署方式。

2. 环境准备

2.1 硬件要求

显卡：NVIDIA显卡，显存≥8GB（推荐RTX 3060及以上）
内存：16GB及以上
存储：至少10GB可用空间

2.2 软件依赖

pip install torch transformers fastapi uvicorn

3. 基础命令行调用

3.1 模型加载与初始化

from transformers import pipeline # 初始化对话管道 chat_pipe = pipeline( "text-generation", model="microsoft/Phi-3.5-Mini-Instruct", torch_dtype="auto", device_map="auto" )

3.2 单次对话示例

response = chat_pipe( "请用Python实现快速排序算法", max_new_tokens=512, temperature=0.7 ) print(response[0]['generated_text'])

3.3 多轮对话实现

# 对话历史管理 conversation = [] def chat(message): global conversation conversation.append({"role": "user", "content": message}) response = chat_pipe( conversation, max_new_tokens=1024, do_sample=True ) assistant_reply = response[0]['generated_text'][-1]["content"] conversation.append({"role": "assistant", "content": assistant_reply}) return assistant_reply

4. API服务封装

4.1 FastAPI基础服务

from fastapi import FastAPI from pydantic import BaseModel app = FastAPI() class ChatRequest(BaseModel): message: str max_tokens: int = 1024 temperature: float = 0.7 @app.post("/chat") async def chat_endpoint(request: ChatRequest): response = chat_pipe( request.message, max_new_tokens=request.max_tokens, temperature=request.temperature ) return {"response": response[0]['generated_text']}

4.2 启动API服务

uvicorn main:app --host 0.0.0.0 --port 8000

4.3 带对话历史的API实现

from fastapi import FastAPI, HTTPException from pydantic import BaseModel from uuid import uuid4 app = FastAPI() sessions = {} class SessionRequest(BaseModel): message: str session_id: str = None max_tokens: int = 1024 temperature: float = 0.7 @app.post("/chat") async def chat_with_history(request: SessionRequest): if not request.session_id: request.session_id = str(uuid4()) sessions[request.session_id] = [] conversation = sessions[request.session_id] conversation.append({"role": "user", "content": request.message}) try: response = chat_pipe( conversation, max_new_tokens=request.max_tokens, temperature=request.temperature ) assistant_reply = response[0]['generated_text'][-1]["content"] conversation.append({"role": "assistant", "content": assistant_reply}) return { "response": assistant_reply, "session_id": request.session_id } except Exception as e: raise HTTPException(status_code=500, detail=str(e))

5. 高级配置与优化

5.1 性能优化参数

# 优化后的管道配置 chat_pipe = pipeline( "text-generation", model="microsoft/Phi-3.5-Mini-Instruct", torch_dtype=torch.bfloat16, device_map="auto", model_kwargs={ "load_in_4bit": True, # 4位量化 "bnb_4bit_compute_dtype": torch.bfloat16, "bnb_4bit_use_double_quant": True } )

5.2 流式响应实现

from fastapi import Response from fastapi.responses import StreamingResponse @app.post("/stream_chat") async def stream_chat(request: ChatRequest): def generate(): for chunk in chat_pipe( request.message, max_new_tokens=request.max_tokens, temperature=request.temperature, stream=True ): yield chunk[0]['generated_text'] return StreamingResponse(generate(), media_type="text/plain")