轻量化微调 Qwen2.5 LoRA 训练全流程详解

视频链接：https://www.bilibili.com/video/BV1FGKz6FEqk/?vd_source=5ba34935b7845cd15c65ef62c64ba82f

代码仓库：https://github.com/LitchiCheng/LLM-learning

分享下使用LoRA进行微调LLM的学习，选择Qwen2.5-0.5B-INstruct，参数量很小，应该随便一台机器都可以训练，这里HF可能下载比较慢，训练的模型

https://www.modelscope.cn/models/Qwen/Qwen2.5-0.5B-Instruct

从魔搭下载

参考例子

https://huggingface.co/docs/peft/index

tokenizer

from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True) tokenizer.pad_token = tokenizer.eos_token

加载对应的分词器，每个汉字/英文单词都会被拆成若干 token ID，在模型眼中就是数字

训练时不同样本长度不同，需要把短序列补到统一长度，eos就是end of sequence，用这个填充，并且告知模型这句话到这儿结束了

.cache/modelscope/models/qwen--Qwen2.5-0.5B-Instruct/snapshots/master/tokenizer.json 中可以看到 ID

模型加载

from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained( model_path, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True )

把模型加载到内存中，dtype决定加载到显存中的权重的类型，可以修改，比如qwen2.5下载下来是fp16，可以改成float32，但显存就会多占一倍

print(f" Params: {sum(p.numel() for p in model.parameters()):,}")

每个张量加起来的总和，共0.49B的参数量

LoRA原理

原始模型的一层线性变换：y = Wx + b，W 是一个大矩阵（比如 4096×4096）。不直接改 W，而是在旁边挂两个小矩阵 A 和 B：y = (W + B·A) · x + b

W: 原始权重（冻结不动） 4096 × 4096

A: 随机初始化的小矩阵 4096 × r

B: 随机初始化的小矩阵 r × 4096

r: 秩（rank），控制小矩阵的大小

因为 B·A 的乘积秩最多是 r，所以只训练 A 和 B ，用最精华的特征来调整，就能近似地"修补"W。

from peft import LoraConfig, get_peft_model, TaskType print("\nConfiguring LoRA...") lora = LoraConfig( r=16, lora_alpha=32, target_modules=[ "q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj" ], lora_dropout=0.05, bias="none", task_type=TaskType.CAUSAL_LM ) model = get_peft_model(model, lora) trainable = sum(p.numel() for p in model.parameters() if p.requires_grad) total = sum(p.numel() for p in model.parameters()) print(f" OK: Trainable {trainable:,} / {total:,} ({100*trainable/total:.2f}%)")

ChatML

https://huggingface.co/docs/transformers/chat_templating

samples = [ {"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "你是谁?"}, {"role": "assistant", "content": "你是LitchiCheng微调的Qwen2.5模型"}]}, {"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "你是谁微调的?"}, {"role": "assistant", "content": "LitchiCheng"}]}, {"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "你是不是标准的Qwen2.5模型?"}, {"role": "assistant", "content": "不是，我是LitchiCheng微调的模型"}]}, ] def format_sample(s): return {"text": tokenizer.apply_chat_template(s["messages"], tokenize=False, add_generation_prompt=False)} dataset = Dataset.from_list([format_sample(s) for s in samples]) print(f" OK: {len(dataset)} samples")

按照OpenAI的ChatML格式，system 告知角色，User 作为输入，assistant 作为输出

微调训练

args = TrainingArguments( output_dir=output_dir, per_device_train_batch_size=1, gradient_accumulation_steps=4, learning_rate=2e-4, max_steps=100, logging_steps=1, save_steps=10, fp16=True, report_to="none", ) trainer = SFTTrainer(model=model, train_dataset=dataset, args=args) start = time.time() result = trainer.train() elapsed = time.time() - start print(f"\n OK: Done in {elapsed:.1f}s") print(f" Loss: {result.training_loss:.4f}") print(f" Speed: {result.global_step/elapsed:.2f} steps/sec") # 保存 model.save_pretrained(f'{output_dir}/lora') tokenizer.save_pretrained(f'{output_dir}/lora')

微调测试

如下为完整代码，测试提问是什么模型，看它怎么回答？

#!/usr/bin/env python3 """Qwen2.5 微调推理脚本 - 加载 LoRA 权重进行推理""" import os, sys, time import torch CACHE_DIR = os.path.join(os.path.dirname(__file__), ".cache", "modelscope", "models", "qwen--Qwen2.5-0.5B-Instruct", "snapshots", "master") BASE_MODEL = CACHE_DIR LORA_DIR = "fine_tune_output/20260718_212052/lora" print("Loading tokenizer...") from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained(LORA_DIR, trust_remote_code=True) tokenizer.pad_token = tokenizer.eos_token print(f" OK: Vocabulary size {len(tokenizer)}") # 加载基座模型 + LoRA 权重 print("\nLoading base model...") from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained( BASE_MODEL, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True ) # 用 PEFT 加载 LoRA adapter 并合并到基座模型 print("Loading LoRA weights...") from peft import PeftModel model = PeftModel.from_pretrained(model, LORA_DIR) model = model.merge_and_unload() # 将 LoRA 权重合并进基座，释放 LoRA 内存 print(f" OK: Model loaded, params: {sum(p.numel() for p in model.parameters()):,}") # 用 chat template 做正式对话测试 print("\nChat test:") messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "你是什么模型"}, ] prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=100) resp = tokenizer.decode(out[0], skip_special_tokens=True)[len(tokenizer.eos_token):].strip() print(f" {resp}") print("\n" + "="*60) print("Fine-tuning inference test passed!") print("="*60)