Unsloth实战：DeepSeek-R1模型高效微调完整步骤解析
Unsloth实战：DeepSeek-R1模型高效微调完整步骤解析
1. 环境准备与模型加载
首先安装必要的库并加载DeepSeek-R1模型：
from unsloth import FastLanguageModel import torch # 设置最大序列长度和数据类型 max_seq_length = 2048 dtype = None # 自动检测 load_in_4bit = True # 使用4bit量化减少显存占用 # 加载预训练模型 model, tokenizer = FastLanguageModel.from_pretrained( model_name = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B", max_seq_length = max_seq_length, dtype = dtype, load_in_4bit = load_in_4bit, )2. 创建LoRA适配器
为模型添加LoRA适配器以实现高效微调：
model = FastLanguageModel.get_peft_model( model, r = 16, # LoRA秩 target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"], lora_alpha = 16, lora_dropout = 0, bias = "none", use_gradient_checkpointing = "unsloth", random_state = 3407, use_rslora = False, loftq_config = None, )3. 准备训练数据
加载并格式化训练数据集：
from datasets import load_dataset # 加载数据集 dataset = load_dataset("json", data_files="your_dataset.json") # 格式化函数 def formatting_prompts_func(examples): texts = examples["text"] return {"text" : texts} # 应用格式化 dataset = dataset.map(formatting_prompts_func, batched=True)4. 配置训练参数
设置训练参数并初始化训练器：
from trl import SFTTrainer from transformers import TrainingArguments trainer = SFTTrainer( model = model, tokenizer = tokenizer, train_dataset = dataset, dataset_text_field = "text", max_seq_length = max_seq_length, args = TrainingArguments( per_device_train_batch_size = 2, gradient_accumulation_steps = 4, warmup_steps = 5, num_train_epochs = 1, learning_rate = 2e-4, fp16 = not torch.cuda.is_bf16_supported(), bf16 = torch.cuda.is_bf16_supported(), logging_steps = 1, optim = "adamw_8bit", weight_decay = 0.01, lr_scheduler_type = "linear", seed = 3407, output_dir = "outputs", ), )5. 执行模型训练
开始微调过程：
trainer.train()6. 模型推理测试
训练完成后进行推理测试：
# 启用快速推理模式 FastLanguageModel.for_inference(model) # 测试函数 def generate_response(question): inputs = tokenizer(question, return_tensors="pt").to("cuda") outputs = model.generate( input_ids=inputs.input_ids, attention_mask=inputs.attention_mask, max_new_tokens=max_seq_length, temperature=0.6, top_p=0.95, use_cache=False, ) return tokenizer.decode(outputs[0], skip_special_tokens=True)7. 保存微调后的模型
将训练好的模型保存到本地：
# 保存为16bit合并模型 model.save_pretrained_merged( save_directory = "DeepSeekR1-1.5B-finetuned-fp16", tokenizer = tokenizer, save_method = "merged_16bit" ) # 也可保存为4bit量化版本 model.save_pretrained_merged( save_directory = "DeepSeekR1-1.5B-finetuned-4bit", tokenizer = tokenizer, save_method = "merged_4bit" )关键参数说明
LoRA配置：
r: LoRA秩，影响模型微调的精细程度lora_alpha: LoRA缩放系数，通常设置为r的1-2倍target_modules: 应用LoRA的目标模块
训练参数：
per_device_train_batch_size: 每个设备的批次大小gradient_accumulation_steps: 梯度累积步数learning_rate: 学习率，LoRA微调通常使用较高学习率
推理参数：
temperature: 控制生成随机性top_p: 核采样参数，控制生成多样性