当前位置：首页 > news >正文

效率翻倍：Unsloth框架2倍速微调DeepSeek-R1模型实战记录

news 2026/5/12 20:08:37

效率翻倍：Unsloth框架2倍速微调DeepSeek-R1模型实战记录

1. 环境准备与模型加载

首先安装必要的库并加载DeepSeek-R1模型：

!pip install -qU unsloth transformers datasets from unsloth import FastLanguageModel import torch # 模型参数配置 max_seq_length = 2048 # 最大序列长度 dtype = None # 自动检测数据类型 load_in_4bit = True # 使用4bit量化节省显存 # 加载DeepSeek-R1模型 model, tokenizer = FastLanguageModel.from_pretrained( model_name = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B", max_seq_length = max_seq_length, dtype = dtype, load_in_4bit = load_in_4bit, )

2. 添加LoRA适配器

使用Unsloth的高效LoRA实现：

model = FastLanguageModel.get_peft_model( model, r = 16, # LoRA秩 target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"], lora_alpha = 16, lora_dropout = 0, # 推荐设为0以获得最佳性能 bias = "none", # 无偏置项 use_gradient_checkpointing = "unsloth", # 使用优化的梯度检查点 random_state = 3407, use_rslora = True, # 使用Rank-Stabilized LoRA )

3. 准备训练数据

加载并格式化训练数据集：

from datasets import load_dataset # 示例数据集格式 dataset = [ {"instruction": "解释牛顿第一定律", "input": "", "output": "牛顿第一定律..."}, # 更多数据... ] def formatting_func(examples): text = f"### Instruction:\n{examples['instruction']}\n\n### Input:\n{examples['input']}\n\n### Response:\n{examples['output']}" return {"text": text} formatted_dataset = dataset.map(formatting_func, batched=False)

4. 配置训练参数

from trl import SFTTrainer from transformers import TrainingArguments trainer = SFTTrainer( model = model, tokenizer = tokenizer, train_dataset = formatted_dataset, dataset_text_field = "text", max_seq_length = max_seq_length, args = TrainingArguments( per_device_train_batch_size = 2, gradient_accumulation_steps = 4, warmup_steps = 10, num_train_epochs = 3, learning_rate = 2e-4, fp16 = not torch.cuda.is_bf16_supported(), bf16 = torch.cuda.is_bf16_supported(), logging_steps = 1, optim = "adamw_8bit", weight_decay = 0.01, lr_scheduler_type = "linear", seed = 42, output_dir = "outputs", ), )

5. 开始训练

trainer.train() # 使用Unsloth加速训练

6. 模型推理测试

def generate_response(prompt): inputs = tokenizer(prompt, return_tensors="pt").to("cuda") outputs = model.generate( input_ids=inputs.input_ids, attention_mask=inputs.attention_mask, max_new_tokens=max_seq_length, temperature=0.7, top_p=0.9, ) return tokenizer.decode(outputs[0], skip_special_tokens=True) # 测试示例 print(generate_response("解释量子纠缠的基本概念"))

7. 模型保存

# 保存完整模型(16bit) model.save_pretrained_merged("finetuned_model", tokenizer, save_method="merged_16bit") # 保存4bit量化版本 model.save_pretrained_merged("finetuned_model_4bit", tokenizer, save_method="merged_4bit")