XGBoost 调参实战：从入门到竞赛金牌

XGBoost 调参实战：从入门到竞赛金牌

1. XGBoost 核心参数

XGBoost 参数分类： ├── 树参数 │ ├── max_depth：树深度（3-10） │ ├── min_child_weight：叶子最小权重 │ ├── gamma：分裂最小增益 │ └── subsample：行采样比例 ├── 学习参数 │ ├── learning_rate：学习率（0.01-0.3） │ ├── n_estimators：树数量 │ └── colsample_bytree：列采样比例 ├── 正则化参数 │ ├── reg_alpha：L1 正则化 │ └── reg_lambda：L2 正则化 └── 类别参数 ├── scale_pos_weight：正负样本比 └── objective：目标函数

2. 分步调参法

importxgboostasxgbfromsklearn.model_selectionimportGridSearchCV# 步骤 1：确定树数量和学习率params_1={'learning_rate':0.1,'n_estimators':100,'max_depth':5,'min_child_weight':1,'subsample':0.8,'colsample_bytree':0.8,}# 步骤 2：调 max_depth 和 min_child_weightparam_grid_2={'max_depth':[3,5,7,9],'min_child_weight':[1,3,5,7],}grid_2=GridSearchCV(xgb.XGBClassifier(**params_1,use_label_encoder=False),param_grid_2,cv=5,scoring='accuracy')grid_2.fit(X_train,y_train)print(f"最佳:{grid_2.best_params_}")# 步骤 3：调 subsample 和 colsample_bytreeparam_grid_3={'subsample':[0.6,0.7,0.8,0.9],'colsample_bytree':[0.6,0.7,0.8,0.9],}# 步骤 4：调正则化param_grid_4={'reg_alpha':[0,0.01,0.1,1],'reg_lambda':[0,0.01,0.1,1],}# 步骤 5：降低学习率，增加树数量params_final={'learning_rate':0.01,'n_estimators':1000,'early_stopping_rounds':50,}

3. Optuna 自动调参

importoptunadefobjective(trial):params={'n_estimators':trial.suggest_int('n_estimators',100,1000),'max_depth':trial.suggest_int('max_depth',3,10),'learning_rate':trial.suggest_float('learning_rate',0.01,0.3,log=True),'min_child_weight':trial.suggest_int('min_child_weight',1,10),'subsample':trial.suggest_float('subsample',0.6,1.0),'colsample_bytree':trial.suggest_float('colsample_bytree',0.6,1.0),'reg_alpha':trial.suggest_float('reg_alpha',1e-8,10.0,log=True),'reg_lambda':trial.suggest_float('reg_lambda',1e-8,10.0,log=True),'gamma':trial.suggest_float('gamma',0,5),}model=xgb.XGBClassifier(**params,use_label_encoder=False,eval_metric='logloss')scores=cross_val_score(model,X_train,y_train,cv=5,scoring='accuracy')returnscores.mean()study=optuna.create_study(direction='maximize')study.optimize(objective,n_trials=200)

4. 竞赛技巧

竞赛常用技巧： ├── 特征工程 > 模型调参 ├── 多模型融合（Blending/Stacking） ├── 5-10 折交叉验证 ├── 数据增强（SMOTE/噪声注入） ├── 伪标签（Pseudo Labeling） └── 后处理（阈值优化）

总结