Xgboost参数调节
转自:https://segmentfault.com/a/1190000014040317
整体:
# 1.调试n_estimators
cv_params = {'n_estimators': [550, 575, 600, 650, 675]}
other_params = {'learning_rate': 0.1, 'n_estimators': 600, 'max_depth': 5, 'min_child_weight': 1, 'seed': 0,
'subsample': 0.8, 'colsample_bytree': 0.8, 'gamma': 0, 'reg_alpha': 0, 'reg_lambda': 1}
# 2.调试max_depth、min_child_weight
# cv_params = {'max_depth': [3, 4, 5, 6, 7, 8, 9, 10], 'min_child_weight': [1, 2, 3, 4, 5, 6]}
# other_params = {'learning_rate': 0.1, 'n_estimators': 550, 'max_depth': 5, 'min_child_weight': 1, 'seed': 0,
# 'subsample': 0.8, 'colsample_bytree': 0.8, 'gamma': 0, 'reg_alpha': 0, 'reg_lambda': 1}
# 3.调试gamma
# cv_params = {'gamma': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6]}
# other_params = {'learning_rate': 0.1, 'n_estimators': 550, 'max_depth': 4, 'min_child_weight': 5, 'seed': 0,
# 'subsample': 0.8, 'colsample_bytree': 0.8, 'gamma': 0, 'reg_alpha': 0, 'reg_lambda': 1}
# 4.调试subsample、colsample_bytree
# cv_params = {'subsample': [0.6, 0.7, 0.8, 0.9], 'colsample_bytree': [0.6, 0.7, 0.8, 0.9]}
# other_params = {'learning_rate': 0.1, 'n_estimators': 550, 'max_depth': 4, 'min_child_weight': 5, 'seed': 0,
# 'subsample': 0.8, 'colsample_bytree': 0.8, 'gamma': 0.1, 'reg_alpha': 0, 'reg_lambda': 1}
# 5.调试reg_alpha、reg_lambda
# cv_params = {'reg_alpha': [0.05, 0.1, 1, 2, 3], 'reg_lambda': [0.05, 0.1, 1, 2, 3]}
# other_params = {'learning_rate': 0.1, 'n_estimators': 550, 'max_depth': 4, 'min_child_weight': 5, 'seed': 0,
# 'subsample': 0.7, 'colsample_bytree': 0.7, 'gamma': 0.1, 'reg_alpha': 0, 'reg_lambda': 1}
# 6.调试learning_rate
# cv_params = {'learning_rate': [0.01, 0.05, 0.07, 0.1, 0.2]}
# other_params = {'learning_rate': 0.1, 'n_estimators': 550, 'max_depth': 4, 'min_child_weight': 5, 'seed': 0,
# 'subsample': 0.7, 'colsample_bytree': 0.7, 'gamma': 0.1, 'reg_alpha': 1, 'reg_lambda': 1} model = xgb.XGBClassifier(**other_params)
optimized_GBM = GridSearchCV(estimator=model, param_grid=cv_params, cv=5, verbose=1, n_jobs=4)
optimized_GBM.fit(X_train, y_train)
evalute_result = optimized_GBM.grid_scores_
print('每轮迭代运行结果:{0}'.format(evalute_result))
print('参数的最佳取值:{0}'.format(optimized_GBM.best_params_))
print('最佳模型得分:{0}'.format(optimized_GBM.best_score_))
1.调节最大迭代次数n_estimators
# 最佳迭代次数:n_estimators
from xgboost import XGBRegressor
from sklearn.model_selection import GridSearchCV
cv_params = {'n_estimators': [20,30,40]}
other_params = {'learning_rate': 0.1, 'n_estimators': 500, 'max_depth': 5, 'min_child_weight': 1, 'seed': 0,
'subsample': 0.8, 'colsample_bytree': 0.8, 'gamma': 0, 'reg_alpha': 0, 'reg_lambda': 1}
model = XGBRegressor(**other_params)
optimized_GBM = GridSearchCV(estimator=model, param_grid=cv_params, scoring='r2', cv=3, verbose=1, n_jobs=-1)
optimized_GBM.fit(x_data, y_data)
evalute_result =optimized_GBM.return_train_score
print('每轮迭代运行结果:{0}'.format(evalute_result))
print('参数的最佳取值:{0}'.format(optimized_GBM.best_params_))
print('最佳模型得分:{0}'.format(optimized_GBM.best_score_))
2.调试的参数是min_child_weight以及max_depth:
# 调试的参数是min_child_weight以及max_depth:
cv_params = {'max_depth': [3, 4, 5, 6, 7, 8, 9, 10], 'min_child_weight': [6,7,8]}
other_params = {'learning_rate': 0.1, 'n_estimators': 20, 'max_depth': 5, 'min_child_weight': 1, 'seed': 0,
'subsample': 0.8, 'colsample_bytree': 0.8, 'gamma': 0, 'reg_alpha': 0, 'reg_lambda': 1}
model = XGBRegressor(**other_params)
optimized_GBM = GridSearchCV(estimator=model, param_grid=cv_params, scoring='r2', cv=3, verbose=1, n_jobs=-1)
optimized_GBM.fit(x_data, y_data)
evalute_result =optimized_GBM.return_train_score
print('每轮迭代运行结果:{0}'.format(evalute_result))
print('参数的最佳取值:{0}'.format(optimized_GBM.best_params_))
print('最佳模型得分:{0}'.format(optimized_GBM.best_score_))
3.调试参数:gamma:
# 调试参数:gamma:
cv_params = {'gamma': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6]}
other_params = {'learning_rate': 0.1, 'n_estimators': 20, 'max_depth': 4, 'min_child_weight': 6, 'seed': 0,
'subsample': 0.8, 'colsample_bytree': 0.8, 'gamma': 0, 'reg_alpha': 0, 'reg_lambda': 1}
model = XGBRegressor(**other_params)
optimized_GBM = GridSearchCV(estimator=model, param_grid=cv_params, scoring='r2', cv=3, verbose=1, n_jobs=-1)
optimized_GBM.fit(x_data, y_data)
evalute_result =optimized_GBM.return_train_score
print('每轮迭代运行结果:{0}'.format(evalute_result))
print('参数的最佳取值:{0}'.format(optimized_GBM.best_params_))
print('最佳模型得分:{0}'.format(optimized_GBM.best_score_))
4. 调试subsample以及colsample_bytree:
# 调试subsample以及colsample_bytree:
cv_params = {'subsample': [0.6, 0.7, 0.8, 0.9], 'colsample_bytree': [0.6, 0.7, 0.8, 0.9]}
other_params = {'learning_rate': 0.1, 'n_estimators': 20, 'max_depth': 4, 'min_child_weight': 6, 'seed': 0,
'subsample': 0.8, 'colsample_bytree': 0.8, 'gamma': 0.2, 'reg_alpha': 0, 'reg_lambda': 1}
model = XGBRegressor(**other_params)
optimized_GBM = GridSearchCV(estimator=model, param_grid=cv_params, scoring='r2', cv=3, verbose=1, n_jobs=4)
optimized_GBM.fit(x_data, y_data)
evalute_result =optimized_GBM.return_train_score
print('每轮迭代运行结果:{0}'.format(evalute_result))
print('参数的最佳取值:{0}'.format(optimized_GBM.best_params_))
print('最佳模型得分:{0}'.format(optimized_GBM.best_score_))
5.调试reg_alpha以及reg_lambda:
# 调试reg_alpha以及reg_lambda:
cv_params = {'reg_alpha': [0.05, 0.1, 1, 2, 3], 'reg_lambda': [0.05, 0.1, 1, 2, 3]}
other_params = {'learning_rate': 0.1, 'n_estimators': 20, 'max_depth': 4, 'min_child_weight': 6, 'seed': 0,
'subsample': 0.8, 'colsample_bytree': 0.9, 'gamma': 0.2, 'reg_alpha': 0, 'reg_lambda': 1}
model = XGBRegressor(**other_params)
optimized_GBM = GridSearchCV(estimator=model, param_grid=cv_params, scoring='r2', cv=3, verbose=1, n_jobs=4)
optimized_GBM.fit(x_data, y_data)
evalute_result =optimized_GBM.return_train_score
print('每轮迭代运行结果:{0}'.format(evalute_result))
print('参数的最佳取值:{0}'.format(optimized_GBM.best_params_))
print('最佳模型得分:{0}'.format(optimized_GBM.best_score_))
6.调试learning_rate:
# 调试learning_rate,一般这时候要调小学习率来测试:
cv_params = {'learning_rate': [0.01, 0.05, 0.07, 0.1, 0.2]}
other_params = {'learning_rate': 0.1, 'n_estimators': 20, 'max_depth': 4, 'min_child_weight': 6, 'seed': 0,
'subsample': 0.8, 'colsample_bytree': 0.9, 'gamma': 0.2, 'reg_alpha': 0.1, 'reg_lambda': 1}
model = XGBRegressor(**other_params)
optimized_GBM = GridSearchCV(estimator=model, param_grid=cv_params, scoring='r2', cv=3, verbose=1, n_jobs=4)
optimized_GBM.fit(x_data, y_data)
evalute_result =optimized_GBM.return_train_score
print('每轮迭代运行结果:{0}'.format(evalute_result))
print('参数的最佳取值:{0}'.format(optimized_GBM.best_params_))
print('最佳模型得分:{0}'.format(optimized_GBM.best_score_))
Xgboost参数调节的更多相关文章
- XGBoost参数调优
XGBoost参数调优 http://blog.csdn.net/hhy518518/article/details/54988024 摘要: 转载:http://blog.csdn.NET/han_ ...
- XGBoost参数调优完全指南(附Python代码)
XGBoost参数调优完全指南(附Python代码):http://www.2cto.com/kf/201607/528771.html https://www.zhihu.com/question/ ...
- XGBoost参数
XGBoost参数 转自http://blog.csdn.net/zc02051126/article/details/46711047 在运行XGboost之前,必须设置三种类型成熟:general ...
- linux 内核参数VM调优 之 参数调节和场景分析
1. pdflush刷新脏数据条件 (linux IO 内核参数调优 之 原理和参数介绍)上一章节讲述了IO内核调优介个重要参数参数. 总结可知cached中的脏数据满足如下几个条件中一个或者多个的时 ...
- xgboost 参数
XGBoost 参数 在运行XGBoost程序之前,必须设置三种类型的参数:通用类型参数(general parameters).booster参数和学习任务参数(task parameters). ...
- (转)linux IO 内核参数调优 之 参数调节和场景分析
1. pdflush刷新脏数据条件 (linux IO 内核参数调优 之 原理和参数介绍)上一章节讲述了IO内核调优介个重要参数参数. 总结可知cached中的脏数据满足如下几个条件中一个或者多个的时 ...
- inux IO 内核参数调优 之 参数调节和场景分析
http://backend.blog.163.com/blog/static/2022941262013112081215609/ http://blog.csdn.net/icycode/arti ...
- 【转】XGBoost参数调优完全指南(附Python代码)
xgboost入门非常经典的材料,虽然读起来比较吃力,但是会有很大的帮助: 英文原文链接:https://www.analyticsvidhya.com/blog/2016/03/complete-g ...
- 机器学习——XGBoost大杀器,XGBoost模型原理,XGBoost参数含义
0.随机森林的思考 随机森林的决策树是分别采样建立的,各个决策树之间是相对独立的.那么,在我们得到了第k-1棵决策树之后,能否通过现有的样本和决策树的信息, 对第m颗树的建立产生有益的影响呢?在随机森 ...
随机推荐
- Python学习之路5☞文件处理
一.文件处理流程 打开文件,得到文件句柄并赋值给一个变量 通过句柄对文件进行操作 关闭文件 正趣果上果 Interesting fruit fruit 词:郭婞 曲:陈粒 编曲/混音/和声:燕池 萧: ...
- SharePoint开发中怎样使用Visual Studio给你的Web Part加入图标
版权声明:本文为博主原创文章.未经博主同意不得转载. https://blog.csdn.net/u012025054/article/details/36051545 SharePoint开发中怎样 ...
- 基于opencv的RandomForest随机森林
2.OpenCV函数使用 OpenCV提供了随机森林的相关类和函数.具体使用方法如下: (1)首先利用CvRTParams定义自己的参数,其格式如下 CvRTParams::CvRTParams(in ...
- IDEA-创建WEB项目并部署Tomcat
一.创建简单web项目 1.创建一个web project File -> new Project ->选择project sdk 为1.6(如果没有sdk的同学请先配置)-> Ne ...
- Mac OSX原生读写NTFS功能开启方法
macOX系统内建的NTFS支持默认只能读不能写 原生读写NTFS,需要自行终端命令手动开启 1. 插上磁盘 此时Mac桌面应该会显示出插入的磁盘,但是当你想把文件拖入磁盘的时候,发现是不能拖进去的, ...
- Java版各种排序算法 (冒泡,快速,选择,插入)
package com.test4; import java.util.*; //Calendar 显示时间 /** * @author qingfeng * 功能:排序算法 */ public cl ...
- HTML标题title滚动
上代码: <script type="text/javascript"> var msg = document.title; msg = "…" + ...
- @codeforces - 708D@ Incorrect Flow
目录 @description@ @solution@ @accepted code@ @details@ @description@ 给定一个有源点与汇点的图 G,并对于每一条边 (u, v) 给定 ...
- H3C 局域网的不足
- 异常解决:java.lang.IllegalStateException: Failed to introspect Class
java.lang.IllegalStateException: Failed to introspect Class 异常详情 原因 解决办法 异常详情 Exception encountered ...