CatBoost算法和调参
python风控建模实战lendingClub(博主录制,catboost,lightgbm建模,2K超清分辨率)
https://study.163.com/course/courseMain.htm?courseId=1005988013&share=2&shareId=400000000398149
catboost官网文档
catboost GPU官方文档
https://catboost.ai/docs/features/training-on-gpu.html
catboost特点:
少量或无需调参,默认参数效果非常好
支持分类变量
支持GPU
catboost代码举例
# -*- coding: utf-8 -*-
"""
Created on Sun Jul 1 12:24:21 2018 @author: Administrator
"""
from sklearn import metrics
from sklearn.model_selection import train_test_split
from sklearn.grid_search import GridSearchCV
import catboost as cb
from sklearn.datasets import load_breast_cancer cancer=load_breast_cancer()
X, y = cancer.data,cancer.target
train_x, test_x, y_train, y_test=train_test_split(X,y,test_size=0.3,random_state=0) cat_features_index = [0,1,2,3,4,5,6] def auc(m, train, test):
return (metrics.roc_auc_score(y_train,m.predict_proba(train)[:,1]),
metrics.roc_auc_score(y_test,m.predict_proba(test)[:,1])) params = {'depth': [4, 7, 10],
'learning_rate' : [0.03, 0.1, 0.15],
'l2_leaf_reg': [1,4,9],
'iterations': [300]}
cb = cb.CatBoostClassifier()
#cb_model = GridSearchCV(cb, params, scoring="roc_auc", cv = 5)
cb.fit(train_x, y_train) print("accuracy on the training subset:{:.3f}".format(cb.score(train_x,y_train)))
print("accuracy on the test subset:{:.3f}".format(cb.score(test_x,y_test)))
'''
accuracy on the training subset:1.000
accuracy on the test subset:0.982
'''
俄罗斯最大搜索引擎Yandex开源了一款梯度提升机器学习库CatBoost
摘要: 俄罗斯搜索巨头Yandex宣布,将向开源社区提交一款梯度提升机器学习库CatBoost。它能够在数据稀疏的情况下“教”机器学习。特别是在没有像视频、文本、图像这类感官型数据的时候,CatBoost也能根据事务型数据或历史数据进行操作。
据开发者所说超越Lightgbm和XGBoost的又一个神器,不过具体性能,还要看在比赛中的表现了。
整理一下里面简单的教程和参数介绍,很多参数不是那种重要,只解释部分重要的参数,训练时需要重点考虑的。
import numpy as np
import catboost as cb train_data = np.random.randint(0, 100, size=(100, 10))
train_label = np.random.randint(0, 2, size=(100))
test_data = np.random.randint(0,100, size=(50,10)) model = cb.CatBoostClassifier(iterations=2, depth=2, learning_rate=0.5, loss_function='Logloss',
logging_level='Verbose')
model.fit(train_data, train_label, cat_features=[0,2,5])
preds_class = model.predict(test_data)
preds_probs = model.predict_proba(test_data)
print('class = ',preds_class)
print('proba = ',preds_probs)
参数
CatBoostClassifier/CatBoostRegressor
通用参数
learning_rate(eta)=automatically
depth(max_depth)=6: 树的深度
l2_leaf_reg(reg_lambda)=3 L2正则化系数
n_estimators(num_boost_round)(num_trees=1000)=1000: 解决ml问题的树的最大数量
one_hot_max_size=2: 对于某些变量进行one-hot编码
loss_function=’Logloss’:
RMSE
Logloss
MAE
CrossEntropy
custom_metric=None
RMSE
Logloss
MAE
CrossEntropy
Recall
Precision
F1
Accuracy
AUC
R2
eval_metric=Optimized objective
RMSE
Logloss
MAE
CrossEntropy
Recall
Precision
F1
Accuracy
AUC
R2
nan_mode=None:处理NAN的方法
Forbidden
Min
Max
1
2
3
leaf_estimation_method=None:迭代求解的方法,梯度和牛顿
Newton
Gradient
1
2
random_seed=None: 训练时候的随机种子
---------------------
性能参数
thread_count=-1:训练时所用的cpu/gpu核数
used_ram_limit=None:CTR问题,计算时的内存限制
gpu_ram_part=None:GPU内存限制
处理单元设置
task_type=CPU:训练的器件
devices=None:训练的GPU设备ID
counter_calc_method=None,
leaf_estimation_iterations=None,
use_best_model=None,
verbose=None,
model_size_reg=None,
rsm=None,
logging_level=None,
metric_period=None,
ctr_leaf_count_limit=None,
store_all_simple_ctr=None,
max_ctr_complexity=None,
has_time=None,
classes_count=None,
class_weights=None,
random_strength=None,
name=None,
ignored_features=None,
train_dir=None,
custom_loss=None,
bagging_temperature=None
border_count=None
feature_border_type=None,
save_snapshot=None,
snapshot_file=None,
fold_len_multiplier=None,
allow_writing_files=None,
final_ctr_computation_mode=None,
approx_on_full_history=None,
boosting_type=None,
simple_ctr=None,
combinations_ctr=None,
per_feature_ctr=None,
device_config=None,
bootstrap_type=None,
subsample=None,
colsample_bylevel=None,
random_state=None,
objective=None,
max_bin=None,
scale_pos_weight=None,
gpu_cat_features_storage=None,
data_partition=None
CatBoostClassifier
属性(attribute):
is_fitted_
tree_count_
feature_importances_
random_seed_
方法(method):
fit
X: 输入数据数据类型可以是,list; pandas.DataFrame; pandas.Series
y=None
cat_features=None: 拿来做处理的类别特征
sample_weight=None: 输入数据的样本权重
logging_level=None: 控制是否输出日志信息,或者何种信息
plot=False: 训练过程中,绘制,度量值,所用时间等
eval_set=None: 验证集合,数据类型list(X, y)tuples
baseline=None
use_best_model=None
verbose=None
predict
返回验证样本所属类别,数据类型为np.array
predict_proba
返回验证样本所属类别的概率,数据类型为np.array
get_feature_importance
eval_metrics
save_model
load_model
get_params
score
教程(tutorial)
catboost GPU版本
https://catboost.ai/docs/features/training-on-gpu.html
Training on GPU
CatBoost supports training on GPUs.
Choose the implementation for more details on the parameters that are required to start training on GPU.
Note. Other training parameters are also available. Some of them are CPU-specific or GPU-specific. See the Python package training parameters section for more details.
For example, use the following code to train a classification model on GPU:
GPU版本测试
# -*- coding: utf-8 -*-
"""
Created on Wed May 29 10:34:20 2019 @author: Administrator
GPU
1m -1m 2s
CPU
28.7s-28.8s
""" from catboost import CatBoostClassifier train_data = [[0, 3],
[4, 1],
[8, 1],
[9, 1]]
train_labels = [0, 0, 1, 1] model = CatBoostClassifier(task_type = "GPU")
#model = CatBoostClassifier()
model.fit(train_data,
train_labels)
# -*- coding: utf-8 -*-
"""
Created on Wed May 22 10:50:59 2019 @author: Administrator
CPU版本:3m 30s-3m 40s
GPU版本:3m 33s-3m 34s
""" from sklearn import metrics
from sklearn.model_selection import train_test_split
import catboost as cb
from sklearn.datasets import load_breast_cancer cancer=load_breast_cancer()
X, y = cancer.data,cancer.target
train_x, test_x, y_train, y_test=train_test_split(X,y,test_size=0.3,random_state=0)
cb = cb.CatBoostClassifier()
#cb = cb.CatBoostClassifier(task_type="CPU")
#cb = cb.CatBoostClassifier(task_type="GPU")
cb.fit(train_x, y_train)
Python package installation
Installation is only supported by the 64-bit version of Python.
- Choose an installation method:
(Optionally) Install additional packages for data visualization support.
- (Optionally) Test CatBoost.
Note that there are additional system requirements if training on GPU is required.
GPU system requirements
The versions of CatBoost available from pip install and conda install have GPU support out-of-the-box.
Devices with compute capability 3.0 and higher are supported in compiled packages.
Training on GPU requires NVIDIA Driver of version 390.xx or higher.
- Step 4 of the Build from source on Linux and macOS operation.
- Step 3 of the Build from source on Windows operation.
python机器学习-乳腺癌细胞挖掘(博主亲自录制视频,含catboost算法讲解)
https://study.163.com/course/introduction.htm?courseId=1005269003&utm_campaign=commission&utm_source=cp-400000000398149&utm_medium=share
欢迎扫描和关注博主主页,学习python视频资源
CatBoost算法和调参的更多相关文章
- 调参、最优化、ml算法(未完成)
最优化方法 调参方法 ml算法 梯度下降gd grid search lr 梯度上升 随机梯度下降 pca 随机梯度下降sgd 贝叶斯调参 lda 牛顿算法 knn 拟牛顿算法 kmeans ...
- 自动调参库hyperopt+lightgbm 调参demo
在此之前,调参要么网格调参,要么随机调参,要么肉眼调参.虽然调参到一定程度,进步有限,但仍然很耗精力. 自动调参库hyperopt可用tpe算法自动调参,实测强于随机调参. hyperopt 需要自己 ...
- k-近邻算法采用for循环调参方法
//2019.08.02下午#机器学习算法中的超参数与模型参数1.超参数:是指机器学习算法运行之前需要指定的参数,是指对于不同机器学习算法属性的决定参数.通常来说,人们所说的调参就是指调节超参数.2. ...
- LightGBM 调参方法(具体操作)
sklearn实战-乳腺癌细胞数据挖掘(博主亲自录制视频) https://study.163.com/course/introduction.htm?courseId=1005269003& ...
- sklearn逻辑回归(Logistic Regression,LR)调参指南
python信用评分卡建模(附代码,博主录制) https://study.163.com/course/introduction.htm?courseId=1005214003&utm_ca ...
- scikit-learn随机森林调参小结
在Bagging与随机森林算法原理小结中,我们对随机森林(Random Forest, 以下简称RF)的原理做了总结.本文就从实践的角度对RF做一个总结.重点讲述scikit-learn中RF的调参注 ...
- scikit-learn 梯度提升树(GBDT)调参小结
在梯度提升树(GBDT)原理小结中,我们对GBDT的原理做了总结,本文我们就从scikit-learn里GBDT的类库使用方法作一个总结,主要会关注调参中的一些要点. 1. scikit-learn ...
- word2vec参数调整 及lda调参
一.word2vec调参 ./word2vec -train resultbig.txt -output vectors.bin -cbow 0 -size 200 -window 5 -neg ...
- 漫谈PID——实现与调参
闲话: 作为一个控制专业的学生,说起PID,真是让我又爱又恨.甚至有时候会觉得我可能这辈子都学不会pid了,但是经过一段时间的反复琢磨,pid也不是很复杂.所以在看懂pid的基础上,写下这篇文章,方便 ...
随机推荐
- 免费开源的会计软件 GnuCash 3.4 发布
导读 GnuCash 3.4已经发布,GnuCash是免费和开源的会计软件.GnuCash开发团队宣布推出GnuCash 3.4,这是3.x稳定版系列的第五版. 变化 在3.3和3.4之间,完成了以下 ...
- windows 动态库的封装以及调用
1.一个程序从源文件编译生成可执行文件的步骤:预编译 --> 编译 --> 汇编 --> 链接(1)预编译,即预处理,主要处理在源代码文件中以“#”开始的预编译指令,如宏展开.处 ...
- Codeforces Round #467 Div. 1
B:显然即相当于能否找一条有长度为奇数的路径使得终点出度为0.如果没有环直接dp即可.有环的话可以考虑死了的spfa,由于每个点我们至多只需要让其入队两次,复杂度变成了优秀的O(kE).事实上就是拆点 ...
- 树形DP和状压DP和背包DP
树形DP和状压DP和背包DP 树形\(DP\)和状压\(DP\)虽然在\(NOIp\)中考的不多,但是仍然是一个比较常用的算法,因此学好这两个\(DP\)也是很重要的.而背包\(DP\)虽然以前考的次 ...
- FastDFS安装与使用
1. FastDFS介绍 FastDFS是一个开源的轻量级分布式文件系统,由跟踪服务器(tracker server).存储服务器(storage server)和客户端(client)三个部分组成, ...
- CODEFORCES掉RATING记 #1
时间:2017.7.16晚 比赛:Educational Codeforces Round 25 比赛开始前去睡觉了...开始后5min才起来 一进去就点开AB,B先加载好,就先做了B.读完题后发现是 ...
- 【BZOJ3522】【BZOJ4543】【POI2014】Hotel 树形DP 长链剖分 启发式合并
题目大意 给你一棵树,求有多少个组点满足\(x\neq y,x\neq z,y\neq z,dist_{x,y}=dist_{x,z}=dist_{y,z}\) \(1\leq n\leq 1 ...
- 【POJ1083】 Moving Tables (并行的搬运)
BUPT2017 wintertraining(15) #6E 题意 房间1和2,3和4,...,399和400共用一节走廊,有q次从房间li到ri的搬运桌子,一次搬运10分钟.两个搬运如果走廊有重叠 ...
- 【Luogu4707】重返现世(min-max容斥)
[Luogu4707]重返现世(min-max容斥) 题面 洛谷 求全集的\(k-max\)的期望 题解 \(min-max\)容斥的证明不难,只需要把所有元素排序之后考虑组合数的贡献,容斥系数先设出 ...
- [luogu3246][bzoj4540][HNOI2016]序列【莫队+单调栈】
题目描述 给定长度为n的序列:a1,a2,...,an,记为a[1:n].类似地,a[l:r](1<=l<=r<=N)是指序列:al,al+1,...,ar-1,ar.若1<= ...