sklearn scoring . xgboost.train . ---> rsme
http://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter
3.3.1. The scoring
parameter: defining model evaluation rules
Model selection and evaluation using tools, such as model_selection.GridSearchCV
andmodel_selection.cross_val_score
, take a scoring
parameter that controls what metric they apply to the estimators evaluated.
3.3.1.1. Common cases: predefined values
For the most common use cases, you can designate a scorer object with the scoring
parameter; the table below shows all possible values. All scorer objects follow the convention that higher return values are better than lower return values. Thus metrics which measure the distance between the model and the data, like metrics.mean_squared_error
, are available as neg_mean_squared_error which return the negated value of the metric.
Scoring | Function | Comment |
---|---|---|
Classification | ||
‘accuracy’ | metrics.accuracy_score |
|
‘average_precision’ | metrics.average_precision_score |
|
‘f1’ | metrics.f1_score |
for binary targets |
‘f1_micro’ | metrics.f1_score |
micro-averaged |
‘f1_macro’ | metrics.f1_score |
macro-averaged |
‘f1_weighted’ | metrics.f1_score |
weighted average |
‘f1_samples’ | metrics.f1_score |
by multilabel sample |
‘neg_log_loss’ | metrics.log_loss |
requires predict_proba support |
‘precision’ etc. | metrics.precision_score |
suffixes apply as with ‘f1’ |
‘recall’ etc. | metrics.recall_score |
suffixes apply as with ‘f1’ |
‘roc_auc’ | metrics.roc_auc_score |
|
Clustering | ||
‘adjusted_rand_score’ | metrics.adjusted_rand_score |
|
Regression | ||
‘neg_mean_absolute_error’ | metrics.mean_absolute_error |
|
‘neg_mean_squared_error’ | metrics.mean_squared_error |
|
‘neg_median_absolute_error’ | metrics.median_absolute_error |
|
‘r2’ | metrics.r2_score |
Usage examples:
>>> from sklearn import svm, datasets
>>> from sklearn.model_selection import cross_val_score
>>> iris = datasets.load_iris()
>>> X, y = iris.data, iris.target
>>> clf = svm.SVC(probability=True, random_state=0)
>>> cross_val_score(clf, X, y, scoring='neg_log_loss')
array([-0.07..., -0.16..., -0.06...])
>>> model = svm.SVC()
>>> cross_val_score(model, X, y, scoring='wrong_choice')
Traceback (most recent call last):
ValueError: 'wrong_choice' is not a valid scoring value. Valid options are ['accuracy', 'adjusted_rand_score', 'average_precision', 'f1', 'f1_macro', 'f1_micro', 'f1_samples', 'f1_weighted', 'neg_log_loss', 'neg_mean_absolute_error', 'neg_mean_squared_error', 'neg_median_absolute_error', 'precision', 'precision_macro', 'precision_micro', 'precision_samples', 'precision_weighted', 'r2', 'recall', 'recall_macro', 'recall_micro', 'recall_samples', 'recall_weighted', 'roc_auc']
Note
The values listed by the ValueError exception correspond to the functions measuring prediction accuracy described in the following sections. The scorer objects for those functions are stored in the dictionarysklearn.metrics.SCORERS
.
3.3.1.2. Defining your scoring strategy from metric functions
The module sklearn.metric
also exposes a set of simple functions measuring a prediction error given ground truth and prediction:
- functions ending with
_score
return a value to maximize, the higher the better. - functions ending with
_error
or_loss
return a value to minimize, the lower the better. When converting into a scorer object usingmake_scorer
, set thegreater_is_better
parameter to False (True by default; see the parameter description below).
Metrics available for various machine learning tasks are detailed in sections below.
Many metrics are not given names to be used as scoring
values, sometimes because they require additional parameters, such as fbeta_score
. In such cases, you need to generate an appropriate scoring object. The simplest way to generate a callable object for scoring is by using make_scorer
. That function converts metrics into callables that can be used for model evaluation.
One typical use case is to wrap an existing metric function from the library with non-default values for its parameters, such as the beta
parameter for the fbeta_score
function:
>>> from sklearn.metrics import fbeta_score, make_scorer
>>> ftwo_scorer = make_scorer(fbeta_score, beta=2)
>>> from sklearn.model_selection import GridSearchCV
>>> from sklearn.svm import LinearSVC
>>> grid = GridSearchCV(LinearSVC(), param_grid={'C': [1, 10]}, scoring=ftwo_scorer)
sklearn scoring . xgboost.train . ---> rsme的更多相关文章
- 【集成学习】sklearn中xgboost模块的XGBClassifier函数
# 常规参数 booster gbtree 树模型做为基分类器(默认) gbliner 线性模型做为基分类器 silent silent=0时,不输出中间过程(默认) silent=1时,输出中间过程 ...
- 【集成学习】sklearn中xgboost模块中plot_importance函数(绘图--特征重要性)
直接上代码,简单 # -*- coding: utf-8 -*- """ ################################################ ...
- sklearn中xgboost模块中plot_importance函数(特征重要性)
# -*- coding: utf-8 -*- """ ######################################################### ...
- Python机器学习笔记:XgBoost算法
前言 1,Xgboost简介 Xgboost是Boosting算法的其中一种,Boosting算法的思想是将许多弱分类器集成在一起,形成一个强分类器.因为Xgboost是一种提升树模型,所以它是将许多 ...
- XGBoost和LightGBM的参数以及调参
一.XGBoost参数解释 XGBoost的参数一共分为三类: 通用参数:宏观函数控制. Booster参数:控制每一步的booster(tree/regression).booster参数一般可以调 ...
- xgboost:
https://www.zybuluo.com/Dounm/note/1031900 GBDT算法详解 http://mlnote.com/2016/10/05/a-guide-to-xgboost- ...
- XGBoost 重要参数(调参使用)
XGBoost 重要参数(调参使用) 数据比赛Kaggle,天池中最常见的就是XGBoost和LightGBM. 模型是在数据比赛中尤为重要的,但是实际上,在比赛的过程中,大部分朋友在模型上花的时间却 ...
- xgboost的遗传算法调参
遗传算法适应度的选择: 机器学习的适应度可以是任何性能指标 —准确度,精确度,召回率,F1分数等等.根据适应度值,我们选择表现最佳的父母(“适者生存”),作为幸存的种群. 交配: 存活下来的群体中的父 ...
- xgboost: 速度快效果好的boosting模型
转自:http://cos.name/2015/03/xgboost/ 本文作者:何通,SupStat Inc(总部在纽约,中国分部为北京数博思达信息科技有限公司)数据科学家,加拿大Simon Fra ...
随机推荐
- WCF Restful 服务 Get/Post请求
Restful Get方式请求: Restful服务 Get请求方式:http://localhost:10718/Service1.svc/Get/A/B/C http://localhost:1 ...
- loadsh学习
由Underscore与Lodash的差异引发的思考:http://ju.outofmemory.cn/entry/106512 JavaScript工具库之Lodash Underscore和Lod ...
- Java-Runoob:Java 修饰符
ylbtech-Java-Runoob:Java 修饰符 1.返回顶部 1. Java 修饰符 Java语言提供了很多修饰符,主要分为以下两类: 访问修饰符 非访问修饰符 修饰符用来定义类.方法或者变 ...
- docker 及 linux 初接触
3天来净折腾这俩东西了,总结一下(很多都是同事给出的解释,先不保证正确性): 1.docker 里的images 和 container images 类似系统盘,container类似用系统盘装起来 ...
- 网络监控之一:netstat命令
netstat命令用于显示与IP.TCP.UDP和ICMP协议相关的统计数据,一般用于检验本机各端口的网络连接情况.netstat是在内核中访问网络及相关信息的程序,它能提供TCP连接,TCP和UDP ...
- PHP面向对象深入研究之【高级特性】
静态属性 <?php class StaticExample { static public $aNum = 0; // 静态共有属性 static public function sayHel ...
- C语言中字符数据的输入和输出
字符的输出 C语言中使用putchar函数来输出字符数据 #include <stdio.h> int main() { char a,b,c,d; //定义字符变量a,b,c,d a = ...
- Centos7.2:搭建Ceph管理系统Inscope
0.引言 好几天没有更新博客了,这几天分配有任务:calamari与inscope管理系统调研.下面就管理系统的环境搭建做一个总结,总结一下搭建流程以及搭建过程中遇到的一些问题.calcamari的搭 ...
- Dev TreeList基本用法
public partial class treelist_shijian : DevExpress.XtraEditors.XtraForm { public treel ...
- java-虚拟机-索引
底层 JVM之堆内存(年经代,老年代) Java内存泄露的理解与解决 内存溢出和内存泄漏的区别.产生原因以及解决方案 JVM内容梳理 Jvm的体系结构