关于one-hot encoding思考
Many learning algorithms either learn a single weight per feature, or they use distances between samples. The former is the case for linear models such as logistic regression, which are easy to explain.
Suppose you have a dataset having only a single categorical feature "nationality", with values "UK", "French" and "US". Assume, without loss of generality, that these are encoded as 0, 1 and 2. You then have a weight w for this feature in a linear classifier, which will make some kind of decision based on the constraint w×x + b > 0, or equivalently w×x < b.
The problem now is that the weight w cannot encode a three-way choice. The three possible values of w×x are 0, w and 2×w. Either these three all lead to the same decision (they're all < b or ≥b) or "UK" and "French" lead to the same decision, or "French" and "US" give the same decision. There's no possibility for the model to learn that "UK" and "US" should be given the same label, with "French" the odd one out.(二分类问题,若dummy encoding,us和uk始终不能单独成为一类,而若one-hot encoding,则可以适应任何情况)
By one-hot encoding, you effectively blow up the feature space to three features, which will each get their own weights, so the decision function is now w[UK]x[UK] + w[FR]x[FR] + w[US]x[US] < b, where all the x's are booleans. In this space, such a linear function can express any sum/disjunction of the possibilities (e.g. "UK or US", which might be a predictor for someone speaking English).
Similarly, any learner based on standard distance metrics (such as k-nearest neighbors) between samples will get confused without one-hot encoding. With the naive encoding and Euclidean distance, the distance between French and US is 1. The distance between US and UK is 2. But with the one-hot encoding, the pairwise distances between [1, 0, 0], [0, 1, 0] and [0, 0, 1] are all equal to √2.
This is not true for all learning algorithms; decision trees and derived models such as random forests, if deep enough, can handle categorical variables without one-hot encoding.
dataframe one-hot encoding:pandas.get_dummies方法
参考:
https://gist.github.com/ramhiser/982ce339d5f8c9a769a0
http://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.get_dummies.html
关于one-hot encoding思考的更多相关文章
- 关于.NET参数传递方式的思考
年关将近,整个人已经没有了工作和写作的激情,估计这个时候很多人跟我差不多,该相亲的相亲,该聚会喝酒的聚会喝酒,总之就是没有了干活的心思(我有很多想法,但就是叫不动我的手脚,所以我只能看着别人在做我想做 ...
- 关于过拟合、局部最小值、以及Poor Generalization的思考
Poor Generalization 这可能是实际中遇到的最多问题. 比如FC网络为什么效果比CNN差那么多啊,是不是陷入局部最小值啊?是不是过拟合啊?是不是欠拟合啊? 在操场跑步的时候,又从SVM ...
- Spring之LoadTimeWeaver——一个需求引发的思考---转
原文地址:http://www.myexception.cn/software-architecture-design/602651.html Spring之LoadTimeWeaver——一个需求引 ...
- 关于学习是UIWebView的一些思考
前几天因为数据中加载有html语言的数据,关于html语言和UIWebView,有一些纠结,经过几天的研究,也有了一些自己的简单的见解. 我有两个页面需要加载html语言(注意,这里 ...
- Python--Cmd窗口运行Python时提示Fatal Python error: Py_Initialize: can't initialize sys standard streams LookupError: unknown encoding: cp65001
源地址连接: http://www.tuicool.com/articles/ryuaUze 最近,我在把一个 Python 2 的视频下载工具 youku-lixian 改写成 Python 3,并 ...
- 基于纯Java代码的Spring容器和Web容器零配置的思考和实现(3) - 使用配置
经过<基于纯Java代码的Spring容器和Web容器零配置的思考和实现(1) - 数据源与事务管理>和<基于纯Java代码的Spring容器和Web容器零配置的思考和实现(2) - ...
- file.encoding到底指的是什么呢?
转载请注明来源:http://blog.csdn.net/loongshawn/article/details/50918506 <Java利用System.getProperty(“file. ...
- Java 小记 — Spring Boot 的实践与思考
前言 本篇随笔用于记录我在学习 Java 和构建 Spring Boot 项目过程中的一些思考,包含架构.组件和部署方式等.下文仅为概要,待闲时逐一整理为详细文档. 1. 组件 开源社区如火如荼,若在 ...
- Android图表库MPAndroidChart(六)——换一种思考方式,水平条形图的实现过程
Android图表库MPAndroidChart(六)--换一种思考方式,水平条形图的实现过程 一.基本实现 我们之前实现了条形图,现在来看下水平条形图是怎么实现的,说白了就是横起来,看下效果: 说起 ...
随机推荐
- 【Java_基础】并发、并行、同步、异步、多线程的区别
1. 并发:位于同一个处理器上的多个已开启未完成的线程,在任意一时刻系统调度只能让一个线程获得CPU资源运行,虽然这种调度机制有多种形式(大多数是以时间片轮巡为主).但无论如何,都是通过不断切换需要运 ...
- 优化mysql查询
mysql提供了一个特别的explain语句,用来分析查询语句的性能 : explain select ... 1.在所有用于where,order by,group by的列上添加索引 创建索引 添 ...
- linux uptime-查看Linux系统负载信息
更多linux 性能监测与优化 关注:linux命令大全 uptime命令能够打印系统总共运行了多长时间和系统的平均负载.uptime命令可以显示的信息显示依次为:现在时间.系统已经运行了多长时间.目 ...
- Linux下关于/tmp目录的清理规则
本文将介绍Linux下/tmp目录的清理规则,rhel6和rhel7将以完全不同的两种方式进行清理. RHEL6 tmpwatch命令 tmpwatch 是专门用于解决“删除 xxx天没有被访问/修改 ...
- jQuery实现Ajax
jQuery.ajax([settings]) type:类型,“POST”或“GET”,默认为GET url:发送地址 data:连同请求发送到服务器的数据 dataType:预期服务器返回的数据类 ...
- python爬虫基础05-beautifulsoup
HTML解析库BeautifulSoup4 本文链接:https://www.jianshu.com/p/e9255c446a77 BeautifulSoup 是一个可以从HTML或XML文件中提取数 ...
- 数据结构( Pyhon 语言描述 ) — —第10章:树
树的概览 树是层级式的集合 树中最顶端的节点叫做根 个或多个后继(子节点). 没有子节点的节点叫做叶子节点 拥有子节点的节点叫做内部节点 ,其子节点位于层级1,依次类推.一个空树的层级为 -1 树的术 ...
- LeetCode(125) Valid Palindrome
题目 Given a string, determine if it is a palindrome, considering only alphanumeric characters and ign ...
- PAT Basic 1071
1071 小赌怡情 常言道“小赌怡情”.这是一个很简单的小游戏:首先由计算机给出第一个整数:然后玩家下注赌第二个整数将会比第一个数大还是小:玩家下注 t 个筹码后,计算机给出第二个数.若玩家猜对了,则 ...
- 使用Lucene的java api 写入和读取索引库
import org.apache.commons.io.FileUtils;import org.apache.lucene.analysis.standard.StandardAnalyzer;i ...