This post builds on a previous post, but can be read and understood independently.

As part of my course on statistical learning, we created 3D graphics to foster a more intuitive understanding of the various methods that are used to relax the assumption of linearity (in the predictors) in regression and classification methods.

The authors of our text (The Elements of Statistical Learning, 2nd Edition) provide a Mixture Simulation data set that has two continuous predictors and a binary outcome. This data is used to demonstrate classification procedures by plotting classification boundaries in the two predictors, which are determined by one or more surfaces (e.g., a probability surface such as that produced by logistic regression, or multiple intersecting surfaces as in linear discriminant analysis). In our class laboratory, we used the R package rgl to create a 3D representation of these surfaces for a variety of semiparametric classification procedures.

Chapter 6 presents local logistic regression and kernel density classification, among other kernel (local) classification and regression methods. Below is the code and graphic (a 2D projection) associated with the local linear logistic regression in these data:

library(rgl)
load(url("http://statweb.stanford.edu/~tibs/ElemStatLearn/datasets/ESL.mixture.rda"))
dat <- ESL.mixture
ddat <- data.frame(y=dat$y, x1=dat$x[,1], x2=dat$x[,2]) ## create 3D graphic, rotate to view 2D x1/x2 projection
par3d(FOV=1,userMatrix=diag(4))
plot3d(dat$xnew[,1], dat$xnew[,2], dat$prob, type="n",
xlab="x1", ylab="x2", zlab="",
axes=FALSE, box=TRUE, aspect=1) ## plot points and bounding box
x1r <- range(dat$px1)
x2r <- range(dat$px2)
pts <- plot3d(dat$x[,1], dat$x[,2], 1,
type="p", radius=0.5, add=TRUE,
col=ifelse(dat$y, "orange", "blue"))
lns <- lines3d(x1r[c(1,2,2,1,1)], x2r[c(1,1,2,2,1)], 1) ## draw Bayes (True) classification boundary in blue
dat$probm <- with(dat, matrix(prob, length(px1), length(px2)))
dat$cls <- with(dat, contourLines(px1, px2, probm, levels=0.5))
pls0 <- lapply(dat$cls, function(p) lines3d(p$x, p$y, z=1, color="blue")) ## compute probabilities plot classification boundary
## associated with local linear logistic regression
probs.loc <-
apply(dat$xnew, 1, function(x0) {
## smoothing parameter
l <- 1/2
## compute (Gaussian) kernel weights
d <- colSums((rbind(ddat$x1, ddat$x2) - x0)^2)
k <- exp(-d/2/l^2)
## local fit at x0
fit <- suppressWarnings(glm(y ~ x1 + x2, data=ddat, weights=k,
family=binomial(link="logit")))
## predict at x0
as.numeric(predict(fit, type="response", newdata=as.data.frame(t(x0))))
}) dat$probm.loc <- with(dat, matrix(probs.loc, length(px1), length(px2)))
dat$cls.loc <- with(dat, contourLines(px1, px2, probm.loc, levels=0.5))
pls <- lapply(dat$cls.loc, function(p) lines3d(p$x, p$y, z=1)) ## plot probability surface and decision plane
sfc <- surface3d(dat$px1, dat$px2, probs.loc, alpha=1.0,
color="gray", specular="gray")
qds <- quads3d(x1r[c(1,2,2,1)], x2r[c(1,1,2,2)], 0.5, alpha=0.4,
color="gray", lit=FALSE)

In the above graphic, the solid blue line represents the true Bayes decision boundary (i.e., {x: Pr("orange"|x) = 0.5}), which is computed from the model used to simulate these data. The probability surface (generated by the local logistic regression) is represented in gray, and the corresponding Bayes decision boundary occurs where the plane f(x) = 0.5 (in light gray) intersects with the probability surface. The solid black line is a projection of this intersection. Here is a link to the interactive version of this graphic: local logistic regression.

Below is the code and graphic associated with the kernel density classification (note: this code below should only be executed after the above code, since the 3D graphic is modified, rather than created anew):

## clear the surface, decision plane, and decision boundary
pop3d(id=sfc); pop3d(id=qds)
for(pl in pls) pop3d(id=pl) ## kernel density classification
## compute kernel density estimates for each class
dens.kde <-
lapply(unique(ddat$y), function(uy) {
apply(dat$xnew, 1, function(x0) {
## subset to current class
dsub <- subset(ddat, y==uy)
## smoothing parameter
l <- 1/2
## kernel density estimate at x0
mean(dnorm(dsub$x1-x0[1], 0, l)*dnorm(dsub$x2-x0[2], 0, l))
})
}) ## compute prior for each class (sample proportion)
prir.kde <- table(ddat$y)/length(dat$y) ## compute posterior probability Pr(y=1|x)
probs.kde <- prir.kde[2]*dens.kde[[2]]/
(prir.kde[1]*dens.kde[[1]]+prir.kde[2]*dens.kde[[2]]) ## plot classification boundary associated
## with kernel density classification
dat$probm.kde <- with(dat, matrix(probs.kde, length(px1), length(px2)))
dat$cls.kde <- with(dat, contourLines(px1, px2, probm.kde, levels=0.5))
pls <- lapply(dat$cls.kde, function(p) lines3d(p$x, p$y, z=1)) ## plot probability surface and decision plane
sfc <- surface3d(dat$px1, dat$px2, probs.kde, alpha=1.0,
color="gray", specular="gray")
qds <- quads3d(x1r[c(1,2,2,1)], x2r[c(1,1,2,2)], 0.5, alpha=0.4,
color="gray", lit=FALSE)

Here are links to the interactive versions of both graphics: local logistic regressionkernel density classification

This entry was posted in Technical and tagged datagraphicsprogrammingRstatistics on February 7, 2015.

More 3D Graphics (rgl) for Classification with Local Logistic Regression and Kernel Density Estimates (from The Elements of Statistical Learning)(转)的更多相关文章

  1. Some 3D Graphics (rgl) for Classification with Splines and Logistic Regression (from The Elements of Statistical Learning)(转)

    This semester I'm teaching from Hastie, Tibshirani, and Friedman's book, The Elements of Statistical ...

  2. 李宏毅机器学习笔记3:Classification、Logistic Regression

    李宏毅老师的机器学习课程和吴恩达老师的机器学习课程都是都是ML和DL非常好的入门资料,在YouTube.网易云课堂.B站都能观看到相应的课程视频,接下来这一系列的博客我都将记录老师上课的笔记以及自己对 ...

  3. Logistic Regression Using Gradient Descent -- Binary Classification 代码实现

    1. 原理 Cost function Theta 2. Python # -*- coding:utf8 -*- import numpy as np import matplotlib.pyplo ...

  4. Classification week2: logistic regression classifier 笔记

    华盛顿大学 machine learning: Classification 笔记. linear classifier 线性分类器 多项式: Logistic regression & 概率 ...

  5. Classification and logistic regression

    logistic 回归 1.问题: 在上面讨论回归问题时.讨论的结果都是连续类型.但假设要求做分类呢?即讨论结果为离散型的值. 2.解答: 假设: 当中: g(z)的图形例如以下: 由此可知:当hθ( ...

  6. Android Programming 3D Graphics with OpenGL ES (Including Nehe's Port)

    https://www3.ntu.edu.sg/home/ehchua/programming/android/Android_3D.html

  7. Logistic Regression and Classification

    分类(Classification)与回归都属于监督学习,两者的唯一区别在于,前者要预测的输出变量\(y\)只能取离散值,而后者的输出变量是连续的.这些离散的输出变量在分类问题中通常称之为标签(Lab ...

  8. Logistic Regression求解classification问题

    classification问题和regression问题类似,区别在于y值是一个离散值,例如binary classification,y值只取0或1. 方法来自Andrew Ng的Machine ...

  9. 分类和逻辑回归(Classification and logistic regression)

    分类问题和线性回归问题问题很像,只是在分类问题中,我们预测的y值包含在一个小的离散数据集里.首先,认识一下二元分类(binary classification),在二元分类中,y的取值只能是0和1.例 ...

随机推荐

  1. Python3.5爬虫统计AcFun所有视频,并按各个类别进行Top100排序展示

    前(b)言(b): 前段时间对Python产生了浓厚的兴趣,所以决定入门学习了1个多月,后来某时我需要对tomcat做一个压力测试,于是我想到了用Python写一个压力测试的脚本吧!最后捣鼓出了一个脚 ...

  2. 开始使用ansible

    ansible是一个设计巧妙,功能强大,安全,使用简单的IT自动化运维工具.它可以实现统一配置管理,持续部署,流程编排等. 目前控制主机必须是linux,被控制主机可以是linux,类UNIX和win ...

  3. 关于sql语句between and的边界问题

    BETWEEN 操作符 操作符 BETWEEN ... AND 会选取介于两个值之间的数据范围.这些值可以是数值.文本或者日期. SQL BETWEEN 语法 SELECT column_name(s ...

  4. tolua#代码简要分析

    简介 tolua#是Unity静态绑定lua的一个解决方案,它通过C#提供的反射信息分析代码并生成包装的类.它是一个用来简化在C#中集成lua的插件,可以自动生成用于在lua中访问Unity的绑定代码 ...

  5. Java实现Android,iOS设备实时监控

    Java实现Android设备实时监控 设计思路: 第一,启动一个实时截图线程,负责实时截取Android设备屏幕,保存到本地路径. 第二,在JSP页面,定义一个img对象,实时更换img对象的src ...

  6. OC 常见面试题以及答案

    一. #improt 和include区别?@class? 1.include 主要用用来引用C 文件  import 主要引用OC 头文件  import 确保一个文件只能被导入一次  避免了交叉编 ...

  7. JS实现banner图轮换

    方法一: <!--灰色背景代码替换图片--> <!DOCTYPE html><html> <head> <meta charset="U ...

  8. Eclipse显示内存占用

  9. 基础并查集poj2236

    An earthquake takes place in Southeast Asia. The ACM (Asia Cooperated Medical team) have set up a wi ...

  10. 1018 Big Number

    Time Limit: 2000/1000 MS (Java/Others)    Memory Limit: 65536/32768 K (Java/Others)Total Submission( ...