R语言与概率统计(六) 主成分分析 因子分析

超高维度分析，N*P的矩阵，N为样本个数，P为指标，N<<P

PCA:抓住对y对重要的影响因素

主要有三种：PCA,因子分析，回归方程+惩罚函数（如LASSO）

为了降维，用更少的变量解决问题，如果是二维的，那么就是找到一条线，要使这些点再线上的投影最大，投影最大，就是越分散，就考虑方差最大。

> conomy<-data.frame(
+   x1=c(149.3, 161.2, 171.5, 175.5, 180.8, 190.7,
+        202.1, 212.4, 226.1, 231.9, 239.0),
+   x2=c(4.2, 4.1, 3.1, 3.1, 1.1, 2.2, 2.1, 5.6, 5.0, 5.1, 0.7),
+   x3=c(108.1, 114.8, 123.2, 126.9, 132.1, 137.7,
+        146.0, 154.1, 162.3, 164.3, 167.6),
+   y=c(15.9, 16.4, 19.0, 19.1, 18.8, 20.4, 22.7,
+       26.5, 28.1, 27.6, 26.3)
+ )
> #### 作线性回归
> lm.sol<-lm(y~x1+x2+x3, data=conomy)
> summary(lm.sol)

Call:
lm(formula = y ~ x1 + x2 + x3, data = conomy)

Residuals:
     Min       1Q   Median       3Q      Max
-0.52367 -0.38953  0.05424  0.22644  0.78313 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)
(Intercept) -10.12799    1.21216  -8.355  6.9e-05 ***
x1           -0.05140    0.07028  -0.731 0.488344
x2            0.58695    0.09462   6.203 0.000444 ***
x3            0.28685    0.10221   2.807 0.026277 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4889 on 7 degrees of freedom
Multiple R-squared:  0.9919,	Adjusted R-squared:  0.9884
F-statistic: 285.6 on 3 and 7 DF,  p-value: 1.112e-07

> #### 作主成分分析
> conomy.pr<-princomp(~x1+x2+x3, data=conomy, cor=T)
> summary(conomy.pr, loadings=TRUE)
Importance of components:
                         Comp.1    Comp.2       Comp.3
Standard deviation     1.413915 0.9990767 0.0518737839
Proportion of Variance 0.666385 0.3327181 0.0008969632
Cumulative Proportion  0.666385 0.9991030 1.0000000000

Loadings:
   Comp.1 Comp.2 Comp.3
x1  0.706         0.707
x2        -0.999
x3  0.707        -0.707
> #### 预测测样本主成分, 并作主成分分析
> pre<-predict(conomy.pr)
> conomy$z1<-pre[,1]
> conomy$z2<-pre[,2]
> lm.sol<-lm(y~z1+z2, data=conomy)
> summary(lm.sol)

Call:
lm(formula = y ~ z1 + z2, data = conomy)

Residuals:
     Min       1Q   Median       3Q      Max
-0.89838 -0.26050  0.08435  0.35677  0.66863 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept)  21.8909     0.1658 132.006 1.21e-14 ***
z1            2.9892     0.1173  25.486 6.02e-09 ***
z2           -0.8288     0.1660  -4.993  0.00106 **
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.55 on 8 degrees of freedom
Multiple R-squared:  0.9883,	Adjusted R-squared:  0.9853
F-statistic: 337.2 on 2 and 8 DF,  p-value: 1.888e-08

> #### 作变换, 得到原坐标下的关系表达式
> beta<-coef(lm.sol); A<-loadings(conomy.pr)
> x.bar<-conomy.pr$center; x.sd<-conomy.pr$scale
> coef<-(beta[2]*A[,1]+ beta[3]*A[,2])/x.sd
> beta0 <- beta[1]- sum(x.bar * coef)
> c(beta0, coef)
(Intercept)          x1          x2          x3
-9.13010782  0.07277981  0.60922012  0.10625939