MATLAB实例：PCA降维

MATLAB实例：PCA降维

作者：凯鲁嘎吉 - 博客园 http://www.cnblogs.com/kailugaji/

1. iris数据

5.1,3.5,1.4,0.2,1
4.9,3.0,1.4,0.2,1
4.7,3.2,1.3,0.2,1
4.6,3.1,1.5,0.2,1
5.0,3.6,1.4,0.2,1
5.4,3.9,1.7,0.4,1
4.6,3.4,1.4,0.3,1
5.0,3.4,1.5,0.2,1
4.4,2.9,1.4,0.2,1
4.9,3.1,1.5,0.1,1
5.4,3.7,1.5,0.2,1
4.8,3.4,1.6,0.2,1
4.8,3.0,1.4,0.1,1
4.3,3.0,1.1,0.1,1
5.8,4.0,1.2,0.2,1
5.7,4.4,1.5,0.4,1
5.4,3.9,1.3,0.4,1
5.1,3.5,1.4,0.3,1
5.7,3.8,1.7,0.3,1
5.1,3.8,1.5,0.3,1
5.4,3.4,1.7,0.2,1
5.1,3.7,1.5,0.4,1
4.6,3.6,1.0,0.2,1
5.1,3.3,1.7,0.5,1
4.8,3.4,1.9,0.2,1
5.0,3.0,1.6,0.2,1
5.0,3.4,1.6,0.4,1
5.2,3.5,1.5,0.2,1
5.2,3.4,1.4,0.2,1
4.7,3.2,1.6,0.2,1
4.8,3.1,1.6,0.2,1
5.4,3.4,1.5,0.4,1
5.2,4.1,1.5,0.1,1
5.5,4.2,1.4,0.2,1
4.9,3.1,1.5,0.1,1
5.0,3.2,1.2,0.2,1
5.5,3.5,1.3,0.2,1
4.9,3.1,1.5,0.1,1
4.4,3.0,1.3,0.2,1
5.1,3.4,1.5,0.2,1
5.0,3.5,1.3,0.3,1
4.5,2.3,1.3,0.3,1
4.4,3.2,1.3,0.2,1
5.0,3.5,1.6,0.6,1
5.1,3.8,1.9,0.4,1
4.8,3.0,1.4,0.3,1
5.1,3.8,1.6,0.2,1
4.6,3.2,1.4,0.2,1
5.3,3.7,1.5,0.2,1
5.0,3.3,1.4,0.2,1
7.0,3.2,4.7,1.4,2
6.4,3.2,4.5,1.5,2
6.9,3.1,4.9,1.5,2
5.5,2.3,4.0,1.3,2
6.5,2.8,4.6,1.5,2
5.7,2.8,4.5,1.3,2
6.3,3.3,4.7,1.6,2
4.9,2.4,3.3,1.0,2
6.6,2.9,4.6,1.3,2
5.2,2.7,3.9,1.4,2
5.0,2.0,3.5,1.0,2
5.9,3.0,4.2,1.5,2
6.0,2.2,4.0,1.0,2
6.1,2.9,4.7,1.4,2
5.6,2.9,3.6,1.3,2
6.7,3.1,4.4,1.4,2
5.6,3.0,4.5,1.5,2
5.8,2.7,4.1,1.0,2
6.2,2.2,4.5,1.5,2
5.6,2.5,3.9,1.1,2
5.9,3.2,4.8,1.8,2
6.1,2.8,4.0,1.3,2
6.3,2.5,4.9,1.5,2
6.1,2.8,4.7,1.2,2
6.4,2.9,4.3,1.3,2
6.6,3.0,4.4,1.4,2
6.8,2.8,4.8,1.4,2
6.7,3.0,5.0,1.7,2
6.0,2.9,4.5,1.5,2
5.7,2.6,3.5,1.0,2
5.5,2.4,3.8,1.1,2
5.5,2.4,3.7,1.0,2
5.8,2.7,3.9,1.2,2
6.0,2.7,5.1,1.6,2
5.4,3.0,4.5,1.5,2
6.0,3.4,4.5,1.6,2
6.7,3.1,4.7,1.5,2
6.3,2.3,4.4,1.3,2
5.6,3.0,4.1,1.3,2
5.5,2.5,4.0,1.3,2
5.5,2.6,4.4,1.2,2
6.1,3.0,4.6,1.4,2
5.8,2.6,4.0,1.2,2
5.0,2.3,3.3,1.0,2
5.6,2.7,4.2,1.3,2
5.7,3.0,4.2,1.2,2
5.7,2.9,4.2,1.3,2
6.2,2.9,4.3,1.3,2
5.1,2.5,3.0,1.1,2
5.7,2.8,4.1,1.3,2
6.3,3.3,6.0,2.5,3
5.8,2.7,5.1,1.9,3
7.1,3.0,5.9,2.1,3
6.3,2.9,5.6,1.8,3
6.5,3.0,5.8,2.2,3
7.6,3.0,6.6,2.1,3
4.9,2.5,4.5,1.7,3
7.3,2.9,6.3,1.8,3
6.7,2.5,5.8,1.8,3
7.2,3.6,6.1,2.5,3
6.5,3.2,5.1,2.0,3
6.4,2.7,5.3,1.9,3
6.8,3.0,5.5,2.1,3
5.7,2.5,5.0,2.0,3
5.8,2.8,5.1,2.4,3
6.4,3.2,5.3,2.3,3
6.5,3.0,5.5,1.8,3
7.7,3.8,6.7,2.2,3
7.7,2.6,6.9,2.3,3
6.0,2.2,5.0,1.5,3
6.9,3.2,5.7,2.3,3
5.6,2.8,4.9,2.0,3
7.7,2.8,6.7,2.0,3
6.3,2.7,4.9,1.8,3
6.7,3.3,5.7,2.1,3
7.2,3.2,6.0,1.8,3
6.2,2.8,4.8,1.8,3
6.1,3.0,4.9,1.8,3
6.4,2.8,5.6,2.1,3
7.2,3.0,5.8,1.6,3
7.4,2.8,6.1,1.9,3
7.9,3.8,6.4,2.0,3
6.4,2.8,5.6,2.2,3
6.3,2.8,5.1,1.5,3
6.1,2.6,5.6,1.4,3
7.7,3.0,6.1,2.3,3
6.3,3.4,5.6,2.4,3
6.4,3.1,5.5,1.8,3
6.0,3.0,4.8,1.8,3
6.9,3.1,5.4,2.1,3
6.7,3.1,5.6,2.4,3
6.9,3.1,5.1,2.3,3
5.8,2.7,5.1,1.9,3
6.8,3.2,5.9,2.3,3
6.7,3.3,5.7,2.5,3
6.7,3.0,5.2,2.3,3
6.3,2.5,5.0,1.9,3
6.5,3.0,5.2,2.0,3
6.2,3.4,5.4,2.3,3
5.9,3.0,5.1,1.8,3

2. MATLAB程序

function [COEFF,SCORE,latent,tsquared,explained,mu,data_PCA]=pca_demo()
x=load('iris.data');
[~,d]=size(x);
k=d-1; %前k个主成分
x=zscore(x(:,1:d-1));  %归一化数据
[COEFF,SCORE,latent,tsquared,explained,mu]=pca(x);
% 1）获取样本数据 X ，样本为行，特征为列。
% 2）对样本数据中心化，得S（S = X的各列减去各列的均值）。
% 3）求 S 的协方差矩阵 C = cov(S)
% 4) 对协方差矩阵 C 进行特征分解 [P,Lambda] = eig(C);
% 5）结束。
% 1、输入参数 X 是一个 n 行 p 列的矩阵。每行代表一个样本观察数据，每列则代表一个属性，或特征。
% 2、COEFF 就是所需要的特征向量组成的矩阵，是一个 p 行 p 列的矩阵，没列表示一个出成分向量，经常也称为（协方差矩阵的）特征向量。并且是按照对应特征值降序排列的。所以，如果只需要前 k 个主成分向量，可通过：COEFF(:,1:k) 来获得。
% 3、SCORE 表示原数据在各主成分向量上的投影。但注意：是原数据经过中心化后在主成分向量上的投影。即通过：SCORE = x0*COEFF 求得。其中 x0 是中心平移后的 X（注意：是对维度进行中心平移，而非样本。），因此在重建时，就需要加上这个平均值了。
% 4、latent 是一个列向量，表示特征值，并且按降序排列。
% 5、tsquared Hotelling的每个观测值X的T平方统计量
% 6、explained 由每个主成分解释的总方差的百分比
% 7、mu 每个变量X的估计平均值
% x= bsxfun(@minus,x,mean(x,1));
data_PCA=x*COEFF(:,1:k);
latent1=100*latent/sum(latent);%将latent总和统一为100，便于观察贡献率
pareto(latent1);%调用matla画图 pareto仅绘制累积分布的前95%，因此y中的部分元素并未显示
xlabel('Principal Component');
ylabel('Variance Explained (%)');
% 图中的线表示的累积变量解释程度
print(gcf,'-dpng','Iris PCA.png');
iris_pac=data_PCA(:,1:2) ;
save iris_pca iris_pac

3. 结果

iris_pca：前两个主成分

-2.25698063306803	0.504015404227653
-2.07945911889541	-0.653216393612590
-2.36004408158421	-0.317413944570283
-2.29650366000389	-0.573446612971233
-2.38080158645275	0.672514410791076
-2.06362347633724	1.51347826673567
-2.43754533573242	0.0743137171331950
-2.22638326740708	0.246787171742162
-2.33413809644009	-1.09148977019584
-2.18136796941948	-0.447131117450110
-2.15626287481026	1.06702095645556
-2.31960685513084	0.158057945820095
-2.21665671559727	-0.706750478104682
-2.63090249246321	-0.935149145374822
-2.18497164997156	1.88366804891533
-2.24394778052703	2.71328133141014
-2.19539570001472	1.50869601039751
-2.18286635818774	0.512587093716441
-1.88775015418968	1.42633236069007
-2.33213619695782	1.15416686250116
-1.90816386828207	0.429027879924458
-2.19728429051438	0.949277150423224
-2.76490709741649	0.487882574439700
-1.81433337754274	0.106394361814184
-2.22077768737273	0.161644638073716
-1.95048968523510	-0.605862870440206
-2.04521166172712	0.265126114804279
-2.16095425532709	0.550173363315497
-2.13315967968331	0.335516397664229
-2.26121491382610	-0.313827252316662
-2.13739396044139	-0.482326258880086
-1.82582143036022	0.443780130732953
-2.59949431958629	1.82237008322707
-2.42981076672382	2.17809479520796
-2.18136796941948	-0.447131117450110
-2.20373717203888	-0.183722323644913
-2.03759040170113	0.682669420156327
-2.18136796941948	-0.447131117450110
-2.42781878392261	-0.879223932713649
-2.16329994558551	0.291749566745466
-2.27889273592867	0.466429134628597
-1.86545776627869	-2.31991965918865
-2.54929404704891	-0.452301129580194
-1.95772074352968	0.495730895348582
-2.12624969840005	1.16752080832811
-2.06842816583668	-0.689607099127106
-2.37330741591874	1.14679073709691
-2.39018434748641	-0.361180775489047
-2.21934619663183	1.02205856145225
-2.19858869176329	0.0321302060908945
1.10030752013391	0.860230593245533
0.730035752246062	0.596636784545418
1.23796221659453	0.612769614333371
0.395980710562889	-1.75229858398514
1.06901265623960	-0.211050862633647
0.383174475987114	-0.589088965722193
0.746215185580377	0.776098608766709
-0.496201068006129	-1.84269556949638
0.923129796737431	0.0302295549588077
0.00495143780650871	-1.02596403732389
-0.124281108093219	-2.64918765259090
0.437265238506424	-0.0586846858581760
0.549792126592992	-1.76666307900171
0.714770518429262	-0.184815166484382
-0.0371339806719297	-0.431350035919633
0.872966018474250	0.508295314415273
0.346844440799832	-0.189985178614466
0.152880381053472	-0.788085297090142
1.21124542423444	-1.62790202112846
0.156417163578196	-1.29875232891050
0.735791135537219	0.401126570248885
0.470792483676532	-0.415217206131680
1.22388807504403	-0.937773165086814
0.627279600231826	-0.415419947028686
0.698133985336190	-0.0632819273014206
0.870620328215835	0.249871517845242
1.25003445866275	-0.0823442389434431
1.35370481019450	0.327722365822153
0.659915359649250	-0.223597000167979
-0.0471236447211597	-1.05368247816741
0.121128417400412	-1.55837168956507
0.0140710866007487	-1.56813894313840
0.235222818975321	-0.773333046281646
1.05316323317206	-0.634774729305402
0.220677797156699	-0.279909968621073
0.430341476713787	0.852281697154445
1.04590946111265	0.520453696157683
1.03241950881290	-1.38781716762055
0.0668436673617666	-0.211910813930204
0.274505447436587	-1.32537578085168
0.271425764670620	-1.11570381243558
0.621089830946741	0.0274506709978046
0.328903506457842	-0.985598883763833
-0.372380114621411	-2.01119457605980
0.281999617970590	-0.851099454545845
0.0887557702224096	-0.174324544331148
0.223607676665854	-0.379214256409087
0.571967341693057	-0.153206717308028
-0.455486948803962	-1.53432438068788
0.251402252309636	-0.593871222060355
1.84150338645482	0.868786147264828
1.14933941416981	-0.698984450845645
2.19898270027627	0.552618780551384
1.43388176486790	-0.0498435417617587
1.86165398830779	0.290220535935809
2.74500070081969	0.785799704159685
0.357177895625210	-1.55488557249365
2.29531637451915	0.408149356863061
1.99505169024551	-0.721448439846371
2.25998344407884	1.91502747107928
1.36134878398531	0.691631011499905
1.59372545693795	-0.426818952656741
1.87796051113409	0.412949339203311
1.24890257443547	-1.16349352357816
1.45917315700813	-0.442664601834978
1.58649439864337	0.674774813132046
1.46636772102851	0.252347085727036
2.42924030093571	2.54822056527013
3.29809226641255	-0.00235343587272177
1.24979406018816	-1.71184899071237
2.03368323142868	0.904369044486726
0.970663302005081	-0.569267277965818
2.88838806680663	0.396463170625287
1.32475563655861	-0.485135293486995
1.69855040646181	1.01076227706927
1.95119099025002	0.999984474306318
1.16799162725452	-0.317831851008113
1.01637609822602	0.0653241212065782
1.78004554289349	-0.192627479858818
1.85855159177699	0.553527164026207
2.42736549094542	0.245830911619345
2.30834922706014	2.61741528404554
1.85415981777379	-0.184055790370030
1.10756129219332	-0.294997832217552
1.19347091639304	-0.814439294423699
2.79159729280499	0.841927657717863
1.57487925633390	1.06889360300461
1.34254676764379	0.420846092290459
0.920349720485088	0.0191661621187343
1.84736314547313	0.670177571688802
2.00942543830962	0.608358978317639
1.89676252747561	0.683734258412757
1.14933941416981	-0.698984450845645
2.03648602144585	0.861797777652503
1.99500750598298	1.04504903502442
1.86427657131500	0.381543630923962
1.55328823048458	-0.902290843047121
1.51576710303099	0.265903772450991
1.37179554779330	1.01296839034343
0.956095566421630	-0.0222095406309480

累计贡献率

可见：前两个主成分已经占了95%的贡献程度。这两个主成分可以近似表示整个数据。

4. pca_data.m

其中normlization.m见MATLAB实例：聚类初始化方法与数据归一化方法

function data=pca_data(data, choose)
% PCA降维，保留90%的特征信息
data = normlization(data, choose); %归一化
score = 0.90; %保留90%的特征信息
[num,dim] = size(data);
xbar = mean(data,1);
means = bsxfun(@minus, data, xbar);
cov = means'*means/num;
[V,D] = eig(cov);
eigval = diag(D);
[~,idx] = sort(eigval,'descend');
eigval = eigval(idx);
V = V(idx,:);
p = 0;
for i=1:dim
   perc = sum(eigval(1:i))/sum(eigval);
   if perc > score
       p = i;
       break;
   end
end
E = V(1:p,:);
data= means*E';

参考：

Junhao Hua. Distributed Variational Bayesian Algorithms. Github, 2017.

MATLAB实例：PCA（主成成分分析）详解