Projection Pursuit Regression----读书笔记

The central idea is to extract linear combinations of the inputs as derived features, and then model the target as a nonlinear function of these features. Assume we have an input vector X with p components, and a target Y. Let ω_m, m=1,2,...,M, be unit p-vectors of unknown parameters. The projection pursuit regression(PPR) model has the form f(X)=Σg_m(ω_m^TX). This is an additive model, but in the derived features V_m=ω_m^TX rather than the inputs themselves. The functions g_m are unspecified and are estimated along with directions ω_m using some flexible smoothing method. The scalar variable V_m=ω_m^TX is the projection of X onto the unit vector ω_m, and we seek ω_m so that the model fits well, hence the name "projection pursuit." As a result, the PPR model is most usefull for prediction, and not very usefull for producing an understandable model for the data. How do we fit a PPR model, given training data (x_i, y_i), i=1,2,...,N? We seek the approximate minimizers of the error function

Σ_i=1^N [y_i-Σ_m=1^Mg_m(ω_m^Tx_i)]².