Nearest-Neighbor Methods(ESL读书笔记)

Nearest-neighbor methods use those observations in the training set T closest in input space to x  form Y-hat.

Specifically, the k-nearest neighbor fit for Y-hat is difined as follows: Y(x)=1/kΣy_i,x_i belong to N_k(x).

where N_k(x) is the neighborhood of x defined by the k closest points x_i in the traing sample.

Closeness implies a metric, which for the moment we assume is Euclidean distance. So, in words, we find the k observations with x_i closest to x in input space, and average their responses.

1.Kernel methods use weights that decrease smoothly to zero with distance from the target point, rather than the effective 0/1 weights used by k-nearest  neighbors.

2.In high-dimensional spaces the distance kernels are modified to emphasize some variable more than others.

3.Local regression fits linear models by locally weighted least squares, rather than fitting constants locally.

4.Local models fit to a basis expansion of the original inputs allow arbitrarily complex models.

5.Projection pursuit and neural network models consist of sums of nonlinearly transformed linear models.