李宏毅老师机器学习第一课Linear regression

机器学习就是让机器学会自动的找一个函数

学习图谱：

1.regression

example appliation

estimating the combat power(cp) of a pokemon after evolution.

varibles:X_cp  ，X_s  ，X_hp  ，X_w  ，X_h

model:

             f(X_cp  ，X_s  ，X_hp  ，X_w  ，X_h)=y(cp after evolution)

linear model :

              y=b+∑W_iX_i

X_i  an  attribute of input x feature             W_i   weight               b  bias

Loss function :   input a function   output  how  bad it is

        L(f)=L(b,w)=∑(y_true-(b+∑W_iX_i))²

target :  best funcation:    f*=argminL(f)

Gradient Descent:

1.randomly pick an initial value w⁰ b⁰

2.compute   ∂L ⁄∂W|_W=W0,b=b0    ∂L ⁄∂b|_{W=W⁰,b=b⁰}

3.compute   W¹=W⁰-Π( ∂L ⁄∂W|_W=W0,b=b0)     b¹=b⁰-Π( ∂L ⁄∂b|_W=W0,b=b0)

Π is called "learning rate"

linear model not exist local optimal

overfitting : a more complex model does not always lead to  better performance on testing dataLoss function Regularization:

　　       L(f)=L(b,w)=∑(y_true-(b+∑W_iX_i))²-λ∑(W_i)²

the function with smaller W_i are better

why?

if some noises corrupt input Xi when testing. A  smoother function has less influence.