本节主要是练习regularization项的使用原则。因为在机器学习的一些模型中,如果模型的参数太多,而训练样本又太少的话,这样训练出来的模型很容易产生过拟合现象。因此在模型的损失函数中,需要对模型的参数进行“惩罚”,这样的话这些参数就不会太大,而越小的参数说明模型越简单,越简单的模型则越不容易产生过拟合现象。

Regularized linear regression

From looking at this plot, it seems that fitting a straight line might be too simple of an approximation. Instead, we will try fitting a higher-order polynomial to the data to capture more of the variations in the points.

Let's try a fifth-order polynomial. Our hypothesis will be

This means that we have a hypothesis of six features, because  are now all features of our regression. Notice that even though we are producing a polynomial fit, we still have a linear regression problem because the hypothesis is linear in each feature.

Since we are fitting a 5th-order polynomial to a data set of only 7 points, over-fitting is likely to occur. To guard against this, we will use regularization in our model.

Recall that in regularization problems, the goal is to minimize the following cost function with respect to :

The regularization parameter is a control on your fitting parameters. As the magnitues of the fitting parameters increase, there will be an increasing penalty on the cost function. This penalty is dependent on the squares of the parameters as well as the magnitude of . Also, notice that the summation after does not include

lamda 越大,训练出的模型越简单 —— 后一项的惩罚越大

Normal equations

Now we will find the best parameters of our model using the normal equations. Recall that the normal equations solution to regularized linear regression is

The matrix following is an diagonal matrix with a zero in the upper left and ones down the other diagonal entries. (Remember that is the number of features, not counting the intecept term). The vector and the matrix have the same definition they had for unregularized regression:

Using this equation, find values for using the three regularization parameters below:

a. (this is the same case as non-regularized linear regression)

b.

c.

Code

clc,clear
%加载数据
x = load('ex5Linx.dat');
y = load('ex5Liny.dat'); %显示原始数据
plot(x,y,'o','MarkerEdgeColor','b','MarkerFaceColor','r') %将特征值变成训练样本矩阵
x = [ones(length(x),) x x.^ x.^ x.^ x.^];
[m n] = size(x);
n = n -; %计算参数sidta,并且绘制出拟合曲线
rm = diag([;ones(n,)]);%lamda后面的矩阵
lamda = [ ]';
colortype = {'g','b','r'};
sida = zeros(n+,); %初始化参数sida
xrange = linspace(min(x(:,)),max(x(:,)))';
hold on;
for i = :
sida(:,i) = inv(x'*x+lamda(i).*rm)*x'*y;%计算参数sida
norm_sida = norm(sida) % norm 求sida的2阶范数
yrange = [ones(size(xrange)) xrange xrange.^ xrange.^,...
xrange.^ xrange.^]*sida(:,i);
plot(xrange',yrange,char(colortype(i)))
hold on
end
legend('traning data', '\lambda=0', '\lambda=1','\lambda=10')%注意转义字符的使用方法
hold off

Regularization —— linear regression的更多相关文章

  1. machine learning(14) --Regularization:Regularized linear regression

    machine learning(13) --Regularization:Regularized linear regression Gradient descent without regular ...

  2. Matlab实现线性回归和逻辑回归: Linear Regression & Logistic Regression

    原文:http://blog.csdn.net/abcjennifer/article/details/7732417 本文为Maching Learning 栏目补充内容,为上几章中所提到单参数线性 ...

  3. Stanford机器学习---第二讲. 多变量线性回归 Linear Regression with multiple variable

    原文:http://blog.csdn.net/abcjennifer/article/details/7700772 本栏目(Machine learning)包括单参数的线性回归.多参数的线性回归 ...

  4. Stanford机器学习---第一讲. Linear Regression with one variable

    原文:http://blog.csdn.net/abcjennifer/article/details/7691571 本栏目(Machine learning)包括单参数的线性回归.多参数的线性回归 ...

  5. Regularized Linear Regression with scikit-learn

    Regularized Linear Regression with scikit-learn Earlier we covered Ordinary Least Squares regression ...

  6. 机器学习笔记-1 Linear Regression with Multiple Variables(week 2)

    1. Multiple Features note:X0 is equal to 1 2. Feature Scaling Idea: make sure features are on a simi ...

  7. Simple tutorial for using TensorFlow to compute a linear regression

    """Simple tutorial for using TensorFlow to compute a linear regression. Parag K. Mita ...

  8. 第五次编程作业-Regularized Linear Regression and Bias v.s. Variance

    1.正规化的线性回归 (1)代价函数 (2)梯度 linearRegCostFunction.m function [J, grad] = linearRegCostFunction(X, y, th ...

  9. [UFLDL] Linear Regression & Classification

    博客内容取材于:http://www.cnblogs.com/tornadomeet/archive/2012/06/24/2560261.html Deep learning:六(regulariz ...

随机推荐

  1. [POI2011]MET-Meteors(整体二分+树状数组)

    题意 给定一个环,每个节点有一个所属国家,k次事件,每次对[l,r]区间上的每个点点权加上一个值,求每个国家最早多少次操作之后所有点的点权和能达到一个值 题解 一个一个国家算会T.这题要用整体二分.我 ...

  2. Apache CXF实战之二 集成Sping与Web容器

    本文链接:http://blog.csdn.net/kongxx/article/details/7525481 Apache CXF实战之一 Hello World Web Service 书接上文 ...

  3. 洛谷 P1071 潜伏者

    P1071 潜伏者 题目描述 R 国和 S 国正陷入战火之中,双方都互派间谍,潜入对方内部,伺机行动.历尽艰险后,潜伏于 S 国的 R 国间谍小 C 终于摸清了 S 国军用密码的编码规则: 1. S ...

  4. HDFS 文件系统流程图。PB级文件存储时序图。

    大小文件通吃, 热点hash功能. 全局唯一KV索引. 百度网盘模式.断点续传功能.MR分析功能. 来自为知笔记(Wiz)

  5. github关联域名,创建个人站点教程终结篇

    1.背景 近期心血来潮,买了一个域名,本来要买fuckgfw的,没想到被抢注了.就拿自己的英文名买了一个.叫做www.garvinli.com.由于知道会有非常多朋友不太清楚整个站点搭建的过程,所以在 ...

  6. IDEA中如何设置自动导包

    IDEA跟eclipse还是有一些差别,一些东西要自己去设置,但同时也还是有快捷键的方式来帮助我们 1.如何设置自动导包:如下图所示 点击FIle--->settings 其次还可以通过按快捷键 ...

  7. Gitblit从一个服务器,迁移到另外一个服务器

    http://gitblit.com/federation.html A Gitblit federation is a mechanism to clone repositories and kee ...

  8. Entity Framework之Code First开发方式

    一.Code First Code First方式只需要代码,不需要Edmx模型.EF通过实体类型结构推断生成SQL并创建数据库中的表.开发人员只需要编写实体类就可以进行EF数据库的开发. Code ...

  9. 分享一个vueui axios-mock-adapter 中的用法

    import axios from 'axios'; import MockAdapter from 'axios-mock-adapter'; import { LoginUsers, Users ...

  10. OpenSUSE Leap 42.3 安装java(Oracle jre)

    一.安装1.从Oracle官网下载jre-8u161-linux-x64.rpm安装包(下载与浏览器位数一样) 2.安装: openSUSE:/home/pipci # rpm -ivh jre-8u ...