Normal Equation

Note: [8:00 to 8:44 - The design matrix X (in the bottom right side of the slide) given in the example should have elements x with subscript 1 and superscripts varying from 1 to m because for all m training sets there are only 2 features x0 and x1. 12:56 - The X matrix is m by (n+1) and NOT n by n. ]

Gradient descent gives one way of minimizing J. Let’s discuss a second way of doing so, this time performing the minimization explicitly and without resorting to an iterative algorithm. In the "Normal Equation" method, we will minimize J by explicitly taking its derivatives with respect to the θj ’s, and setting them to zero. This allows us to find the optimum theta without iteration. The normal equation formula is given below:

There is no need to do feature scaling with the normal equation.

The following is a comparison of gradient descent and the normal equation:

With the normal equation, computing the inversion has complexity  So if we have a very large number of features, the normal equation will be slow. In practice, when n exceeds 10,000 it might be a good time to go from a normal solution to an iterative process.

Normal Equation Noninvertibility

When implementing the normal equation in octave we want to use the 'pinv' function rather than 'inv.' The 'pinv' function will give you a value of θ even if  is not invertible.

If  is noninvertible, the common causes might be having :

  • Redundant features, where two features are very closely related (i.e. they are linearly dependent)
  • Too many features (e.g. m ≤ n). In this case, delete some features or use "regularization" (to be explained in a later lesson).

Solutions to the above problems include deleting a feature that is linearly dependent with another or deleting one or more features when there are too many features.

Normal Equation of Computing Parameters Analytically的更多相关文章

  1. Linear regression with multiple variables(多特征的线型回归)算法实例_梯度下降解法(Gradient DesentMulti)以及正规方程解法(Normal Equation)

    ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, ,, , ...

  2. 5种方法推导Normal Equation

    引言: Normal Equation 是最基础的最小二乘方法.在Andrew Ng的课程中给出了矩阵推到形式,本文将重点提供几种推导方式以便于全方位帮助Machine Learning用户学习. N ...

  3. coursera机器学习笔记-多元线性回归,normal equation

    #对coursera上Andrew Ng老师开的机器学习课程的笔记和心得: #注:此笔记是我自己认为本节课里比较重要.难理解或容易忘记的内容并做了些补充,并非是课堂详细笔记和要点: #标记为<补 ...

  4. (三)用Normal Equation拟合Liner Regression模型

    继续考虑Liner Regression的问题,把它写成如下的矩阵形式,然后即可得到θ的Normal Equation. Normal Equation: θ=(XTX)-1XTy 当X可逆时,(XT ...

  5. 【转】Derivation of the Normal Equation for linear regression

    I was going through the Coursera "Machine Learning" course, and in the section on multivar ...

  6. 机器学习入门:Linear Regression与Normal Equation -2017年8月23日22:11:50

    本文会讲到: (1)另一种线性回归方法:Normal Equation: (2)Gradient Descent与Normal Equation的优缺点:   前面我们通过Gradient Desce ...

  7. Normal Equation

    一.Normal Equation 我们知道梯度下降在求解最优参数\(\theta\)过程中需要合适的\(\alpha\),并且需要进行多次迭代,那么有没有经过简单的数学计算就得到参数\(\theta ...

  8. CS229 3.用Normal Equation拟合Liner Regression模型

    继续考虑Liner Regression的问题,把它写成如下的矩阵形式,然后即可得到θ的Normal Equation. Normal Equation: θ=(XTX)-1XTy 当X可逆时,(XT ...

  9. 正规方程 Normal Equation

    正规方程 Normal Equation 前几篇博客介绍了一些梯度下降的有用技巧,特征缩放(详见http://blog.csdn.net/u012328159/article/details/5103 ...

随机推荐

  1. 【Codeforces Round #426 (Div. 2) B】The Festive Evening

    [Link]:http://codeforces.com/contest/834/problem/B [Description] [Solution] 模拟水题; 注意一个字母单个出现的时候,结束和开 ...

  2. netstat -p 显示 -

    http://4735839.blog.51cto.com/4725839/1418945 https://yq.aliyun.com/articles/63060

  3. hive load文件第一个字段为NULL

    在hive中,通常须要载入外部数据源.load文件时.第一个字段会出现NULL. 比如: 1.运行load语句: LOAD DATA LOCAL INPATH 'test.txt' OVERWRITE ...

  4. [Angular] Create a custom validator for template driven forms in Angular

    User input validation is a core part of creating proper HTML forms. Form validators not only help yo ...

  5. STL_算法_查找算法(search、find_end)

    C++ Primer 学习中. .. 简单记录下我的学习过程 (代码为主) search          //从左往右找第一个符合条件的子区间    全部容器适用 find_end  //从右往左找 ...

  6. 【Android】利用安卓的数据接口、多媒体处理编写内存卡Mp3播放器app

    通过调用安卓的MediaPlayer能够直接完毕Mp3等主流音频的播放,同一时候利用ContentResolver与Cursor能够直接读取安卓内在数据库的信息.直接获取当前sdcard中全部音频的列 ...

  7. VBA调试利器debug.print

    作者:iamlaosong 百度一下.非常easy找到debug.print解释和使用介绍.事实上非常简单.就是将代码运行结果显示在"马上窗体"中,但不影响程序运行.VBA程序调试 ...

  8. RelativeLayout-代码中成员控件width height

    今天需要在代码中动态的设置一个textview的width跟height属性,记录下来. textview在xml中的布局如下 <RelativeLayout android:layout_wi ...

  9. Android学习笔记进阶15之Shader渲染

    Android提供的Shader类主要是渲染图像以及一些几何图形. Shader有几个直接子类: BitmapShader    : 主要用来渲染图像 LinearGradient  :用来进行线性渲 ...

  10. HTML5手机应用的最大优势就是可以在网页上直接调试和修改

    HTML5手机应用的最大优势就是可以在网页上直接调试和修改