1. Variable definitions

m : training examples' count

\(X\) : design matrix. each row of \(X\) is a training example, each column of \(X\) is a feature

\[X =
\begin{pmatrix}
1 & x^{(1)}_1 & ... & x^{(1)}_n \\
1 & x^{(2)}_1 & ... & x^{(2)}_n \\
... & ... & ... & ... \\
1 & x^{(n)}_1 & ... & x^{(n)}_n \\
\end{pmatrix}\]

\[\theta =
\begin{pmatrix}
\theta_0 \\
\theta_1 \\
... \\
\theta_n \\
\end{pmatrix}\]

2. Hypothesis

\[x=
\begin{pmatrix}
x_0 \\
x_1 \\
... \\
x_n \\
\end{pmatrix}
\]

\[h_\theta(x) = g(\theta^T x) = g(x_0\theta_0 + x_1\theta_1 + ... + x_n\theta_n) = \frac{1}{1 + e^{(-\theta^Tx)}},
\]

sigmoid function

\[g(z) = \frac{1}{1 + e^{-z}},
\]

g = 1 ./ (1 + e .^ (-z));

3. Cost function

\[J(\theta) = \frac{1}{m}\sum_{i=1}^m[-y^{(i)}log(h_\theta(x^{(i)})) - (1-y^{(i)})log(1 - h_\theta(x^{(i)}))],
\]

vectorization edition of Octave

J = -(1 / m) * sum(y' * log(sigmoid(X * theta)) + (1 - y)' * log(1 - sigmoid(X * theta)));

4. Goal

find \(\theta\) to minimize \(J(\theta)\), \(\theta\) is a vector here

4.1 Gradient descent

\[\frac{\partial J(\theta)}{\partial \theta_j} = \frac{1}{m} \sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})x^{(i)}_j,
\]

repeat until convergence{

     \(\theta_j := \theta_j - \alpha \sum_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)}) x^{(i)}_j\)

}

vectorization

\(S\)

\[=
\begin{pmatrix}
h_\theta(x^{(1)})-y^{(1)} & h_\theta(x^{(2)})-y^{(2)} & ... & h_\theta(x^{(n)}-y^{(n)})
\end{pmatrix}
\begin{pmatrix}
x^{(1)}_0 & x^{(1)}_1 & ... & x^{(1)}_3 \\
x^{(2)}_0 & x^{(2)}_1 & ... & x^{(2)}_3 \\
... & ... & ... & ... \\
x^{(n)}_0 & x^{(n)}_1 & ... & x^{(n)}_3 \\
\end{pmatrix}
\]

\[=
\begin{pmatrix}
\sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})x^{(i)}_0 &
\sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})x^{(i)}_1 &
... &
\sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})x^{(i)}_n
\end{pmatrix}
\]

\[\theta = \theta - S^T
\]

\[h_\theta(X) = g(X\theta) = \frac{1}{1 + e^{(-X\theta)}}
\]

\(X\theta\) is nx1, \(y\) is nx1

\(\frac{1}{1+e^{(-X\theta)}} - y\) is nx1

\[\frac{1}{1 + e^{(-X\theta)}} - y=
\begin{pmatrix}
h_\theta(x^{(1)})-y^{(1)} & h_\theta(x^{(2)})-y^{(2)} & ... & h_\theta(x^{(n)})-y^{(n)}
\end{pmatrix}
\]

\[\theta = \theta - \alpha(\frac{1}{1 + e^{(-X\theta)}} - y)X
\]

5. Regularized logistic regression

to avoid overfitting or underfitting

Cost function

\[J(\theta) = \frac{1}{m}\sum_{i=1}^m[-y^{(i)}log(h_\theta(x^{(i)})) - (1-y^{(i)})log(1 - h_\theta(x^{(i)}))] + \frac{\lambda}{2m} \sum_{j=1}^m \theta^2_j,
\]

Gradient descent

\[\frac{\partial J(\theta)}{\partial \theta_0} = \frac{1}{m} \sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})x^{(i)}_0,
\]

\[\frac{\partial J(\theta)}{\partial \theta_j} = \frac{1}{m} \sum_{i=1}^m(h_\theta(x^{(i)}) - y^{(i)})x^{(i)}_j, (j \ge 1)
\]

[Machine learning] Logistic regression的更多相关文章

  1. 机器学习---逻辑回归(二)(Machine Learning Logistic Regression II)

    在<机器学习---逻辑回归(一)(Machine Learning Logistic Regression I)>一文中,我们讨论了如何用逻辑回归解决二分类问题以及逻辑回归算法的本质.现在 ...

  2. 机器学习---逻辑回归(一)(Machine Learning Logistic Regression I)

    逻辑回归(Logistic Regression)是一种经典的线性分类算法.逻辑回归虽然叫回归,但是其模型是用来分类的. 让我们先从最简单的二分类问题开始.给定特征向量x=([x1,x2,...,xn ...

  3. Machine Learning—Linear Regression

    Evernote的同步分享:Machine Learning-Linear Regression 版权声明:本文博客原创文章.博客,未经同意,不得转载.

  4. 机器学习---三种线性算法的比较(线性回归,感知机,逻辑回归)(Machine Learning Linear Regression Perceptron Logistic Regression Comparison)

    最小二乘线性回归,感知机,逻辑回归的比较:   最小二乘线性回归 Least Squares Linear Regression 感知机 Perceptron 二分类逻辑回归 Binary Logis ...

  5. [Machine Learning] logistic函数和softmax函数

    简单总结一下机器学习最常见的两个函数,一个是logistic函数,另一个是softmax函数,若有不足之处,希望大家可以帮忙指正.本文首先分别介绍logistic函数和softmax函数的定义和应用, ...

  6. 机器学习---线性回归(Machine Learning Linear Regression)

    线性回归是机器学习中最基础的模型,掌握了线性回归模型,有利于以后更容易地理解其它复杂的模型. 线性回归看似简单,但是其中包含了线性代数,微积分,概率等诸多方面的知识.让我们先从最简单的形式开始. 一元 ...

  7. [Machine Learning] Linear regression

    1. Variable definitions m : training examples' count \(y\) : \(X\) : design matrix. each row of \(X\ ...

  8. How do I learn machine learning?

    https://www.quora.com/How-do-I-learn-machine-learning-1?redirected_qid=6578644   How Can I Learn X? ...

  9. 机器学习---最小二乘线性回归模型的5个基本假设(Machine Learning Least Squares Linear Regression Assumptions)

    在之前的文章<机器学习---线性回归(Machine Learning Linear Regression)>中说到,使用最小二乘回归模型需要满足一些假设条件.但是这些假设条件却往往是人们 ...

随机推荐

  1. 模板汇总——ST(暂)

    int Log[N]; struct ST { ], a[N]; void init(int n) { ]=-); i < N; i++) Log[i] = Log[i - ] + ((i &a ...

  2. 淘淘购物系统 (Python)

    #首页def tao_first(): t1 = '欢迎进入淘淘购物'.center(110) print(t1) print('~' * 130) t2 = '注册'.center(20) prin ...

  3. A - 猜数字

    http://acm.hdu.edu.cn/showproblem.php?pid=1172 猜数字 猜数字游戏是gameboy最喜欢的游戏之一.游戏的规则是这样的:计算机随机产生一个四位数,然后玩家 ...

  4. 数论 Day 13

    数论_CRT(中国剩余定理)& Lucas (卢卡斯定理) 前言 又是一脸懵逼的一天. 正文 按照道理来说,我们应该先做一个介绍. 中国剩余定理 中国剩余定理,Chinese Remainde ...

  5. 在eclipse中引入mybatis和spring的约束文件

    eclipse中引入mybatis约束文件步骤: 首先: config的key值 http://mybatis.org/dtd/mybatis-3-config.dtd mapper的key值 htt ...

  6. 【原创】(四)Linux内存模型之Sparse Memory Model

    背景 Read the fucking source code! --By 鲁迅 A picture is worth a thousand words. --By 高尔基 说明: Kernel版本: ...

  7. Android 网络通信框架Volley(三)

    NetworkImageView 分析:public class NetworkImageView extends ImageView 他继承自ImageView,并且添加了一个新方法: public ...

  8. Java中各种引用(Reference)解析

    目录 1,引用类型 2, FinalReference 2.1, Finalizer 3, SoftReference 4, WeakReference 5, PhantomReference 6, ...

  9. Dart函数、类和运算符-处理信息

    编程语言虽然千差万别,但归根结底,它们的设计思想无非就是回答两个问题: 1.如何表示信息: 2.如何处理信息: 函数 函数是一段用来独立地完成某个功能的代码.函数是对象类型,它的类型叫做Functio ...

  10. 【linux】【NodeJs】Centos7安装node-v10.16.3环境

    前言 Node.js 是一个基于 Chrome V8 引擎的 JavaScript 运行环境. Node.js 使用了一个事件驱动.非阻塞式 I/O 的模型,使其轻量又高效. https://node ...