Author: 相忠良(Zhong-Liang Xiang)

Email: ugoood@163.com

Date: Sep. 23st, 2017

根据 Andrew Ng 老师的深度学习课程课后作业及指导,参照吴老师代码完成了这个LR的coding.

(重要)吴老师建议,数据应组织成下列形式,有利于扫除编程bug:

  • X.shape = (n_x, m), n_x是样本维度,m是样本个数
  • Y.shape = (1, m)
  • w, b应该分开,其中:
    • b is a scaler
    • w.shape = (n_x, 1)
  • A = sigmoid(np.dot(w.T, X)+b), A.shape = (1, m)
  • dw.shape = (n_x, 1)
  • db is a scaler
  • dZ = A - Y, dZ.shape = A.shape = Y.shape = (1, m)
  • 重要建议:
    1. 勇于使用 reshape, 使之成为我们需要的维度, 要始终使用明确维度的行、列向量和 matrix;
    2. 绝不使用 a = np.random.randn(5), a.shape = (5,)这种"rank 1 array".因为这东西使用时不符合直觉;
    3. 应该用 a = np.random.randn(5,1) 或者 (1,5) 这种非常明确的列或行向量(very important)!
    4. 若出现2所示内容,解决办法是:a = a.reshape(5,1) 或者 (1,5)重新明确shape!;
    5. 要经常并随意使用 assert(a.shape == (5,1)) 这种断言;
    6. 要仔细检查我们的 matrix, vector的维度.

自己的总结:
1. 先完成推导,明确输入输出以及哪些变量是已知的,哪些是待求的.
2. 写出程序伪代码.
3. 针对伪代码,逐条完成程序的 vectorize 过程. 这时要小心地,自输入开始地,维护好各种 vector, matrix 的维度, 必要时随需求,毫不犹豫地使用 reshape.
4. 上述第3条保证了程序中尽量地少使用 for loop.
5. 遵从 Andrew Ng 老师的上述建议,尤其是对 X, Y, A, w, b, dw, db, dZ 这些 vector, matrix 们的 shape 的把握.

符合上述规则和自己的总结,编出个机器学习算法就很简单了.

我整合吴老师的课后作业,加了少许修改,做出 Logisitc Regression 算法的代码, 如下:

# !/usr/bin/python
# -*- coding:utf-8 -*- """
Re-implement Logistic Regression algorithm as a practice
使用该 LR re-implement 的前提:
Due to the binary classifier of LR
The label of a sample must be as probability
train data 的标签必须转成0,1的形式
""" # Author: 相忠良(Zhong-Liang Xiang) <ugoood@163.com>
# Finished on September 23rd, 2017 import h5py
import numpy as np def load_dataset():
train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r")
train_set_x_orig = np.array(train_dataset["train_set_x"][:]) # your train set features
train_set_y_orig = np.array(train_dataset["train_set_y"][:]) # your train set labels test_dataset = h5py.File('datasets/test_catvnoncat.h5', "r")
test_set_x_orig = np.array(test_dataset["test_set_x"][:]) # your test set features
test_set_y_orig = np.array(test_dataset["test_set_y"][:]) # your test set labels classes = np.array(test_dataset["list_classes"][:]) # the list of classes train_set_y_orig = train_set_y_orig.reshape((1, train_set_y_orig.shape[0]))
test_set_y_orig = test_set_y_orig.reshape((1, test_set_y_orig.shape[0])) return train_set_x_orig, train_set_y_orig, test_set_x_orig, test_set_y_orig, classes def load_data():
train_set_x_orig, train_set_y_orig, test_set_x_orig, test_set_y_orig, classes = load_dataset()
train_X = (train_set_x_orig.reshape(train_set_x_orig.shape[0], -1).T) / 255. # flatten and divide 255
test_X = (test_set_x_orig.reshape(test_set_x_orig.shape[0], -1).T) / 255. # flatten and divide 255
return train_X, train_set_y_orig, test_X, test_set_y_orig, classes def sigmoid(z):
"""
Compute the sigmoid of z Arguments:
z -- A scalar or numpy array of any size. Return:
s -- sigmoid(z)
""" s = 1.0 / (1 + np.exp(-z))
return s def init_with_zeros(dim):
"""
This function creates a vector of zeros of shape (dim, 1) for w and initializes b to 0. Argument:
dim -- size of the w vector we want (or number of parameters in this case) Returns:
w -- initialized vector of shape (dim, 1)
b -- initialized scalar (corresponds to the bias)
""" w = np.zeros((dim, 1))
b = 0
assert (w.shape == (dim, 1))
assert (isinstance(b, float) or isinstance(b, int))
return w, b def propagate(w, b, X, Y):
"""
Implement the cost function and its gradient for the propagation Arguments:
w -- weights, a numpy array of size (num_px * num_px * 3, 1)
b -- bias, a scalar
X -- data of size (num_px * num_px * 3, number of examples)
Y -- true "label" vector (containing 0 if non-cat, 1 if cat) of size (1, number of examples) Return:
cost -- negative log-likelihood cost for logistic regression
dw -- gradient of the loss with respect to w, thus same shape as w
db -- gradient of the loss with respect to b, thus same shape as b
""" # FORWARD PROPAGATION(FROM X TO COST)
m = X.shape[1] # 样本个数
A = sigmoid(np.dot(w.T, X) + b) # activation (1 * m)
cost = (np.dot(np.log(A), Y.T) + np.dot(np.log(1 - A), (1 - Y).T)) / -m # a scaler # BACKWARD PROPAGATION (TO FIND GRAD)
dZ = A - Y # (1 * m)
dw = np.dot(X, dZ.T) / m # (n_x, 1) n_x 是 样本的维度
db = np.sum(dZ) / m # a scaler # ASSERT
assert (dw.shape == w.shape)
assert (db.dtype == float) cost = np.squeeze(cost) # 变成个数字
assert (cost.shape == ()) grads = {"dw": dw,
"db": db} return grads, cost def optimize(w, b, X, Y, num_iterations, learning_rate, print_cost):
"""
This function optimizes w and b by running a gradient descent algorithm Arguments:
w -- weights, a numpy array of size (num_px * num_px * 3, 1)
b -- bias, a scalar
X -- data of shape (num_px * num_px * 3, number of examples)
Y -- true "label" vector (containing 0 if non-cat, 1 if cat), of shape (1, number of examples)
num_iterations -- number of iterations of the optimization loop
learning_rate -- learning rate of the gradient descent update rule
print_cost -- True to print the loss every 100 steps Returns:
params -- dictionary containing the weights w and bias b
grads -- dictionary containing the gradients of the weights and bias with respect to the cost function
costs -- list of all the costs computed during the optimization, this will be used to plot the learning curve.
""" costs = [] # 将迭代过程中算出的cost收集起来 for i in range(num_iterations): # Cost and gradient calculation
grads, cost = propagate(w, b, X, Y) # Retrieve derivatives from grads
dw = grads["dw"]
db = grads["db"] # update dw, db
w = w - learning_rate * dw
b = b - learning_rate * db # Record the costs
if i % 100 == 0:
costs.append(cost) # Print the cost every 100 iterations
if print_cost and i % 100 == 0:
print("Cost after iteration %i: %f" % (i, cost)) params = {"w": w,
"b": b}
grads = {"dw": dw,
"db": db}
return params, grads, costs class MyLogisticRegression:
costs = []
params = {} # w, b
grads = {} # dw, db
num_iterations = 0
learning_rate = 0.
print_cost = False def __init__(self, num_iterations=1000, learning_rate=0.01, print_cost=False):
# 初始化超參數 num_iterations, learning_rate
self.num_iterations = num_iterations
self.learning_rate = learning_rate
self.print_cost = print_cost
return def fit(self, X, Y):
n_x = X.shape[0] # dim of X
w, b = init_with_zeros(n_x) # initialize w,b with zeros, w.shape=(n_x, 1), b=0 a scaler. # 前向传播获取cost,反向传播获取grads,并更新params.这种事情做了num_iterations次,学习率为learning_rate
self.params, self.grads, self.costs = optimize(w, b, X, Y, self.num_iterations, self.learning_rate,
self.print_cost)
# fit函数的结果是获取params.顺便得到了grads, costs, 便于我们查看并对costs画图,以检查模型是否学到了东西. def predict(self, X):
m = X.shape[1] # the number of samples
Y_predict = np.zeros((1, m)) # initialize Y_predict
w = self.params["w"] # 获取已经训练好的 w
b = self.params["b"] # 获取已经训练好的 b
A = sigmoid(np.dot(w.T, X) + b) # 根据 训练好的w,b,计算 p(Y=1|X) # 将预测概率p(Y=1|X)转换为标签值, 大于0.5的标签值为1,否则为0
for i in range(A.shape[1]):
Y_predict[0, i] = 1 if A[0, i] > 0.5 else 0 assert (Y_predict.shape == (1, m))
return Y_predict def score(self, X, y):
pass ## 测试用例
train_X, train_y, test_X, test_y, classes = load_data()
cls = MyLogisticRegression(num_iterations=2000, learning_rate=0.005, print_cost=True) cls.fit(train_X, train_y) Y_predict_test = cls.predict(test_X)
Y_predict_train = cls.predict(train_X) print("train accuracy: {} %".format(100 - np.mean(np.abs(Y_predict_train - train_y)) * 100))
print("test accuracy: {} %".format(100 - np.mean(np.abs(Y_predict_test - test_y)) * 100)) """
运行结果 /usr/bin/python2.7 /home/xiang/桌面/ML_Course_20170314/xiang_code/Xiang_ml_in_practice/MyLogisticRegression.py
Cost after iteration 0: 0.693147
Cost after iteration 100: 0.584508
Cost after iteration 200: 0.466949
Cost after iteration 300: 0.376007
Cost after iteration 400: 0.331463
Cost after iteration 500: 0.303273
Cost after iteration 600: 0.279880
Cost after iteration 700: 0.260042
Cost after iteration 800: 0.242941
Cost after iteration 900: 0.228004
Cost after iteration 1000: 0.214820
Cost after iteration 1100: 0.203078
Cost after iteration 1200: 0.192544
Cost after iteration 1300: 0.183033
Cost after iteration 1400: 0.174399
Cost after iteration 1500: 0.166521
Cost after iteration 1600: 0.159305
Cost after iteration 1700: 0.152667
Cost after iteration 1800: 0.146542
Cost after iteration 1900: 0.140872
train accuracy: 99.043062201 %
test accuracy: 70.0 %
"""

Logistic Regression 算法向量化实现及心得的更多相关文章

  1. 学习Logistic Regression的笔记与理解(转)

    学习Logistic Regression的笔记与理解 1.首先从结果往前来看下how logistic regression make predictions. 设我们某个测试数据为X(x0,x1, ...

  2. Neural Networks and Deep Learning(week2)Logistic Regression with a Neural Network mindset(实现一个图像识别算法)

    Logistic Regression with a Neural Network mindset You will learn to: Build the general architecture ...

  3. 逻辑回归(Logistic Regression)算法小结

    一.逻辑回归简述: 回顾线性回归算法,对于给定的一些n维特征(x1,x2,x3,......xn),我们想通过对这些特征进行加权求和汇总的方法来描绘出事物的最终运算结果.从而衍生出我们线性回归的计算公 ...

  4. 机器学习算法与Python实践之(七)逻辑回归(Logistic Regression)

    http://blog.csdn.net/zouxy09/article/details/20319673 机器学习算法与Python实践之(七)逻辑回归(Logistic Regression) z ...

  5. 【算法】Logistic regression (逻辑回归) 概述

    Logistic regression (逻辑回归)是当前业界比较常用的机器学习方法,用于估计某种事物的可能性.比如某用户购买某商品的可能性,某病人患有某种疾病的可能性,以及某广告被用户点击的可能性等 ...

  6. 分类算法之逻辑回归(Logistic Regression

    分类算法之逻辑回归(Logistic Regression) 1.二分类问题 现在有一家医院,想要对病人的病情进行分析,其中有一项就是关于良性\恶性肿瘤的判断,现在有一批数据集是关于肿瘤大小的,任务就 ...

  7. 通俗地说逻辑回归【Logistic regression】算法(二)sklearn逻辑回归实战

    前情提要: 通俗地说逻辑回归[Logistic regression]算法(一) 逻辑回归模型原理介绍 上一篇主要介绍了逻辑回归中,相对理论化的知识,这次主要是对上篇做一点点补充,以及介绍sklear ...

  8. 机器学习---三种线性算法的比较(线性回归,感知机,逻辑回归)(Machine Learning Linear Regression Perceptron Logistic Regression Comparison)

    最小二乘线性回归,感知机,逻辑回归的比较:   最小二乘线性回归 Least Squares Linear Regression 感知机 Perceptron 二分类逻辑回归 Binary Logis ...

  9. Python机器学习算法 — 逻辑回归(Logistic Regression)

    逻辑回归--简介 逻辑回归(Logistic Regression)就是这样的一个过程:面对一个回归或者分类问题,建立代价函数,然后通过优化方法迭代求解出最优的模型参数,然后测试验证我们这个求解的模型 ...

随机推荐

  1. 【第二十篇】C#微信H5支付 非微信内浏览器H5支付 浏览器微信支付

    微信开发者文档 微信H5支付官方文档   请阅读清楚  最起码把所有参数看一遍 这个地方也可以看看 微信案例 http://wxpay.wxutil.com/mch/pay/h5.v2.php,请在微 ...

  2. VCS使用学习笔记(1)——Verilog相关的仿真知识

    本文主要学习Verilog的仿真特性,以及仿真器对Verilog的处理,算是对Verilog知识的增量学习.本文内容与我的另一篇博文(http://www.cnblogs.com/IClearner/ ...

  3. python打包压缩文件夹zip+组装文件夹

    无意间想到的一个需求,然后就顺手写了写,留下来,方便以后用 列表版:(基本没用,仅提供思路,字典版稍微改动可以直接用) 大体需求: 把重复的文件名进行改名,达到浏览器下载相同文件的效果 下载完成后再把 ...

  4. vue中的数据监听以及数据交互

    现在我们来看一下vue中的数据监听事件$watch, js代码: new Vue({ el:"#div", data:{ arr:[,,] } }).$watch("ar ...

  5. ZOJ-1649 Rescue---BFS+优先队列

    题目链接: https://vjudge.net/problem/ZOJ-1649 题目大意: 天使的朋友要去救天使,a是天使,r 是朋友,x是卫兵.每走一步需要时间1,打倒卫兵需要另外的时间1,问救 ...

  6. 03、NetCore2.0下Web应用之搭建最小框架

    03.NetCore2.0下Web应用之搭建最小框架 这里我们不使用VS2017或者CLI命令的方式创建Asp.Net Core 2.0网页应用程序,而是完全手工的一点点搭建一个Web框架,以便更好的 ...

  7. Python3NumPy——数组(1)之创建

    开篇 numpy库作为科学计算的基础库,其地位相当重要,它是对数组操作的基石.它的存在使得线性代数以及矩阵论等相关知识在计算机上的表达更加方便与简单,集中体现出了人想办法,计算机去工作. Python ...

  8. libevent源码阅读笔记(一):libevent对epoll的封装

    title: libevent源码阅读笔记(一):libevent对epoll的封装 最近开始阅读网络库libevent的源码,阅读源码之前,大致看了张亮写的几篇博文(libevent源码深度剖析 h ...

  9. 使Asp.net Core同时支持输出Json/Xml

    我们知道Asp.net Core是支持输出为Json格式的.同时也支持输出为xml格式.只要我们正确的配置.并在Request时指定正确的Accept,即可根据不同的Header来输出不同的格式. 前 ...

  10. Java基础详解

    从写Java系列的第一篇到现在已经三个月了,因为在网络上或书籍中没有见到一些很适合初学者的学习流程,所以下决心自己写一写,也当作回顾一下Java的知识.网上有许多Java教程之类的内容,都是从概念起步 ...