深度学习

So far this week

  • Edge detection
  • RANSAC
  • SIFT
  • K-Means
  • Linear classifier
  • Mean-shift
  • PCA/Eigenfaces
  • Image features

Current Research

  • Learning hierarchical representations from data
  • End-to-end learning: raw inputs to predictions
  • can use a small set of simple tools to solve many problems
  • has led to rapid progress on many problems
  • Inspired by the brain(very loosely!)

Deep learning for different problems

vision tasks

  • visual recognition



  • object detection: what and where

  • object segmentation

  • image caption

  • visual question answering

  • super resolution

  • image retrieval

  • style transfer

outside vision tasks

  • Machine Translation
  • Text Synthesis
  • Speech Recognition
  • Speech Synthesis

Motivation

Data-driven approach:

  1. collect a dataset of images and labels
  2. use machine learning to train an image calssifier
  3. evaluate the classifier on a withheld set of test images

feature learning

what is feature learning?[^what is feature learning]

deep learning

Supervised learning

linear regression

neural network

neural networks with many layers

Gradient descent

how to find the best weights \(w^T\)

which way is down hill?

gradient descent

fancier rules:

  • Momentum
  • NAG
  • Adagrad
  • Adadelta
  • Rmsprop



这里以后可以再 看看!

Backpropagation

a two-layer neural network in 25 lines of code

import numpy as np
D,H,N = 8, 64,32
#randomly initialize weights
W1 = np.random.randn(D,H)
W2 = np.random.randn(H,D)
for t in xrange(10000):
x = np.random.randn(N,D)
y = np.sin(x)
s = x.dot(W1)
a = np.maxium(s,0)
y_hat = a.dot(W2)
loss = 0.5*np.sum((y_hat-y)**2.0)
dy_hat = y_hat - y
dW2 = a.T.dot(W2.T)
da = dy_hat.dot(W2.T)
ds = (s > 0)*da
dW1 = x.T.dot(ds)
W1 -= learning_rate*dW1
W2 -= learning_rate*dW2

[^what is feature learning]:

In Machine Learning, feature learning or representation learningis a set of techniques that learn a feature: a transformation of raw data input to a representation that can be effectively exploited in machine learning tasks. This obviates manual feature engineering, which is otherwise necessary, and allows a machine to both learn at a specific task (using the features) and learn the features themselves.

Feature learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process. However, real-world data such as images, video, and sensor measurement is usually complex, redundant, and highly variable. Thus, it is necessary to discover useful features or representations from raw data. Traditional hand-crafted features often require expensive human labor and often rely on expert knowledge. Also, they normally do not generalize well. This motivates the design of efficient feature learning techniques, to automate and generalize this.

Feature learning can be divided into two categories: supervised and unsupervised feature learning, analogous to these categories in machine learning generally.

In supervised feature learning, features are learned with labeled input data. Examples include Supervised Neural Networks, Multilayer Perceptron, and (supervised) dictionary Learning.

In unsupervised feature learning, features are learned with unlabeled input data. Examples include dictionary learning, independent component analysis, autoencoders, and various forms of clustering.

[学习笔记] CS131 Computer Vision: Foundations and Applications:Lecture 9 深度学习的更多相关文章

  1. [学习笔记] CS131 Computer Vision: Foundations and Applications:Lecture 1 课程介绍

    课程大纲:http://vision.stanford.edu/teaching/cs131_fall1718/syllabus.html 课程定位: 课程交叉: what is (computer) ...

  2. [学习笔记] CS131 Computer Vision: Foundations and Applications:Lecture 2 颜色和数学基础

    大纲 what is color? The result of interaction between physical light in the environment and our visual ...

  3. [学习笔记] CS131 Computer Vision: Foundations and Applications:Lecture 4 像素和滤波器

    Background reading: Forsyth and Ponce, Computer Vision Chapter 7 Image sampling and quantization Typ ...

  4. [学习笔记] CS131 Computer Vision: Foundations and Applications:Lecture 3 线性代数初步

    向量和矩阵 什么是矩阵/向量? Vectors and matrix are just collections of ordered numbers that represent something: ...

  5. Computer Vision: Algorithms and ApplicationsのImage processing

    实在是太喜欢Richard Szeliski的这本书了.每一章节(after chapter3)都详述了该研究方向比較新的成果.还有很多很多的reference,假设你感兴趣.全然能够看那些參考论文 ...

  6. Deep Learning 10_深度学习UFLDL教程:Convolution and Pooling_exercise(斯坦福大学深度学习教程)

    前言 理论知识:UFLDL教程和http://www.cnblogs.com/tornadomeet/archive/2013/04/09/3009830.html 实验环境:win7, matlab ...

  7. Sony深度学习框架 - Neural Network Console - 教程(1)- 原来深度学习可以如此简单

    “什么情况!?居然不是黑色背景+白色文字的命令行.对,今天要介绍的是一个拥有白嫩的用户界面的深度学习框架.” 人工智能.神经网络.深度学习,这些概念近年已经涌入每个人的生活中,我想很多人早就按捺不住想 ...

  8. 百度DMLC分布式深度机器学习开源项目(简称“深盟”)上线了如xgboost(速度快效果好的Boosting模型)、CXXNET(极致的C++深度学习库)、Minerva(高效灵活的并行深度学习引擎)以及Parameter Server(一小时训练600T数据)等产品,在语音识别、OCR识别、人脸识别以及计算效率提升上发布了多个成熟产品。

    百度为何开源深度机器学习平台?   有一系列领先优势的百度却选择开源其深度机器学习平台,为何交底自己的核心技术?深思之下,却是在面对业界无奈时的远见之举.   5月20日,百度在github上开源了其 ...

  9. Python入门学习笔记4:他人的博客及他人的学习思路

    看其他人的学习笔记,可以保证自己不走弯路.并且一举两得,即学知识又学方法! 廖雪峰:https://www.liaoxuefeng.com/wiki/0014316089557264a6b348958 ...

随机推荐

  1. php多维数组的指定单个字段排序

    多维数组如何根据指定键值?比如现在有数组结构如下: ,,,,,'subject' => 'math'), 1 => array('name' => '3班','avgScore'=& ...

  2. [SDOI2016]数字配对(费用流+贪心+trick)

    重点是如何找到可以配对的\(a[i]\)和\(a[j]\). 把\(a[i]\)分解质因数.设\(a[i]\)分解出的质因数的数量为\(cnt[i]\). 设\(a[i]\geq a[j]\) 那么\ ...

  3. CentOS 笔记(五) 常用工具

    远程 :XShell6  ,PuTTy FPT:Xfpt ,pscp.exe

  4. 00074_Array类

    1.Array类的概述 此类包含用来操作数组(比如排序和搜索)的各种方法.需要注意,如果指定数组引用为 null,则访问此类中的方法都会抛出空指针异常NullPointerException. 2.常 ...

  5. mysql修改配置文件

    在Apache, PHP, MySQL的体系架构中,MySQL对于性能的影响最大,也是关键的核心部分.对于Discuz!论坛程序也是如此,MySQL的设置是否合理优化,直接影响到论坛的速度和承载量!同 ...

  6. C#--二维码生成代码

    需要用到ThoughtWorks.QRCode.dll文件 string str = Server.UrlDecode(Request.QueryString["id"]); QR ...

  7. java cocurrent包

    1. java.util.concurrent - Java 并发工具包 Java 5 添加了一个新的包到 Java 平台,java.util.concurrent 包.这个包包含有一系列能够让 Ja ...

  8. [SharePoint][SharePoint Designer 入门经典]Chapter13 客户端Silverlight编程

    1.使用Silverlight,CAML和Linq取得数据 2.编程性创建更新删除列表数据项 3.修饰列表和库的配置 4.管理文件和文件夹 5.修改快速启动和顶部导航条 [使用Silverlight, ...

  9. iOS绘图系统UIKit与Core Graphics

    概述 iOS主要的绘图系统有UIKit,Core Graphics,Core Animation,Core Image,Open GL等,本片博文主要介绍UIKit与Core Graphics的绘图系 ...

  10. tensorflow利用预训练模型进行目标检测(二):预训练模型的使用

    一.运行样例 官网链接:https://github.com/tensorflow/models/blob/master/research/object_detection/object_detect ...