DEEP LEARNING WITH STRUCTURE


Charlie Tang is a PhD student in the Machine Learning group at the University of Toronto, working with Geoffrey Hinton and Ruslan Salakhutdinov, whose research interests include machine learning, computer vision and cognitive science. More specifically, he has developed various higher-order extensions to generative models in deep learning for vision.

At the Deep Learning Summit in Boston next month, Charlie will present 'Deep Learning with Structure'. Supervised neural networks trained on massive datasets have recently achieved impressive performance in computer vision, speech recognition, and many other tasks. While extremely flexible, neural nets are often criticized because their internal representations are distributed codes and lack interpretability; during his presentation at the summit, Charlie will reveal how we can address some of these concerns.

We had a quick Q&A with Charlie ahead of the Deep Learning Summit, to hear more of his thoughts on developments and challenges in deep learning.

What are the key factors that have enabled recent advancements in deep learning? 
The three key factors are:
- The steadfast belief and knowledge that supervised neural networks trained with enough labelled data can achieve great test set generalization.
- The availability of high performance hardware and software, in particular, Nvidia's CUDA architecture and SDK. This allowed more experimentation and the learning from large-scale data.
- The development of superior models: switching to rectified linear hidden units from the sigmoid or hyperbolic tangent units and the invention of regularization techniques, specifically "Dropout".

What are the main types of problems now being addressed in the deep learning space? 
Almost all problems in statistical machine learning are currently being investigated using deep learning techniques. They include visual and speech recognition, reinforcement learning, natural language processing, medical and health applications, financial engineering and many others.

What are the practical applications of your work and what sectors are most likely to be affected?
The deep learning revolution allows models trained on big data to drastically improve accuracy. This means that many artificial intelligence recognition tasks can be now automated, which previously necessitated a human in-the-loop.

What developments can we expect to see in deep learning in the next 5 years?
Deep learning algorithms will be gradually adopted for more tasks and will "solve" more problems. For example, 5 years ago, algorithmic face recognition accuracy was still somewhat worse than human performance. However, currently, super-human performances are reported on the main face recognition dataset (LFW) and the standard image classification dataset (Imagenet). In the next 5 years, harder and harder problems such as video recognition, medical imaging or text processing will be successfully tackled by deep learning algorithms. We can also expect deep learning algorithms to be ported to commercial products, much like how the face detector was incorporated into consumer cameras in the past 10 years.

What advancements excite you most in the field?
I feel like the most exciting advance is the availability of low-energy mobile hardware that supports deep learning algorithms. This will inevitably lead to many real-time systems and mobile products which will be a part of our daily lives.

The Deep Learning Summit is taking place in Boston on 26-27 May. For more information and to register, please visit the event website here.

Join the conversation with the event hashtag #reworkDL

DEEP LEARNING WITH STRUCTURE的更多相关文章

  1. Can deep learning help you find the perfect girl?

    Can deep learning help you find the perfect girl? One of the first things I did when I moved to Mont ...

  2. (转) Awesome Deep Learning

    Awesome Deep Learning  Table of Contents Free Online Books Courses Videos and Lectures Papers Tutori ...

  3. (转) The major advancements in Deep Learning in 2016

    The major advancements in Deep Learning in 2016 Pablo Tue, Dec 6, 2016 in MACHINE LEARNING DEEP LEAR ...

  4. (转) Deep Learning Research Review Week 2: Reinforcement Learning

      Deep Learning Research Review Week 2: Reinforcement Learning 转载自: https://adeshpande3.github.io/ad ...

  5. (转)Deep Learning Research Review Week 1: Generative Adversarial Nets

    Adit Deshpande CS Undergrad at UCLA ('19) Blog About Resume Deep Learning Research Review Week 1: Ge ...

  6. (转) Deep Learning in a Nutshell: Core Concepts

    Deep Learning in a Nutshell: Core Concepts Share:   Posted on November 3, 2015by Tim Dettmers 7 Comm ...

  7. (转)The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)

    Adit Deshpande CS Undergrad at UCLA ('19) Blog About The 9 Deep Learning Papers You Need To Know Abo ...

  8. Applied Deep Learning Resources

    Applied Deep Learning Resources A collection of research articles, blog posts, slides and code snipp ...

  9. Machine and Deep Learning with Python

    Machine and Deep Learning with Python Education Tutorials and courses Supervised learning superstiti ...

随机推荐

  1. C#字符格式化占位符

    using System; using System.Diagnostics; using System.Text; using System.Collections; using System.Co ...

  2. windows 7 安装 scrapy

    基于64位 win7 系统 先到 http://www.lfd.uci.edu/~gohlke/pythonlibs/ 下载四个 wheel 文件: 1. lxml-3.4.4-cp27-none-w ...

  3. 利用Maven管理工程项目本地启动报错及解决方案

    目前利用Maven工具来构建自己的项目已比较常见.今天主要不是介绍Maven工具,而是当你本地启动这样的服务时,如果遇到报错,该如何解决?下面只是参考的解决方案,具体的解法还是得看log的信息. 1. ...

  4. LeetCode:Clone Graph

    题目如下:实现克隆图的算法  题目链接 Clone an undirected graph. Each node in the graph contains a label and a list of ...

  5. Linux第六次学习笔记

    存储器层次结构 存储器系统是一个具有不同容量.成本和访问时间的存储设备的层次结构. CPU寄存器保存着最常用的数据. 主存储器(简称主存)暂时存放存储在容量较大的.慢速磁盘上的数据. 高速缓存存储器作 ...

  6. RocEDU.阅读.写作《你的灯亮着吗?》

    <你的灯亮着吗?> 一.对本书的认识 这本书的作者就如何训练思维能力指点迷津.书中提及的观点包括"问题是理想状态和现实状态之间的差别",以及"无论表面上表现的 ...

  7. git的简介,安装以及使用

    1git的简介 Git是什么? Git是目前世界上最先进的分布式版本控制系统(没有之一). Git有什么特点?简单来说就是:高端大气上档次! 2Linus一直痛恨的CVS及SVN都是集中式的版本控制系 ...

  8. jquery实现文件异步上传

    前言 这里用了2个JS插件,一个是Jquery原生js,我的版本是jquery-1.7.2.min.js,另一个是jquery.form.js.这个form.js 是关键,不可少哦.另外, 我的服务器 ...

  9. 20145222黄亚奇《Java程序设计》实验一实验报告

    实验一 Java开发环境的熟悉(Linux+Eclipse) 实验内容及步骤 使用JDK编译.运行简单的Java程序 在NetBeans IDEA中输入如下代码: package ljp; publi ...

  10. javascript中prototype、constructor以及__proto__之间的三角关系

    三者暧昧关系简单整理 在javascript中,prototype.constructor以及__proto__之间有着“著名”的剪不断理还乱的三角关系,楼主就着自己对它们的浅显认识,来粗略地理理以备 ...