Combining STDP and Reward-Modulated STDP in Deep Convolutional Spiking Neural Networks for Digit Recognition

郑重声明:原文参见标题,如有侵权,请联系作者,将会撤销发布! Abstract 灵长类视觉系统激发了深度人工神经网络的发展,使计算机视觉领域发生了革命性的变化.然而,这些网络的能量效率比它们的生物学对应体要低得多,而且它们通常使用反向传播进行训练,这是非常需要数据的.为了解决这些限制,我们使用了深度卷积脉冲神经网络(DCSNN)和延迟编码方案.我们将最低层的脉冲时序依赖可塑性(STDP)和最高层的奖励调节STDP(R-STDP)结合起来训练.简而言之,在R-STDP中,正确(错误)决策导致STD…

论文笔记之：UNSUPERVISED REPRESENTATION LEARNING WITH DEEP CONVOLUTIONAL GENERATIVE ADVERSARIAL NETWORKS

UNSUPERVISED REPRESENTATION LEARNING WITH DEEP CONVOLUTIONAL GENERATIVE ADVERSARIAL NETWORKS ICLR 2016 摘要:近年来 CNN 在监督学习领域的巨大成功和无监督学习领域的无人问津形成了鲜明的对比,本文旨在链接上这两者之间的缺口.提出了一种 deep convolutional generative adversarial networks (DCGANs),that have certai…

Coursera, Deep Learning 1, Neural Networks and Deep Learning - week4, Deep Neural Networks

Deep Neural Network Getting your matrix dimention right 选hyper-pamameter 完全是凭经验补充阅读: cost 函数的计算公式: 求导公式…

Coursera, Deep Learning 1, Neural Networks and Deep Learning - week1, Introduction to deep learning

整个deep learing 系列课程主要包括哪些内容 Intro to Deep learning…

Coursera, Deep Learning 1, Neural Networks and Deep Learning - week3, Neural Networks Basics

NN representation 这一课主要是讲3层神经网络下面是常见的 activation 函数.sigmoid, tanh, ReLU, leaky ReLU. Sigmoid 只用在输出0/1 时候的output layer, 其他情况基本不用,因为tanh 总是比sigmoid 好. 两种 ReLU 使用起来总是要比sigmoid 和 tanh 快.ReLU 是最常用的 activation. 为什么Activation function 要是non-linear的?因为如下图所示…

Coursera, Deep Learning 1, Neural Networks and Deep Learning - week2, Neural Networks Basics

Logistic regression Cost function for logistic regression Gradient Descent 接下来主要讲 Vectorization Logistic Regression 的向量实现 Vectorizing LR Gradient output Python/Numpy and Jupyter Notebook 上图中 axis=0 表示竖直方向,axis=1 是水平方向…

ReLU——Deep Sparse Rectifier Neural Networks

1. 摘要 ReLU 相比 Tanh 能产生相同或者更好的性能,而且能产生真零的稀疏表示,非常适合自然就稀疏的数据. 采用 ReLU 后,在大量的有标签数据下,有没有无监督预训练模型取得的最好效果是一样的,这可以被看做是训练深层有监督网络的一个新的里程碑. 2. 背景 2.1. 神经元科学的观察对脑能量消耗的研究表明,神经元以稀疏和分布的方式编码信息,同时活跃的神经元的百分比估计在 1% 到 4% 之间.这是信息表示丰富度和潜在能量消耗的一种平衡.但是,如果没有额外的正则化,比如 \(L_1\…

PVANET: Deep but Lightweight Neural Networks for Real-time Object Detection

URL: https://arxiv.org/abs/1608.08021 year: 2016 TL;DR PVANet 一个轻量级多物体目标检测架构, 遵循 "less channels with more layers" 的设计原则, 通过结合 CReLU, Inception, HyperNet 3 个模块构成了一个高效的目标检测架构, 在达到了当时 SOTA. CReLU 应用于PVANet 早期阶段, 以将计算次数减少一半而不会失去准确性. Inception 应用于特征生…

[C1W3] Neural Networks and Deep Learning - Shallow neural networks

第三周:浅层神经网络(Shallow neural networks) 神经网络概述(Neural Network Overview) 本周你将学习如何实现一个神经网络.在我们深入学习具体技术之前,我希望快速的带你预览一下本周你将会学到的东西.如果在本节课中的某些细节你没有看懂你也不用担心,我们将在后面的几节课中深入讨论技术细节. 现在我们开始快速浏览一下如何实现神经网络.首先你需要输入特征 \(x\),参数 \(w\) 和 \(b\),通过这些你就可以计算出 \(z\),接下来使用 \…

Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activations

郑重声明:原文参见标题,如有侵权,请联系作者,将会撤销发布! arXiv:2003.10399v2 [cs.CV] 23 Jul 2020 ECCV 2020 1 https://github.com/ssharmin/spikingNN-adversarial-attack Abstract 在最近对可信任的神经网络的探索中,我们提出了一个潜在的候选,即脉冲神经网络(SNN)之于对抗攻击的内在鲁棒性.在这项工作中,我们证明对CIFAR数据集上的深度VGG和ResNet结构,在基于梯度的攻击下,…

Must Know Tips/Tricks in Deep Neural Networks

Must Know Tips/Tricks in Deep Neural Networks (by Xiu-Shen Wei) Deep Neural Networks, especially Convolutional Neural Networks (CNN), allows computational models that are composed of multiple processing layers to learn representations of data with…

(转) Ensemble Methods for Deep Learning Neural Networks to Reduce Variance and Improve Performance

Ensemble Methods for Deep Learning Neural Networks to Reduce Variance and Improve Performance 2018-12-19 13:02:45 This blog is copied from: https://machinelearningmastery.com/ensemble-methods-for-deep-learning-neural-networks/ Deep learning neural ne…

Must Know Tips/Tricks in Deep Neural Networks (by Xiu-Shen Wei)

http://lamda.nju.edu.cn/weixs/project/CNNTricks/CNNTricks.html Deep Neural Networks, especially Convolutional Neural Networks (CNN), allows computational models that are composed of multiple processing layers to learn representations of data with mul…

VGGNet论文翻译-Very Deep Convolutional Networks for Large-Scale Image Recognition

Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan[‡] & Andrew Zisserman[§] Visual Geometry Group, Department of Engineering Science, University of Oxford {karen,az}@robots.ox.ac.uk 用于大规模图像识别的深度卷积网络 Karen Simonyan[‡] &am…

[C4] Andrew Ng - Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization

About this Course This course will teach you the "magic" of getting deep learning to work well. Rather than the deep learning process being a black box, you will understand what drives performance, and be able to more systematically get good res…

（转）Understanding, generalisation, and transfer learning in deep neural networks

Understanding, generalisation, and transfer learning in deep neural networks FEBRUARY 27, 2017 This is the first in a series of posts looking at the ‘top 100 awesome deep learning papers.’ Deviating from the normal one-paper-per-day format, I’ll ta…

【神经网络与深度学习】生成式对抗网络GAN研究进展（五）——Deep Convolutional Generative Adversarial Nerworks，DCGAN

[前言] 本文首先介绍生成式模型,然后着重梳理生成式模型(Generative Models)中生成对抗网络(Generative Adversarial Network)的研究与发展.作者按照GAN主干论文.GAN应用性论文.GAN相关论文分类整理了45篇近两年的论文,着重梳理了主干论文之间的联系与区别,揭示生成式对抗网络的研究脉络. 本文涉及的论文有: Goodfellow Ian, Pouget-Abadie J, Mirza M, et al. Generative adver…

Hyperspectral Image Classification Using Similarity Measurements-Based Deep Recurrent Neural Networks

用RNN来做像素分类,输入是一系列相近的像素,长度人为指定为l,相近是利用像素相似度或是范围相似度得到的,计算个欧氏距离或是SAM. 数据是两个高光谱数据 1.Pavia University,Reflective Optics System Imaging Spectrometer (ROSIS) sensor得到,102个波段,1.3米空间分辨率,总大小610*340像素,9类地物 2.Salinas image,Airborne Visible/Infrared Imaging Spect…

Deep learning_CNN_Review：A Survey of the Recent Architectures of Deep Convolutional Neural Networks——2019

CNN综述文章的翻译 [2019 CVPR] A Survey of the Recent Architectures of Deep Convolutional Neural Networks 翻译综述深度卷积神经网络架构:从基本组件到结构创新目录摘要 1.引言 2.CNN基本组件 2.1 卷积层 2.2 池化层 2.3 激活函数 2.4 批次归一化 2.5 Dropout 2.6 全连接层…

论文翻译：2018_Source localization using deep neural networks in a shallow water environment

论文地址:https://asa.scitation.org/doi/abs/10.1121/1.5036725 深度神经网络在浅水环境中的源定位摘要: 深度神经网络(DNNs)在表征复杂的非线性关系方面具有优势.本文将DNNs应用于浅水环境下的源定位.提出了两种方法,通过不同的神经网络结构来估计宽带源的范围和深度.第一阶段采用经典的两阶段方案,特征提取和DNN分析是两个独立的步骤;与模态信号空间相关联的特征向量被提取为输入特征.然后,利用时滞神经网络对长期特征表示进行建模,构建回归模型;第二…

Classifying plankton with deep neural networks

Classifying plankton with deep neural networks The National Data Science Bowl, a data science competition where the goal was to classify images of plankton, has just ended. I participated with six other members of my research lab, the Reservoir lab o…

AlexNet论文翻译-ImageNet Classification with Deep Convolutional Neural Networks

ImageNet Classification with Deep Convolutional Neural Networks 深度卷积神经网络的ImageNet分类 Alex Krizhevsky University of Toronto 多伦多大学 kriz@cs.utoronto.ca Ilya Sutskever University of Toronto 多伦多大学 ilya@cs.utoronto.ca Geoffrey E. Hinton University of Toront…

为什么深度神经网络难以训练Why are deep neural networks hard to train?

Imagine you're an engineer who has been asked to design a computer from scratch. One day you're working away in your office, designing logical circuits, setting out AND gates, OR gates, and so on, when your boss walks in with bad news. The customer h…

Paper: ImageNet Classification with Deep Convolutional Neural Network

本文介绍了Alex net 在imageNet Classification 中的惊人表现,获得了ImagaNet LSVRC2012第一的好成绩,开启了卷积神经网络在cv领域的广泛应用. 1.数据集 ImageNet [6], which consists of over 15 million labeled high-resolution images in over 22,000 categories. here,ILSVRC uses a subset of ImageNet with…

[C3] Andrew Ng - Neural Networks and Deep Learning

About this Course If you want to break into cutting-edge AI, this course will help you do so. Deep learning engineers are highly sought after, and mastering deep learning will give you numerous new career opportunities. Deep learning is also a new "s…

1 - ImageNet Classification with Deep Convolutional Neural Network （阅读翻译）

ImageNet Classification with Deep Convolutional Neural Network 利用深度卷积神经网络进行ImageNet分类 Abstract We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 d…