本文转自:http://www.jianshu.com/p/2acb804dd811

GAN论文整理

作者 FinlayLiu 已关注

2016.11.09 13:21 字数 1551 阅读 1263评论 0喜欢 7

原始GAN

Goodfellow和Bengio等人发表在NIPS 2014年的文章Generative adversary network,是生成对抗网络的开创文章,论文思想启发自博弈论中的二人零和博弈。在二人零和博弈中,两位博弈方的利益之和为零或一个常数,即一方有所得,另一方必有所失。GAN模型中的两位博弈方分别由生成式模型(generative model)和判别式模型(discriminative model)充当。生成模型G捕捉样本数据的分布,判别模型D是一个二分类器,估计一个样本来自于训练数据(而非生成数据)的概率。G和D一般都是非线性映射函数,例如多层感知机、卷积神经网络等。

如图所示,左图是一个判别式模型,当输入训练数据x时,期待输出高概率(接近1);右图下半部分是生成模型,输入是一些服从某一简单分布(例如高斯分布)的随机噪声z,输出是与训练图像相同尺寸的生成图像。向判别模型D输入生成样本,对于D来说期望输出低概率(判断为生成样本),对于生成模型G来说要尽量欺骗D,使判别模型输出高概率(误判为真实样本),从而形成竞争与对抗。

GAN.png

GAN优势很多:根据实际的结果,看上去产生了更好的样本;GAN能训练任何一种生成器网络;GAN不需要设计遵循任何种类的因式分解的模型,任何生成器网络和任何鉴别器都会有用;GAN无需利用马尔科夫链反复采样,无需在学习过程中进行推断,回避了近似计算棘手的概率的难题。

GAN主要存在的以下问题:网络难以收敛,目前所有的理论都认为GAN应该在纳什均衡上有很好的表现,但梯度下降只有在凸函数的情况下才能保证实现纳什均衡。

GAN发展

一方面GAN的发展很快,这里只是简单粗略将相关论文分了几类,欢迎反馈,持续更新。此外最近ICLR 2017 在进行Open Review,可以关注下ICLR 2017 Conference Track,也有相应论文笔记分享ICLR 2017 | GAN Missing Modes 和 GAN

GAN从2014年到现在发展很快,特别是最近ICLR 2016/2017关于GAN的论文很多,GAN现在有很多问题还有到解决,潜力很大。总体可以将已有的GANs论文分为以下几类

  1. GAN Theory
  2. GAN in Semi-supervised
  3. Muti-GAN
  4. GAN with other Generative model
  5. GAN with RNN
  6. GAN in Application

GAN Theory

此类关注与无监督GAN本身原理的研究:比较两个分布的距离;用DL的一些方法让GAN快速收敛等等。相关论文有:

  • GAN: Goodfellow, Ian, et al. "Generative adversarial nets." Advances in Neural Information Processing Systems. 2014.
  • LAPGAN: Denton, Emily L., Soumith Chintala, and Rob Fergus. "Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks." Advances in neural information processing systems. 2015.
  • DCGAN: Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).
  • Improved GAN: Salimans, Tim, et al. "Improved techniques for training gans." arXiv preprint arXiv:1606.03498 (2016).
  • InfoGAN: Chen, Xi, et al. "Infogan: Interpretable representation learning by information maximizing generative adversarial nets." arXiv preprint arXiv:1606.03657(2016).**
  • EnergyGAN: Zhao, Junbo, Michael Mathieu, and Yann LeCun. "Energy-based Generative Adversarial Network." arXiv preprint arXiv:1609.03126 (2016).
  • Creswell, Antonia, and Anil A. Bharath. "Task Specific Adversarial Cost Function." arXiv preprint arXiv:1609.08661 (2016).
  • f-GAN: Nowozin, Sebastian, Botond Cseke, and Ryota Tomioka. "f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization." arXiv preprint arXiv:1606.00709 (2016).
  • Unrolled Generative Adversarial Networks, ICLR 2017 Open Review
  • Improving Generative Adversarial Networks with Denoising Feature Matching, ICLR 2017 Open Review
  • Mode Regularized Generative Adversarial Networks, ICLR 2017 Open Review
  • b-GAN: Unified Framework of Generative Adversarial Networks, ICLR 2017 Open Review
  • Mohamed, Shakir, and Balaji Lakshminarayanan. "Learning in Implicit Generative Models." arXiv preprint arXiv:1610.03483 (2016).

GAN in Semi-supervised

此类研究将GAN用于半监督学习,相关论文有:

  • Springenberg, Jost Tobias. "Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks." arXiv preprint arXiv:1511.06390 (2015).
  • Odena, Augustus. "Semi-Supervised Learning with Generative Adversarial Networks." arXiv preprint arXiv:1606.01583 (2016).

Muti-GAN

此类研究将多个GAN进行组合,相关论文有:

  • CoupledGAN: Liu, Ming-Yu, and Oncel Tuzel. "Coupled Generative Adversarial Networks." arXiv preprint arXiv:1606.07536 (2016).
  • Wang, Xiaolong, and Abhinav Gupta. "Generative Image Modeling using Style and Structure Adversarial Networks." arXiv preprint arXiv:1603.05631(2016).
  • Generative Adversarial Parallelization, ICLR 2017 Open Review
  • LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation, ICLR 2017 Open Review

GAN with other Generative model

此类研究将GAN与其他生成模型组合,相关论文有:

  • Dosovitskiy, Alexey, and Thomas Brox. "Generating images with perceptual similarity metrics based on deep networks." arXiv preprint arXiv:1602.02644(2016).
  • Larsen, Anders Boesen Lindbo, Søren Kaae Sønderby, and Ole Winther. "Autoencoding beyond pixels using a learned similarity metric." arXiv preprint arXiv:1512.09300 (2015).
  • Theis, Lucas, and Matthias Bethge. "Generative image modeling using spatial lstms." Advances in Neural Information Processing Systems. 2015.

GAN with RNN

此类研究将GAN与RNN结合(也以参考Pixel RNN),相关论文有:

  • Im, Daniel Jiwoong, et al. "Generating images with recurrent adversarial networks." arXiv preprint arXiv:1602.05110 (2016).
  • Kwak, Hanock, and Byoung-Tak Zhang. "Generating Images Part by Part with Composite Generative Adversarial Networks." arXiv preprint arXiv:1607.05387 (2016).
  • Yu, Lantao, et al. "SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient." arXiv preprint arXiv:1609.05473 (2016).

GAN in Application

此类研究将GAN的实际运用(不包括图像生成),相关论文有:

  • Zhu, Jun-Yan, et al. "Generative visual manipulation on the natural image manifold." European Conference on Computer Vision. Springer International Publishing, 2016.
  • Creswell, Antonia, and Anil Anthony Bharath. "Adversarial Training For Sketch Retrieval." European Conference on Computer Vision. Springer International Publishing, 2016.
  • Reed, Scott, et al. "Generative adversarial text to image synthesis." arXiv preprint arXiv:1605.05396 (2016).
  • Ravanbakhsh, Siamak, et al. "Enabling Dark Energy Science with Deep Generative Models of Galaxy Images." arXiv preprint arXiv:1609.05796(2016).
  • Abadi, Martín, and David G. Andersen. "Learning to Protect Communications with Adversarial Neural Cryptography." arXiv preprint arXiv:1610.06918(2016).
  • Odena, Augustus, Christopher Olah, and Jonathon Shlens. "Conditional Image Synthesis With Auxiliary Classifier GANs." arXiv preprint arXiv:1610.09585 (2016).
  • Ledig, Christian, et al. "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network." arXiv preprint arXiv:1609.04802 (2016).
  • Nguyen, Anh, et al. "Synthesizing the preferred inputs for neurons in neural networks via deep generator networks." arXiv preprint arXiv:1605.09304(2016).

(转) GAN论文整理的更多相关文章

  1. REST架构简析(原论文整理)

    0 引言        目前,互联网在社会中扮演的角色越来越重要.通过互联网为广大群众提供服务,也是互联网成功的关键.互联网服务架构目前大多数都是基于REST架构来完成的.REST从它诞生至今,可以说 ...

  2. Generative Adversarial Networks,gan论文的畅想

    前天看完Generative Adversarial Networks的论文,不知道有什么用处,总想着机器生成的数据会有机器的局限性,所以百度看了一些别人 的看法和观点,可能我是机器学习小白吧,看完之 ...

  3. 深度学习-Wasserstein GAN论文理解笔记

    GAN存在问题 训练困难,G和D多次尝试没有稳定性,Loss无法知道能否优化,生成样本单一,改进方案靠暴力尝试 WGAN GAN的Loss函数选择不合适,使模型容易面临梯度消失,梯度不稳定,优化目标不 ...

  4. CVPapers论文整理工具-开源

    一.工具介绍及运行实例 相信计算机视觉领域的同道中人都知道这个Computer Vision Resource网站, http://www.cvpapers.com/  网页部分截图如下: 可以看到有 ...

  5. 条件GAN论文简单解读

        条件GAN(Conditional Generative Adversarial Nets),原文地址为CGAN. Abstract     生成对抗网络(GAN)是最近提出的训练生成模型(g ...

  6. OCR论文整理

    论文地址:https://github.com/ChanChiChoi/awesome-ocr 下面是已经看过的论文: CTPN CRNN TextBoxes EAST FOTS PixelLink

  7. Gan-based zero-shot learning 论文整理

    1 Feature Generating Networks for Zero-Shot Learning Suffering from the extreme training data imbala ...

  8. 《Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering》论文整理

    融合异构知识进行常识问答 论文标题 -- <Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense ...

  9. 网络压缩论文整理(network compression)

    1. Parameter pruning and sharing 1.1 Quantization and Binarization Compressing deep convolutional ne ...

随机推荐

  1. XMLHttpRequest对象(Ajax)的状态码(readystate) HTTP状态代码(status)

    2018-11-28 14:19:00 来自 :XMLHttpRequest对象(Ajax)的状态码(readystate)  HTTP状态代码(status) XMLHttpRequest对象(Aj ...

  2. Spark学习之路 (十二)SparkCore的调优之资源调优

    摘抄自:https://tech.meituan.com/spark-tuning-basic.html 一.概述 在开发完Spark作业之后,就该为作业配置合适的资源了.Spark的资源参数,基本都 ...

  3. (Review cs231n) Gradient Vectorized

    注意: 1.每次更新,都要进行一次完整的forward和backward,想要进行更新,需要梯度,所以你需要前馈样本,马上反向求导,得到梯度,然后根据求得的梯度进行权值微调,完成权值更新. 2.前馈得 ...

  4. window下nodejs用nodemon启动koa2项目(用cmd启动不了,要用Git Bash Here 启动才可以)

    window下nodejs用nodemon启动koa2项目(用cmd启动不了,要用Git Bash Here 启动才可以)nodemon --watch 'app/**/*' -e ts --exec ...

  5. CSS 选择器权重计算规则

    其实,CSS有自己的优先级计算公式,而不仅仅是行间>内部>外部样式:ID>class>元素. 一.样式类型 1.行间 <h1 style="font-size: ...

  6. ARQC与ARPC的生成和校验方法

    转载:https://www.cnblogs.com/ttss/p/4364328.html ARQC:authenticate request cryptogram,授权请求报文 ARPC:auth ...

  7. 查询在某一个时间段内的sql(oracel)

    ( to_char(t.TUIJIAN_TIME, 'yyyy-MM-dd') between   #{begin_time}   and #{end_time} )

  8. Linux下MySQL远程链接配置

    配置步骤: 1).首先进入数据库,使用系统数据库mysql mysql -u root -p mysql 2).接着对系统数据库的root账户设置远程访问的密码,与本地的root访问密码并不冲突 gr ...

  9. MyEclipse中项目运行时发生了Tomcat报错:[java.lang.OutOfMemoryError: PermGen space]

    Tomcat内存溢出,异常信息如下: 十一月 26, 2017 1:52:26 下午 org.apache.catalina.core.ContainerBase$ContainerBackgroun ...

  10. P3690 【模板】Link Cut Tree (动态树)

    P3690 [模板]Link Cut Tree (动态树) 认父不认子的lct 注意:不 要 把 $fa[x]$和$nrt(x)$ 混 在 一 起 ! #include<cstdio> v ...