Macro-Micro Adversarial Network for Human Parsing

ECCV-2018   2018-10-27 15:15:07

Paper: https://arxiv.org/pdf/1807.08260.pdf

Codehttps://github.com/RoyalVane/MMAN

Motiviation-1: Why use the Adversarial Loss ? 

Based on CNN architecture, the pixel-wise classification loss is usually used [19,34,10] which punishes the classification error for each pixel. Despite providing an effective baseline, the pixel-wise classification loss which is designed for per-pixel category prediction, has two drawbacks.

First, the pixel-wise classification loss may lead to local inconsistency, such as holes and blur. The reason is that it merely penalizes the false prediction on every pixel without explicitly considering the correlation among the adjacent pixels.

Second, pixel-wise classification loss may lead to semantic inconsistency in the overall segmentation map, such as unreasonable human poses and incorrect spatial relationship of body parts. Compared to the local inconsistency, the semantic inconsistency is generated from deeper layers. When only looking at a local region, the learned model does not have an overall sense of the topology of body parts.

In the attempt to address the inconsistency problems, the conditional random fields (CRFs) [17] can be employed as a post processing method. However, CRFs usually handle inconsistency in very limited scope (locally) due to the pairwise potentials, and may even generate worse label maps given poor initial segmentation result. As an alternative to CRFs, a recent work proposes the use of adversarial network [24]. Since the adversarial loss assesses whether a label map is real or fake by joint configuration of many label variables, it can enforce higher-level consistency, which cannot be achieved with pairwise terms or the per-pixel classification loss. Now, an increasing number of works adopt the routine of combining the cross entropy loss with an adversarial loss to produce label maps closer to the ground truth [5,27,12].

Motiviation-2: Why use the Two Discriminator ? 

Nevertheless, the previous adversarial network also has its limitations.

First, the single discriminator back propagates only one adversarial loss to the generator. However, the local inconsistency is generated from top layers and the semantic inconsistency is generated from deep layers. The two targeted layers can not be discretely trained with only one adversarial loss.

Second, a single discriminator has to look at overall high-resolution image (or a large part of it) in order to supervise the global consistency. As mentioned by numbers of literatures [7,14], it is very difficult for a generator to fool the discriminator on a high-resolution image. As a result, the single discriminator back propagates a maximum adversarial loss invariably, which makes the training unbalanced. We call it poor convergence problem, as shown in Fig. 2.

Our Proposed Approach: 

In this paper, the basic objective is to improve the local and semantic consistency of label maps in human parsing. We adopt the idea of adversarial training and at the same time aim to addresses its limitations, i.e., the inferior ability in improving parsing consistency with a single adversarial loss and the poor convergence problem. Specifically, we introduce the Macro-Micro Adversarial Nets (MMAN). MMAN consists of a dual-output generator (G) and two discriminators (D), named Macro D and Micro D. The three modules constitute two adversarial networks (Macro AN, Micro AN), addressing the semantic consistency and the local consistency, respectively.

Difference with Previous Works: 

A brief pipeline of the proposed framework is shown in Fig. 3. It is in two critical aspects that MMAN departs from previous works.

First, our method explicitly copes with the local inconsistency and semantic inconsistency problem using two task-specific adversarial networks individually.

Second, our method does not use large-sized FOVs on high-resolution image, so we can avoid the poor convergence problem. More detailed description of the merits of the proposed network is provided in Section 3.5.

Our Contributions: 

We propose a new framework called Macro-Micro Adversarial Network (MMAN) for human parsing. The Macro AN and Micro AN focus on semantic and local inconsistency respectively, and work in complementary way to improve the parsing quality.

The two discriminators in our framework achieve local and global supervision on the label maps with small field of views (FOVs), which avoids the poor convergence problem caused by high-resolution images.

– The proposed adversarial net achieves very competitive mIoU on the LIP and PASCAL-Person-Part datasets, and can be well generalized on a relatively small dataset PPSS.

==

Macro-Micro Adversarial Network for Human Parsing的更多相关文章

  1. 《Macro-Micro Adversarial Network for Human Parsing》论文阅读笔记

    <Macro-Micro Adversarial Network for Human Parsing> 摘要:在人体语义分割中,像素级别的分类损失在其低级局部不一致性和高级语义不一致性方面 ...

  2. 论文阅读之:Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

    Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network  2016.10.23 摘要: ...

  3. 论文笔记: Mutual Learning to Adapt for Joint Human Parsing and Pose Estimation

    Mutual Learning to Adapt for Joint Human Parsing and Pose Estimation 2018-11-03 09:58:58 Paper: http ...

  4. Face Aging with Conditional Generative Adversarial Network 论文笔记

    Face Aging with Conditional Generative Adversarial Network 论文笔记 2017.02.28  Motivation: 本文是要根据最新的条件产 ...

  5. 生成对抗网络(Generative Adversarial Network)阅读笔记

    笔记持续更新中,请大家耐心等待 首先需要大概了解什么是生成对抗网络,参考维基百科给出的定义(https://zh.wikipedia.org/wiki/生成对抗网络): 生成对抗网络(英语:Gener ...

  6. GAN Generative Adversarial Network 生成式对抗网络-相关内容

    参考: https://baijiahao.baidu.com/s?id=1568663805038898&wfr=spider&for=pc Generative Adversari ...

  7. ASRWGAN: Wasserstein Generative Adversarial Network for Audio Super Resolution

    ASEGAN:WGAN音频超分辨率 这篇文章并不具有权威性,因为没有发表,说不定是外国的某个大学的毕业设计,或者课程结束后的作业.或者实验报告. CS230: Deep Learning, Sprin ...

  8. 论文阅读:Single Image Dehazing via Conditional Generative Adversarial Network

    Single Image Dehazing via Conditional Generative Adversarial Network Runde Li∗ Jinshan Pan∗ Zechao L ...

  9. Speech Super Resolution Generative Adversarial Network

    博客作者:凌逆战 博客地址:https://www.cnblogs.com/LXP-Never/p/10874993.html 论文作者:Sefik Emre Eskimez , Kazuhito K ...

随机推荐

  1. HTTP状态码--含义

    以下是HTTP状态码(HTTP Status Code)及其解释 1xx(临时响应) (继续) 请求者应当继续提出请求. 服务器返回此代码表示已收到请求的第一部分,正在等待其余部分. (切换协议) 请 ...

  2. TCP 套叠字

    一.  TCP 协议 # ------------TCP套叠字-------------------- server 端 import socket,time ip_port=('localhost' ...

  3. BOM设计的一些问题及解决方案探讨----合版BOM

    BOM是ERP的核心资料,也是比较难的一块,不仅涉及的内容多,要求准确性高,时效性也要求高.但传统的ERP在处理BOM时有不少问题,因此也有些软件公司引入了各种BOM类型,像"标准BOM&q ...

  4. linux vue uwsgi nginx 部署路飞学城 安装 vue

    vue+uwsgi+nginx部署路飞学城 有一天,老男孩的苑日天给我发来了两个神秘代码,听说是和mjj的结晶 超哥将这两个代码,放到了一个网站上,大家可以自行下载 路飞学城django代码#这个代码 ...

  5. GAN (Generative Adversarial Network)

    https://www.bilibili.com/video/av9770302/?p=15 前面说了auto-encoder,VAE可以用于生成 VAE的问题, AE的训练是让输入输出尽可能的接近, ...

  6. RoR- Database setup& SQLite ... Migrations

    *rails uses SQLite for database by default *Built-in command-line DB viewer *Self-contained,server-l ...

  7. Apache 2.4.27外网访问403(Forbidden)错误

    httpd.conf <Directory /> AllowOverride none #Require all denied 注释这句 Allow from all Require al ...

  8. python数据结构-如何根据字典中值的大小对字典项排序

    如何根据字典中值的大小对字典项排序 问题举例 某班英语成绩以字典形式存储,如何根据成绩高低,计算学生成绩排名 { “tom”:80, "lily":88, "marton ...

  9. nginx----------nginx日志详细分解

    1.客户端(用户)IP地址.如:上例中的 47.52.45.228 2.访问时间.如:上例中的 [03/Jan/2013:21:17:20 -0600] 3.请求方式(GET或者POST等).如:上例 ...

  10. JAVA的值传递问题

    为什么 Java 中只有值传递? 首先回顾一下在程序设计语言中有关将参数传递给方法(或函数)的一些专业术语.按值调用(call by value)表示方法接收的是调用者提供的值,而按引用调用(call ...