• Plese see this answer for a detailed example of how tf.nn.conv2d_backprop_input and tf.nn.conv2d_backprop_filter in an example.

In tf.nn, there are 4 closely related 2d conv functions:

  • tf.nn.conv2d
  • tf.nn.conv2d_backprop_filter
  • tf.nn.conv2d_backprop_input
  • tf.nn.conv2d_transpose
def conv2d(input, filter, strides, padding, use_cudnn_on_gpu=True, data_format="NHWC", name=None):
r"""Computes a 2-D convolution given 4-D `input` and `filter` tensors. Given an input tensor of shape `[batch, in_height, in_width, in_channels]`
and a filter / kernel tensor of shape
`[filter_height, filter_width, in_channels, out_channels]`, this op
performs the following: 1. Flattens the filter to a 2-D matrix with shape
`[filter_height * filter_width * in_channels, output_channels]`.
2. Extracts image patches from the input tensor to form a *virtual*
tensor of shape `[batch, out_height, out_width,
filter_height * filter_width * in_channels]`.
3. For each patch, right-multiplies the filter matrix and the image patch
vector. In detail, with the default NHWC format, output[b, i, j, k] =
sum_{di, dj, q} input[b, strides[1] * i + di, strides[2] * j + dj, q] *
filter[di, dj, q, k] Must have `strides[0] = strides[3] = 1`. For the most common case of the same
horizontal and vertices strides, `strides = [1, stride, stride, 1]`.

Given out = conv2d(x, w) and the output gradient d_out:

  • Use tf.nn.conv2d_backprop_filter to compute the filter gradient d_w
  • Use tf.nn.conv2d_backprop_input to compute the filter gradient d_x
  • tf.nn.conv2d_backprop_input can be implemented by tf.nn.conv2d_transpose
  • All 4 functions above can be implemented by tf.nn.conv2d
  • Actually, use TF's autodiff is the fastest way to compute gradients

Long Answer

Now, let's give an actual working code example of how to use the 4 functions above to compute d_x and d_w given d_out. This shows how conv2dconv2d_backprop_filterconv2d_backprop_input, and conv2d_transpose are related to each other. Please find the full scripts here.

Computing d_x in 4 different ways:

# Method 1: TF's autodiff
d_x = tf.gradients(f, x)[0] # Method 2: manually using conv2d
d_x_manual = tf.nn.conv2d(input=tf_pad_to_full_conv2d(d_out, w_size),
filter=tf_rot180(w),
strides=strides,
padding='VALID') # Method 3: conv2d_backprop_input
d_x_backprop_input = tf.nn.conv2d_backprop_input(input_sizes=x_shape,
filter=w,
out_backprop=d_out,
strides=strides,
padding='VALID') # Method 4: conv2d_transpose
d_x_transpose = tf.nn.conv2d_transpose(value=d_out,
filter=w,
output_shape=x_shape,
strides=strides,
padding='VALID')

Computing d_w in 3 different ways:

# Method 1: TF's autodiff
d_w = tf.gradients(f, w)[0] # Method 2: manually using conv2d
d_w_manual = tf_NHWC_to_HWIO(tf.nn.conv2d(input=x,
filter=tf_NHWC_to_HWIO(d_out),
strides=strides,
padding='VALID')) # Method 3: conv2d_backprop_filter
d_w_backprop_filter = tf.nn.conv2d_backprop_filter(input=x,
filter_sizes=w_shape,
out_backprop=d_out,
strides=strides,
padding='VALID')

Please see the full scripts for the implementation of tf_rot180tf_pad_to_full_conv2dtf_NHWC_to_HWIO. In the scripts, we check that the final output values of different methods are the same; a numpy implementation is also available.

tensorflow 卷积/反卷积-池化/反池化操作详解的更多相关文章

  1. 超详细的Tensorflow模型的保存和加载(理论与实战详解)

    1.Tensorflow的模型到底是什么样的? Tensorflow模型主要包含网络的设计(图)和训练好的各参数的值等.所以,Tensorflow模型有两个主要的文件: a) Meta graph: ...

  2. jdbc连接池中c3p0的配置文件的详解以及在在java中如何使用

    <c3p0-config> <!-- 默认配置,如果没有指定则使用这个配置 --> <default-config> <property name=" ...

  3. opencv-python图像二值化函数cv2.threshold函数详解及参数cv2.THRESH_OTSU使用

    cv2.threshold()函数的作用是将一幅灰度图二值化,基本用法如下: #ret:暂时就认为是设定的thresh阈值,mask:二值化的图像 ret,mask = cv2.threshold(i ...

  4. CentOS Minimal版最小化安装后VMware联网详解

    最近想搞个mailman邮件列表,又不想在我常用的CentOS 6.4上做实验,怕破坏了环境,于是就想装个试验机,又嫌它占空间太大,于是找了半天发现CentOS 6.0的minimal版本最适合了,装 ...

  5. Windows里安装wireshark或者ethereal工具(包括汉化破解)(图文详解)

    不多说,直接上干货! https://www.wireshark.org/download.html 我这里,读取的是,来自于https://www.ll.mit.edu/ideval/data/19 ...

  6. Java学习笔记 线程池使用及详解

    有点笨,参考了好几篇大佬们写的文章才整理出来的笔记.... 字面意思上解释,线程池就是装有线程的池,我们可以把要执行的多线程交给线程池来处理,和连接池的概念一样,通过维护一定数量的线程池来达到多个线程 ...

  7. 第十四节,TensorFlow中的反卷积,反池化操作以及gradients的使用

    反卷积是指,通过测量输出和已知输入重构未知输入的过程.在神经网络中,反卷积过程并不具备学习的能力,仅仅是用于可视化一个已经训练好的卷积神经网络,没有学习训练的过程.反卷积有着许多特别的应用,一般可以用 ...

  8. 『TensorFlow』卷积层、池化层详解

    一.前向计算和反向传播数学过程讲解

  9. 深度学习原理与框架-Tensorflow卷积神经网络-卷积神经网络mnist分类 1.tf.nn.conv2d(卷积操作) 2.tf.nn.max_pool(最大池化操作) 3.tf.nn.dropout(执行dropout操作) 4.tf.nn.softmax_cross_entropy_with_logits(交叉熵损失) 5.tf.truncated_normal(两个标准差内的正态分布)

    1. tf.nn.conv2d(x, w, strides=[1, 1, 1, 1], padding='SAME')  # 对数据进行卷积操作 参数说明:x表示输入数据,w表示卷积核, stride ...

随机推荐

  1. Future和FutureTask

    上次介绍了Callable实现多线程的方法.现在介绍和Callable搭配的类.上一篇只是简单的用Callable做了一个demo. 一.Future 1.关于callable和runable的区别( ...

  2. hdu 5734 Acperience 水题

    Acperience 题目连接: http://acm.hdu.edu.cn/showproblem.php?pid=5734 Description Deep neural networks (DN ...

  3. SlickOne 敏捷开发框架介绍(二) -- 多用户/多租户/SAAS软件基础框架实现

    前言:在应用于集团版客户或SAAS平台服务的业务系统中,流程管理系统需要支持多用户组织模型.其中包括角色数据.流程定义数据和流程实例数据的多用户标识绑定.本文旨在全面描述如何基于SlickOne敏捷开 ...

  4. Ubuntu 16.04实现SSH无密码登录/免密登录/自动登录(ssh-keygen/ssh-copy-id)

    ssh-keygen:产生公钥与私钥(在~/.ssh) ssh-copy-id:将本机的公钥复制到远程机器的authorized_keys文件中(在~/.ssh),ssh-copy-id也能让你有到远 ...

  5. spring cloud 学习(9) - turbine stream无法在eureka注册的解决办法

    turbine是啥就不多解释了,初次接触的可以移步spring cloud 学习(4) - hystrix 服务熔断处理 拉到最后看一下,turbine stream默认情况下启动成功后,eureka ...

  6. css卷叶效果

    <!DOCTYPE HTML><html lang="en-US"><head> <meta charset="UTF-8&qu ...

  7. STM32 System and Timer Clock Configurations

    STM32 System and Timer Clock Configurations I've started writing some software to drive a series of  ...

  8. Programmed Adjustable Power

    Programmed Adjustable Power I just explored an easy scheme to design a high precision programmed adj ...

  9. ASP.Net中关于WebAPI与Ajax进行跨域数据交互时Cookies数据的传递

    本文主要介绍了ASP.Net WebAPI与Ajax进行跨域数据交互时Cookies数据传递的相关知识.具有很好的参考价值.下面跟着小编一起来看下吧 前言 最近公司项目进行架构调整,由原来的三层架构改 ...

  10. Win10专业版永久激活方法

    自从升级安装了Windows10系统以后,我想很多朋友和我一样,想要激活Windows10系统,但是小编找了半天以后发现,很多激活工具都是批量激活的,也就是只有180天的使用时间,那么我们怎么永久激活 ...