tensorflow 卷积/反卷积-池化/反池化操作详解
- Plese see this answer for a detailed example of how
tf.nn.conv2d_backprop_input
andtf.nn.conv2d_backprop_filter
in an example.
In tf.nn
, there are 4 closely related 2d conv functions:
tf.nn.conv2d
tf.nn.conv2d_backprop_filter
tf.nn.conv2d_backprop_input
tf.nn.conv2d_transpose
def conv2d(input, filter, strides, padding, use_cudnn_on_gpu=True, data_format="NHWC", name=None):
r"""Computes a 2-D convolution given 4-D `input` and `filter` tensors. Given an input tensor of shape `[batch, in_height, in_width, in_channels]`
and a filter / kernel tensor of shape
`[filter_height, filter_width, in_channels, out_channels]`, this op
performs the following: 1. Flattens the filter to a 2-D matrix with shape
`[filter_height * filter_width * in_channels, output_channels]`.
2. Extracts image patches from the input tensor to form a *virtual*
tensor of shape `[batch, out_height, out_width,
filter_height * filter_width * in_channels]`.
3. For each patch, right-multiplies the filter matrix and the image patch
vector. In detail, with the default NHWC format, output[b, i, j, k] =
sum_{di, dj, q} input[b, strides[1] * i + di, strides[2] * j + dj, q] *
filter[di, dj, q, k] Must have `strides[0] = strides[3] = 1`. For the most common case of the same
horizontal and vertices strides, `strides = [1, stride, stride, 1]`.
Given out = conv2d(x, w)
and the output gradient d_out
:
- Use
tf.nn.conv2d_backprop_filter
to compute the filter gradientd_w
- Use
tf.nn.conv2d_backprop_input
to compute the filter gradientd_x
tf.nn.conv2d_backprop_input
can be implemented bytf.nn.conv2d_transpose
- All 4 functions above can be implemented by
tf.nn.conv2d
- Actually, use TF's autodiff is the fastest way to compute gradients
Long Answer
Now, let's give an actual working code example of how to use the 4 functions above to compute d_x
and d_w
given d_out
. This shows how conv2d
, conv2d_backprop_filter
, conv2d_backprop_input
, and conv2d_transpose
are related to each other. Please find the full scripts here.
Computing d_x
in 4 different ways:
# Method 1: TF's autodiff
d_x = tf.gradients(f, x)[0]
# Method 2: manually using conv2d
d_x_manual = tf.nn.conv2d(input=tf_pad_to_full_conv2d(d_out, w_size),
filter=tf_rot180(w),
strides=strides,
padding='VALID')
# Method 3: conv2d_backprop_input
d_x_backprop_input = tf.nn.conv2d_backprop_input(input_sizes=x_shape,
filter=w,
out_backprop=d_out,
strides=strides,
padding='VALID')
# Method 4: conv2d_transpose
d_x_transpose = tf.nn.conv2d_transpose(value=d_out,
filter=w,
output_shape=x_shape,
strides=strides,
padding='VALID')
Computing d_w
in 3 different ways:
# Method 1: TF's autodiff
d_w = tf.gradients(f, w)[0]
# Method 2: manually using conv2d
d_w_manual = tf_NHWC_to_HWIO(tf.nn.conv2d(input=x,
filter=tf_NHWC_to_HWIO(d_out),
strides=strides,
padding='VALID'))
# Method 3: conv2d_backprop_filter
d_w_backprop_filter = tf.nn.conv2d_backprop_filter(input=x,
filter_sizes=w_shape,
out_backprop=d_out,
strides=strides,
padding='VALID')
Please see the full scripts for the implementation of tf_rot180
, tf_pad_to_full_conv2d
, tf_NHWC_to_HWIO
. In the scripts, we check that the final output values of different methods are the same; a numpy implementation is also available.
- 第十四节,TensorFlow中的反卷积,反池化操作以及gradients的使用
- http://www.cnblogs.com/pinard/p/6494810.html :卷积神经网络(CNN)反向传播算法
- http://blog.csdn.net/yunpiao123456/article/details/52437794
tensorflow 卷积/反卷积-池化/反池化操作详解的更多相关文章
- 超详细的Tensorflow模型的保存和加载(理论与实战详解)
1.Tensorflow的模型到底是什么样的? Tensorflow模型主要包含网络的设计(图)和训练好的各参数的值等.所以,Tensorflow模型有两个主要的文件: a) Meta graph: ...
- jdbc连接池中c3p0的配置文件的详解以及在在java中如何使用
<c3p0-config> <!-- 默认配置,如果没有指定则使用这个配置 --> <default-config> <property name=" ...
- opencv-python图像二值化函数cv2.threshold函数详解及参数cv2.THRESH_OTSU使用
cv2.threshold()函数的作用是将一幅灰度图二值化,基本用法如下: #ret:暂时就认为是设定的thresh阈值,mask:二值化的图像 ret,mask = cv2.threshold(i ...
- CentOS Minimal版最小化安装后VMware联网详解
最近想搞个mailman邮件列表,又不想在我常用的CentOS 6.4上做实验,怕破坏了环境,于是就想装个试验机,又嫌它占空间太大,于是找了半天发现CentOS 6.0的minimal版本最适合了,装 ...
- Windows里安装wireshark或者ethereal工具(包括汉化破解)(图文详解)
不多说,直接上干货! https://www.wireshark.org/download.html 我这里,读取的是,来自于https://www.ll.mit.edu/ideval/data/19 ...
- Java学习笔记 线程池使用及详解
有点笨,参考了好几篇大佬们写的文章才整理出来的笔记.... 字面意思上解释,线程池就是装有线程的池,我们可以把要执行的多线程交给线程池来处理,和连接池的概念一样,通过维护一定数量的线程池来达到多个线程 ...
- 第十四节,TensorFlow中的反卷积,反池化操作以及gradients的使用
反卷积是指,通过测量输出和已知输入重构未知输入的过程.在神经网络中,反卷积过程并不具备学习的能力,仅仅是用于可视化一个已经训练好的卷积神经网络,没有学习训练的过程.反卷积有着许多特别的应用,一般可以用 ...
- 『TensorFlow』卷积层、池化层详解
一.前向计算和反向传播数学过程讲解
- 深度学习原理与框架-Tensorflow卷积神经网络-卷积神经网络mnist分类 1.tf.nn.conv2d(卷积操作) 2.tf.nn.max_pool(最大池化操作) 3.tf.nn.dropout(执行dropout操作) 4.tf.nn.softmax_cross_entropy_with_logits(交叉熵损失) 5.tf.truncated_normal(两个标准差内的正态分布)
1. tf.nn.conv2d(x, w, strides=[1, 1, 1, 1], padding='SAME') # 对数据进行卷积操作 参数说明:x表示输入数据,w表示卷积核, stride ...
随机推荐
- Future和FutureTask
上次介绍了Callable实现多线程的方法.现在介绍和Callable搭配的类.上一篇只是简单的用Callable做了一个demo. 一.Future 1.关于callable和runable的区别( ...
- hdu 5734 Acperience 水题
Acperience 题目连接: http://acm.hdu.edu.cn/showproblem.php?pid=5734 Description Deep neural networks (DN ...
- SlickOne 敏捷开发框架介绍(二) -- 多用户/多租户/SAAS软件基础框架实现
前言:在应用于集团版客户或SAAS平台服务的业务系统中,流程管理系统需要支持多用户组织模型.其中包括角色数据.流程定义数据和流程实例数据的多用户标识绑定.本文旨在全面描述如何基于SlickOne敏捷开 ...
- Ubuntu 16.04实现SSH无密码登录/免密登录/自动登录(ssh-keygen/ssh-copy-id)
ssh-keygen:产生公钥与私钥(在~/.ssh) ssh-copy-id:将本机的公钥复制到远程机器的authorized_keys文件中(在~/.ssh),ssh-copy-id也能让你有到远 ...
- spring cloud 学习(9) - turbine stream无法在eureka注册的解决办法
turbine是啥就不多解释了,初次接触的可以移步spring cloud 学习(4) - hystrix 服务熔断处理 拉到最后看一下,turbine stream默认情况下启动成功后,eureka ...
- css卷叶效果
<!DOCTYPE HTML><html lang="en-US"><head> <meta charset="UTF-8&qu ...
- STM32 System and Timer Clock Configurations
STM32 System and Timer Clock Configurations I've started writing some software to drive a series of ...
- Programmed Adjustable Power
Programmed Adjustable Power I just explored an easy scheme to design a high precision programmed adj ...
- ASP.Net中关于WebAPI与Ajax进行跨域数据交互时Cookies数据的传递
本文主要介绍了ASP.Net WebAPI与Ajax进行跨域数据交互时Cookies数据传递的相关知识.具有很好的参考价值.下面跟着小编一起来看下吧 前言 最近公司项目进行架构调整,由原来的三层架构改 ...
- Win10专业版永久激活方法
自从升级安装了Windows10系统以后,我想很多朋友和我一样,想要激活Windows10系统,但是小编找了半天以后发现,很多激活工具都是批量激活的,也就是只有180天的使用时间,那么我们怎么永久激活 ...