Element-wise operations

An element-wise operation operates on corresponding elements between tensors.

Two tensors must have the same shape in order to perform element-wise operations on them.

Suppose we have the following two tensors(Both of these tensors are rank-2 tensors with a shape of 2 \(\times\) 2):

t1 = torch.tensor([
[1, 2],
[3, 4]
], dtype=torch.float32) t2 = torch.tensor([
[9, 8],
[7, 6]
], dtype=torch.float32)

The elements of the first axis are arrays and the elements of the second axis are numbers.

# Example of the first axis
> print(t1[0])
tensor([1., 2.]) # Example of the second axis
> print(t1[0][0])
tensor(1.)

Addition is an element-wise operation.

> t1 + t2
tensor([[10., 10.],
[10., 10.]])

In fact, all the arithmetic operations, add, subtract, multiply, and divide are element-wise operations. There are two ways we can do this:

  1. Using these symbolic operations:
> t + 2
tensor([[3., 4.],
[5., 6.]]) > t - 2
tensor([[-1., 0.],
[1., 2.]]) > t * 2
tensor([[2., 4.],
[6., 8.]]) > t / 2
tensor([[0.5000, 1.0000],
[1.5000, 2.0000]])
  1. Or equivalently, these built-in tensor methods:
> t.add(2)
tensor([[3., 4.],
[5., 6.]]) > t.sub(2)
tensor([[-1., 0.],
[1., 2.]]) > t.mul(2)
tensor([[2., 4.],
[6., 8.]]) > t.div(2)
tensor([[0.5000, 1.0000],
[1.5000, 2.0000]])

Broadcasting tensors

Broadcasting is the concept whose implementation allows us to add scalars to higher dimensional tensors.

We can see what the broadcasted scalar value looks like using the broadcast_to()Numpy function:

> np.broadcast_to(2, t.shape)
array([[2, 2],
[2, 2]])
//This means the scalar value is transformed into a rank-2 tensor just like t, and //just like that, the shapes match and the element-wise rule of having the same //shape is back in play.

Trickier example of broadcasting

t1 = torch.tensor([
[1, 1],
[1, 1]
], dtype=torch.float32) t2 = torch.tensor([2, 4], dtype=torch.float32)

Even through these two tensors have differing shapes, the element-wise operation is possible, and broadcasting is what makes the operation possible.

> np.broadcast_to(t2.numpy(), t1.shape)
array([[2., 4.],
[2., 4.]], dtype=float32) >t1 + t2
tensor([[3., 5.],
[3., 5.]])

When do we actually use broadcasting? We often need to use broadcasting when we are preprocessing our data, and especially during normalization routines.


Comparison operations are element-wise. For a given comparison operation between tensors, a new tensor of the same shape is returned with each element containing either a 0 or a 1.

> t = torch.tensor([
[0, 5, 0],
[6, 0, 7],
[0, 8, 0]
], dtype=torch.float32)

Let's check out some of the comparison operations.

> t.eq(0)
tensor([[1, 0, 1],
[0, 1, 0],
[1, 0, 1]], dtype=torch.uint8) > t.ge(0)
tensor([[1, 1, 1],
[1, 1, 1],
[1, 1, 1]], dtype=torch.uint8) > t.gt(0)
tensor([[0, 1, 0],
[1, 0, 1],
[0, 1, 0]], dtype=torch.uint8) > t.lt(0)
tensor([[0, 0, 0],
[0, 0, 0],
[0, 0, 0]], dtype=torch.uint8) > t.le(7)
tensor([[1, 1, 1],
[1, 1, 1],
[1, 0, 1]], dtype=torch.uint8)

Element-wise operations using functions

Here are some examples:

> t.abs()
tensor([[0., 5., 0.],
[6., 0., 7.],
[0., 8., 0.]]) > t.sqrt()
tensor([[0.0000, 2.2361, 0.0000],
[2.4495, 0.0000, 2.6458],
[0.0000, 2.8284, 0.0000]]) > t.neg()
tensor([[-0., -5., -0.],
[-6., -0., -7.],
[-0., -8., -0.]]) > t.neg().abs()
tensor([[0., 5., 0.],
[6., 0., 7.],
[0., 8., 0.]])

Element-wise operations的更多相关文章

  1. 向量的一种特殊乘法 element wise multiplication

    向量的一种特殊乘法 element wise multiplication 物体反射颜色的计算采用这样的模型: vec3 reflectionColor = objColor * lightColor ...

  2. [C2P1] Andrew Ng - Machine Learning

    About this Course Machine learning is the science of getting computers to act without being explicit ...

  3. TensorRT 3:更快的TensorFlow推理和Volta支持

    TensorRT 3:更快的TensorFlow推理和Volta支持 TensorRT 3: Faster TensorFlow Inference and Volta Support 英伟达Tens ...

  4. (转)A Beginner's Guide To Understanding Convolutional Neural Networks Part 2

    Adit Deshpande CS Undergrad at UCLA ('19) Blog About A Beginner's Guide To Understanding Convolution ...

  5. Understanding Convolution in Deep Learning

    Understanding Convolution in Deep Learning Convolution is probably the most important concept in dee ...

  6. Must Know Tips/Tricks in Deep Neural Networks

    Must Know Tips/Tricks in Deep Neural Networks (by Xiu-Shen Wei)   Deep Neural Networks, especially C ...

  7. [Tensorflow] Cookbook - Neural Network

    In this chapter, we'll cover the following recipes: Implementing Operational Gates Working with Gate ...

  8. [Tensorflow] Cookbook - Object Classification based on CIFAR-10

    Convolutional Neural Networks (CNNs) are responsible for the major breakthroughs in image recognitio ...

  9. Must Know Tips/Tricks in Deep Neural Networks (by Xiu-Shen Wei)

    http://lamda.nju.edu.cn/weixs/project/CNNTricks/CNNTricks.html Deep Neural Networks, especially Conv ...

  10. [转]An Intuitive Explanation of Convolutional Neural Networks

    An Intuitive Explanation of Convolutional Neural Networks https://ujjwalkarn.me/2016/08/11/intuitive ...

随机推荐

  1. 从零开始写STL—模板元编程之any

    any class any; (since C++17) The class any describes a type-safe container for single values of any ...

  2. [Bzoj3631][JLOI2014]松鼠的新家 (树上前缀和)

    3631: [JLOI2014]松鼠的新家 Time Limit: 10 Sec  Memory Limit: 128 MBSubmit: 2350  Solved: 1212[Submit][Sta ...

  3. sendEmail实现邮件报警发送

    安装wget http://caspian.dotconf.net/menu/Software/SendEmail/sendEmail-v1.56.tar.gz tar -xf sendEmail-v ...

  4. 抓包工具Fiddler使用宝典之捕获手机报文

    Fiddler 是通过代理来实现数据捕获的.对 Android 手机来说,也是通过将网络连接的代理指向 PC 机的 Fiddler port.来实现数据包的拦截. 以下,我以我的一次实践为例,向大家介 ...

  5. github的提交源码到服务器

    github是现代的代码库,各种牛人,各种开源,也是现在大公司招聘的一个考察点, 这里介绍一下怎样把本地源码提交到github上. 首先我们需要在github上创建一个respository. 2,输 ...

  6. Bash Shell 解析路径获取文件名称和文件夹名

    前言 还是今天再写一个自己主动化打包脚本.用到了从路径名中获取最后的文件名称.这里记录一下实现过程. 当然,最后我也会给出官方的做法.(ps:非常囧,实现完了才发现原来Bash Shell有现成的函数 ...

  7. [IT学习]Python 小项目 通讯录 思路

    建立一个通讯录查询软件,暂时只支持按姓名检索.出发点:无需登录企业门户,即可检索.要注意保护员工手机号,除非他自己同意显示. 欢迎您访问www.cnblogs.com/viphhs.转载请联系作者授权 ...

  8. Fluently NHibernate 插入CLOB字段

    ORA-01461: can bind a LONG value only for insert into a LONG column 插入oracle某表时报的错. 查来查去,是插入的某个字段值超长 ...

  9. lineage 世系 血缘 容错机制 DAG

    当某个RDD的部分数据丢失时候,Saprk会根据记录的世系关系找到该RDD的父RDD以及更上级的RDD.只需要将该RDD依赖的上级RDD重新计算就可以将该RDD进行恢复. Directed Acycl ...

  10. 使用buildroot搭建linux文件系统【转】

    本文转载自:http://blog.csdn.net/metalseed/article/details/45423061 (文件系统搭建,强烈建议直接用buildroot,官网上有使用教程非常详细b ...