This tutorial aims to familiarize you with the concept of tensors in PyTorch and introduce you to the operations involving tensors in PyTorch. The Pytorch module works with data structures called ** tensors**, which are much similar to those of Tensorflow. Pytorch however, doesn’t require you to define the entire computational graph a priori. This makes Pytorch much easier to debug and understand.

Table of Contents

## Tensors in Pytorch

Tensors are multi-dimensional structures similar to those occurring in NumPy module. PyTorch allows you to define and manipulate tensors in a similar manner to NumPy and also convert NumPy tensors to PyTorch and vice-versa.

PyTorch resides in the torch module. You should be able to run the following code and obtain the version of PyTorch once you finished the installation following this tutorial.

import torch from __future__ import print_function torch.version.__version__

Output:

'1.4.0'

Let us define our first tensor. Using the **torch.tensor()** method is one of the many ways to do this.

x=torch.tensor([[2,3,4],[3,4,5]]) x

Output:

tensor([[2, 3, 4], [3, 4, 5]])

PyTorch has an is_tensor() method that checks whether a variable is a tensor or not.

#Define an array of numbers. x=[10,20,30,40,50] #Check if x is a tensor torch.is_tensor(x)

Output:

False

To convert the array x into a tensor, we need to do the following.

import numpy as np arr=np.array(x) # creating a numpy array from the list we defined earlier c=torch.from_numpy(arr) #create a tensor from the array torch.is_tensor(c)

Output:

True

Other methods of creating tensors are as follows:

#Create a tensor of random normal numbers using randn() function y=torch.randn(3, 3) #Create a tensor of zeros using torch.zeros() a=torch.zeros(2,2) #Create an identity tensor using torch.eye() b=torch.eye(3,4) #torch.linspace() - returns points within a given range in a linear space. lin = torch.linspace(2,10,steps=25) #torch.logspace() - returns points in a logarithmic space log = torch.logspace(start=-10,end=10,steps=10) #torch.rand() - returns specified number of random numbers within the # interval :math:`[0, 1)` random = torch.rand(2, 3) #random permutation of values between 0 to 10 perm = torch.randperm(10) #items between 2 and 10, equally spaced by 2. If the last parameter is # ignored, step size will be 1. seq = torch.arange(2,10,2)

Now let us examine what values are stored in each of the tensors above.

print(y) print(a) print(b) print(lin) print(log) print(random) print(perm) print(seq)

Output:

tensor([[ 0.9039, 0.6291, 1.0795], [ 0.1586, 2.1939, -0.4900], [-0.1909, -0.7503, 1.9355]]) tensor([[0., 0.], [0., 0.]]) tensor([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.]]) tensor([ 2.0000, 2.3333, 2.6667, 3.0000, 3.3333, 3.6667, 4.0000, 4.3333, 4.6667, 5.0000, 5.3333, 5.6667, 6.0000, 6.3333, 6.6667, 7.0000, 7.3333, 7.6667, 8.0000, 8.3333, 8.6667, 9.0000, 9.3333, 9.6667, 10.0000]) tensor([1.0000e-10, 1.6681e-08, 2.7826e-06, 4.6416e-04, 7.7426e-02, 1.2916e+01, 2.1544e+03, 3.5938e+05, 5.9949e+07, 1.0000e+10]) tensor([[ 0.8237, 0.5781, 0.6879], [ 0.3816, 0.7249, 0.0998]]) tensor([9, 1, 4, 5, 8, 2, 7, 6, 3, 0]) tensor([2, 4, 6, 8])

## Restructuring Tensors

It helps a lot of times to be able to modify the shape and structure of tensors to suit our algorithm. PyTorch has several functions that add these flexibilities. First, let us define a tensor to illustrate this.

t1=torch.rand(3,4) t1

Output:

tensor([[0.0800, 0.4987, 0.3337, 0.5070], [0.5671, 0.2567, 0.9441, 0.8524], [0.5343, 0.8898, 0.9503, 0.3900]])

The following code transposes the tensor:

t1.t()

tensor([[0.0800, 0.5671, 0.5343], [0.4987, 0.2567, 0.8898], [0.3337, 0.9441, 0.9503], [0.5070, 0.8524, 0.3900]])

Another alternative is using the transpose() function.

#transpose needs dimension1 and dimension2 as attributes to transpose along the specified directions. t1.transpose(1,0)

tensor([[0.0800, 0.5671, 0.5343], [0.4987, 0.2567, 0.8898], [0.3337, 0.9441, 0.9503], [0.5070, 0.8524, 0.3900]])

Reshaping tensors can be done in multiple ways:

**t1.reshape(a, b)**will return a new tensor with the same data as t1 with size (a, b). This function copies the data to another part of memory, so it can be thought of as a clone.**t1.resize_(a, b)**returns the same tensor with a different shape, but some elements will be removed from the tensor if the new shape results in less number of elements than the original tensor. Note that these elements won’t be removed from the memory. However, if the new shape results in more elements than the tensor, those new elements will remain uninitialized in memory. Underscore shows that the method is performed in place.**t1.view(a, b)**will return a new tensor with the same data as t1 with size (a, b).

All three methods work in the same way.

ty=torch.randn(4,4) t2=ty.reshape(2,8) print(t2)

tensor([[-0.1995, -0.5073, 0.0820, -1.7935, -0.1631, 0.2086, 0.5033, 0.3686], [ 0.0686, 0.0247, -0.4558, -0.1373, 1.1020, 0.6841, 1.1238, -0.4326]])

## Mathematical Operations on Tensors in Pytorch

PyTorch offers a rich list of arithmetic operations that can be performed upon tensors for implementing any algorithm. Let us look at some of those in detail.

### Addition of tensors

Tensor addition can be performed using `torch.add()`

function.

t1 = torch.tensor([2,3,4,5]) t2 = torch.tensor([1,5,9,8]) #Adds t1 and t2 and displays the result on console torch.add(t1,t2) #Adds t1 to t2 and stores the result in t1 t1.add_(t2) #Define a new empty tensor t3=torch.tensor(4) #Add t1 and t2 and store the result in t3 torch.add(t1,t2, out= t3) print(t1) print(t3)

tensor([ 3, 8, 13, 13]) tensor([ 4, 13, 22, 21])

A scalar can be added to every element of tensor in the following manner.

torch.add(t1,5)

tensor([8, 13, 18, 18])

### Multiplication of tensors

The function `torch.mul()`

performs the element-wise multiplication of two tensors.

torch.mul(t1,t2)

tensor([ 3, 40, 117, 104])

### Matrix Multiplication

Matrix and vector multiplication are supported by PyTorch using the `torch.mm(matrix,matrix)`

and `torch.mv(matrix,vector)`

functions.

#Define a vector vec = torch.randn(4) #Define a matrix mat = torch.randn(3,4) print(vec) print(mat)

tensor([ 0.4888, 0.9948, -0.5621, -0.8673]) tensor([[-0.8077, 0.9913, -0.7615, -1.4975], [-0.8250, 0.9404, 0.3022, -0.1274], [-1.2982, 0.3702, 0.5074, 1.4800]])

torch.mv(mat,vec)

tensor([ 2.3182, 0.4729, -1.8350])

Similarly, matrix-matrix multiplication can be done using `torch.mm()`

function.

mat1 = torch.tensor([[2,3],[4,5]]) mat2 = torch.tensor([[4,5],[6,7]]) torch.mm(mat1,mat2)

tensor([[26, 31], [46, 55]])

## Conclusion

We’ve covered the basic working of tensors within PyTorch today. We’ll continue to work with the PyTorch module in the upcoming tutorials and cover further topics within this module. For now, I hope you’ve understood the concept and the basics of the module really well. If you have any questions related to the module, do let us know in the comments below.