# PyTorch Tensor – A Detailed Overview

Filed Under: PyTorch In this PyTorch tutorial, we’ll discuss PyTorch Tensor, which are the building blocks of this Deep Learning Framework.

Let’s get started!

## PyTorch Tensor

Have you worked with Python numpy before? If yes, then this section is going to be very simple for you! Even if you don’t have experience with numpy, you can seamlessly transition between PyTorch and NumPy!

A Tensor in PyTorch is similar to `numpy` arrays, with the additional flexibility of using a GPU for calculations.

### 1. 2D Pytorch Tensor

Imagine a tensor as an array of numbers, with a potentially arbitrary number of dimensions. The only difference between a Tensor and a multidimensional array in C/C++/Java is that the size of all the columns in a dimension is the same.

For example, the below can be a valid representation of a 2 Dimensional Tensor.

```[[1 2 3 4],
[5 6 7 8]]
```

Note, however, that the below example is NOT a valid example, since Tensors are not jagged arrays.

```[[1 2 3 4],
[5 6 7]]
```

PyTorch Tensors are really convenient for programmers, since they are almost the same as `numpy` arrays.

There are a couple of differences to `numpy` methods, though, so it is advised that you also refer the official Documentation for further information.

### 2. Initializing an Empty PyTorch Tensor

Let’s consider the below example, which initializes an empty Tensor.

```import torch
# Creates a 3 x 2 matrix which is empty
a = torch.empty(3, 2)
```

An empty tensor does NOT mean that it does not contain anything. It’s just that there is memory allocated for it.

```import torch
# Creates a 3 x 2 matrix which is empty
a = torch.empty(3, 2)
print(a)

# Create a zero initialized float tensor
b = torch.zeros(3, 2, dtype=torch.float32)
print(b)
```

Output

```tensor([[3.4655e-37, 0.0000e+00],
[4.4842e-44, 0.0000e+00],
[       nan, 6.1657e-44]])
tensor([[0., 0.],
[0., 0.],
[0., 0.]])
```

The first tensor is a result of PyTorch simply allocating memory for the tensor. Whatever previous content in the memory is not erased.

The second tensor is filled with zeros, since PyTorch allocates memory and zero-initializes the tensor elements.

Notice the similarity to `numpy.empty()` and `numpy.zeros()`. This is because PyTorch is designed to replace `numpy`, since the GPU is available.

### 3. Finding PyTorch Tensor Size

Let’s create a basic tensor and determine its size.

```import torch
# Create a tensor from data
c = torch.tensor([[3.2 , 1.6, 2], [1.3, 2.5 , 6.9]])
print(c)
```

Output

```tensor([[3.2000, 1.6000, 2.0000],
[1.3000, 2.5000, 6.9000]])
```

To get the size of the tensor, we can use tensor.size()

```print(c.size())
```

Output

```torch.Size([2, 3])
```

## PyTorch Tensor Operations

Like `numpy`, PyTorch supports similar tensor operations.

The summary is given in the below code block.

### 1. Basic Mathematical Operations on Tensors

```import torch
# Tensor Operations
x = torch.tensor([[2, 3, 4], [5, 6, 7]])
y = torch.tensor([[2, 3, 4], [1.3, 2.6, 3.9]])

print(x + y)
# We can also use torch.add()
print(x + y == torch.add(x, y))

# Subtraction
print(x - y)
# We can also use torch.sub()
print(x-y == torch.sub(x, y))
```

Output

```tensor([[ 4.0000,  6.0000,  8.0000],
[ 6.3000,  8.6000, 10.9000]])
tensor([[True, True, True],
[True, True, True]])
tensor([[0.0000, 0.0000, 0.0000],
[3.7000, 3.4000, 3.1000]])
tensor([[True, True, True],
[True, True, True]])
```

We can also assign the result to a tensor. Add the following code snippet to the code above.

```# We can assign the output to a tensor
z = torch.zeros(x.shape)
print(z)
```

Output

```tensor([[ 4.0000,  6.0000,  8.0000],
[ 6.3000,  8.6000, 10.9000]])
```

### 2. Inline Addition and Subtraction with PyTorch Tensor

PyTorch also supports in-place operations like addition and subtraction, when suffixed with an underscore (_). Let’s continue on with the same variables from the operations summary code above.

```# In-place addition
```

Output

```Before In-Place Addition: tensor([[2.0000, 3.0000, 4.0000],
[1.3000, 2.6000, 3.9000]])
After addition: tensor([[ 4.0000,  6.0000,  8.0000],
[ 6.3000,  8.6000, 10.9000]])
```

### 3. Accessing Tensor Index

We can also use numpy based indexing in PyTorch

```# Use numpy slices for indexing
print(y[:, 1]
```

Output

```tensor([6.0000, 8.6000])
```

## Reshape a PyTorch Tensor

Similar to `numpy`, we can use `torch.reshape()` to reshape a tensor. We can also use `tensor.view()` to achieve the same functionality.

```import torch
x = torch.randn(5, 3)
# Return a view of the x, but only having
# one dimension
y = x.view(5 * 3)

print('Size of x:', x.size())
print('Size of y:', y.size())

print(x)
print(y)

# Get back the original tensor with reshape()
z = y.reshape(5, 3)
print(z)
```

Output

```Size of x: torch.Size([5, 3])
Size of y: torch.Size()

tensor([[ 0.3224,  0.1021, -1.4290],
[-0.3559,  0.2912, -0.1044],
[ 0.3652,  2.3112,  1.4784],
[-0.9630, -0.2499, -1.3288],
[-0.0667, -0.2910, -0.6420]])

tensor([ 0.3224,  0.1021, -1.4290, -0.3559,  0.2912, -0.1044,  0.3652,  2.3112,
1.4784, -0.9630, -0.2499, -1.3288, -0.0667, -0.2910, -0.6420])

tensor([[ 0.3224,  0.1021, -1.4290],
[-0.3559,  0.2912, -0.1044],
[ 0.3652,  2.3112,  1.4784],
[-0.9630, -0.2499, -1.3288],
[-0.0667, -0.2910, -0.6420]])
```

The list of all Tensor Operations is available in PyTorch’s Documentation.

## PyTorch – NumPy Bridge

We can convert PyTorch tensors to numpy arrays and vice-versa pretty easily.

PyTorch is designed in such a way that a Torch Tensor on the CPU and the corresponding `numpy` array will have the same memory location. So if you change one of them, the other one will automatically be changed.

To prove this, let’s test it using the `torch.numpy()` and the `torch.from_numpy()` methods.

torch.numpy() is used to convert a Tensor to a numpy array, and torch.from_numpy() will do the reverse.

```import torch
# We also need to import numpy to declare numpy arrays
import numpy as np

a = torch.tensor([[1, 2, 3], [4, 5, 6]])
print('Original Tensor:', a)

b = a.numpy()
print('Tensor to a numpy array:', b)

```

Output

```Original Tensor: tensor([[1, 2, 3],
[4, 5, 6]])
Tensor to a numpy array: [[1 2 3]
[4 5 6]]
Tensor after addition: tensor([[3, 4, 5],
[6, 7, 8]])
Numpy Array after addition: [[3 4 5]
[6 7 8]]
```

Indeed, the numpy array has also changed it’s value!

Let’s do the reverse as well

```import torch
import numpy as np

c = np.array([[4, 5, 6], [7, 8, 9]])
print('Numpy array:', c)

# Convert to a tensor
d = torch.from_numpy(c)
print('Tensor from the array:', d)

# Add 3 to each element in the numpy array

```

Output

```Numpy array: [[4 5 6]
[7 8 9]]
Tensor from the array: tensor([[4, 5, 6],
[7, 8, 9]])
Numpy array after addition: [[ 7  8  9]
[10 11 12]]
Tensor after addition: tensor([[ 7,  8,  9],
[10, 11, 12]])
```

NOTE: If you do not use the `numpy` in-place addition using `a += 3` or `np.add(out=a)`, then the Tensor will not reflect the changes in the numpy array.

For example, if you try this:

```c = np.add(c, 3)
```

Since you’re using `=`, this means that Python will create a new object and assign that new object to the name called `c`. So the original memory location is still unchanged.

## Use the CUDA GPU with a PyTorch Tensor

We can make the NVIDIA CUDA GPU perform the computations and have a speedup, by moving the tensor to the GPU.

NOTE: This applies only if you have an NVIDIA GPU with CUDA enabled. If you’re not sure of what these terms are, I would advise you to search online.

We can check if we have the GPU available for PyTorch using `torch.cuda.is_available()`

```import torch
if torch.cuda.is_available():
print('Your device is supported. We can use the GPU for PyTorch!')
else:
print('Your GPU is either not supported by PyTorch or you haven't installed the GPU version')
```

For me, it is available, so just make sure you install CUDA before proceeding further if your laptop supports it.

We can move a tensor from the CPU to the GPU using `tensor.to(device)`, where `device` is a device object.

This can be `torch.device("cuda")`, or simply `cpu`.

```import torch
x = torch.tensor([1, 2, 3], dtype=torch.long)

if torch.cuda.is_available():
print('CUDA is available')
# Create a CUDA Device object
device = torch.device("cuda")

# Create a tensor from x and store on the GPU
y = torch.ones_like(x, device=device)

# Move the tensor from CPU to GPU
x = x.to(device)

# This is done on the GPU
z = x + y
print(z)

# Move back to CPU and also change dtype
print(z.to("cpu", torch.double))
print(z)
else:
print('CUDA is not available')
```

Output

```CUDA is available
tensor([2, 3, 4], device='cuda:0')
tensor([2., 3., 4.], dtype=torch.float64)
tensor([2, 3, 4], device='cuda:0')
```

As you can see, the output does show that our program is now being run on the GPU instead!

## Conclusion

In this article, we learned about using Tensors in PyTorch. Feel free to ask any doubts or even suggestions/corrections in the comment section below!

We’ll be covering more in our upcoming PyTorch tutorials. Stay tuned!

## References

close
Generic selectors
Exact matches only
Search in title
Search in content