Linear Algebra for Machine Learning – Part 2/2

Filed Under: Machine Learning
Linear Algebra 101 (1)

Welcome back to the second part. If you’ve stumbled directly onto this part, then visit the first part here:

Linear Algebra for Machine Learning – part 1/2

So in the last article, we covered the basics of linear algebra like the coordinate systems, random variables, and linear equations.

Linear Algebra for Machine Learning – Working with Matrices

In this part, we’ll focus on the most important aspect of solving a collection of linear equations.

System of Linear Equations

A System of Linear Equations is when we have two or more linear equations working together.

Example: Here are two linear equations:

  • 2x + y = 5
  • −x + y = 2

Together they are a system of linear equations.

There can be many ways to solve linear equations. The substitution method is one of them:

From the first equation, we get: y = 5 - 2x.

Substituting that value in the second equation:

  • -x + 5 - 2x = 2
  • 3x = 3
  • x = 1

But if the number of equations is>4, this method becomes really cumbersome. So we use the other method: Matrix method.

Any system of linear equations can be expressed in the form:

Ax = B


  • A = matrix of coefficients (/coefficient matrix)
  • x = unknown variable column matrix
  • B = constraint / constant matrix

Matrices and Matrix Operations

A rectangular array of m x n numbers (real or complex) in the form of m horizontal lines (called rows) and n vertical lines (called columns), is called a matrix of order m by n, written as m x n matrix.

Such an array is enclosed by [ ] or ( ).

An m x n matrix is usually written as:

Representation Of A Matrix
Representation Of A Matrix

Consider the system:

Example System Of Linear Equations
Example. System Of Linear Equations

which can be represented as :

A x = B

Matrix Of Linear Equations
Matrix Of Linear Equations

Once we solve this: x = 1/6, y = -29/6 and z = 25/6.

1. Matrix Addition/Subtraction

The sum of two matrices can only be found if both matrices have the same dimension. We add matrices by adding corresponding elements.

Matrix Addition
Matrix Addition

Similarly, the difference of two matrices can only be found if both matrices have the same dimension. We subtract matrices by subtracting corresponding elements.

2. Matrix Multiplication

To multiply two matrices, we’ll follow along with the below steps:

  1. In the first array, verify that the number of columns is equal to the number of rows in the second array. (If they are not identical, then these two matrices should not be multiplied.)
  2. Multiply element by element through the first matrix rows and down the second matrix columns and apply the products. (For more information, see the videos below)
  3. Verify that the output is an array with the same number of rows as the first array and the same number of columns as the second array.

Let’s take an example. Multiplying these matrices:

Multiply 1 1
Matrix Multiplication
Multiply 3 1
Multiply 4 1
Multiply 5 1
Multiply 6 1
Multiply 7 1
Multiply 8 1
Multiply 9 1
Multiply 10 1

3. Determinant of a Matrix

Let the determinant of a square matrix A be |A|. For example, consider a matrix:

Example Of Matrix Determinant - Linear Algebra in Machine learning
Example Of Matrix Determinant

We find the determinant as:

  • |A| =4(1×3×1+(−1)×1×3+3×(−3)×3−(3×3×3+3×1×1+1×(−3)×(−1)))
  • =4(3-3-27-(27+3+3))
  • =4×(-60)
  • = -240

4. Transpose of a Matrix

The transpose operation is very important and is one of the easiest.

For any matrix Am x n, the transpose is given by ATn x m and the elements are given by:

AT = [Aji]n x m

5. Inverse of a Matrix

For the inverse, let’s consider a matrix and I’ll walk you through the process.

Inverse 1
Inverse 2 2
Step 1
Inverse 3 2 Linear Algebra in Machine learning
Inverse 4 2
Inverse 5 2
Inverse 6 2
Inverse 7 2
Inverse 8 2
Inverse 9 2
Inverse 10 2
Inverse 11 2
Inverse 12 2
Inverse 13 2

And voila! Now you know 90% of the basics of linear algebra that you need to know for Machine learning. I would, however, suggest you go through the recommended books below.

Books Recommended

Generic selectors
Exact matches only
Search in title
Search in content