Linear Algebra for Machine Learning – Part 1/2

Filed Under: Machine Learning
Linear Algebra 101

In most mathematics programs linear algebra comes in the first or second year, following or along with at least one course in calculus.

Since we don’t have 2 years, we’ll have to make do with 2 articles. I will be completely straightforward about a college curriculum. It is great to build fundamentals and a strong understanding of important principles, and I’m a big proponent of getting the knowledge from the horse’s mouth.

However, if you are short on time and cant afford the time, effort or cost of university, then I will make a suggestion.

After reading this article, go through the books I mention at the end. It’s imperative to have a clear understanding of these topics. Go to the second part of this tutorial here:

Linear Algebra 101 for Machine Learning – part 2/2

Also, enjoy learning. The goal is not the money you’ll earn at the end after learning all the data science and machine learning courses. The goal is to better your own knowledge and skills. The rest will come.

Basics of Linear Algebra for Machine Learning

Linear algebra is a vast topic. So here I’m going to try and cover the necessary parts that will help you get started on your journey.

1. Coordinate System in Linear Algebra

In general, data contains an array of numbers. Spatial data is identical, but numerical detail is often included that helps you to locate it on Earth.

These numbers are part of a coordinate system that provide your data with a frame of reference to place characteristics on the earth’s surface, to match your data with other data, to carry out spatially precise analysis, and to construct charts.

A Coordinate System is the arrangement of reference lines or curves used to determine the positions of points in space.

In 2D, like a piece of paper, the Cartesian (after René Descartes) system is the most common method. Points are designated by their distance from the point of reference along the horizontal ( x) and vertical (y) axes, center, designated (0, 0).

2D Coordinate System
2D Coordinate System

Cartesian coordinates for three (or more) dimensions can also be used.

3D Coordinate System
3D Coordinate System

However, the cartesian x-y-z system is not the only coordinate system. Other popular systems include:

  • Polar coordinate system
  • Cylindrical-coordinate system, and
  • Spherical coordinate systems

2. Vectors and Dimensions

So all data points that look like a single number can be represented on a number line. Identification of points on a line of real numbers using the number line is the easiest definition of a coordinate system.

  • 2D data are represented in the form (x,y) and can be represented on two axes.
  • 3D data are represented in the form (x,y,z) and can be represented on three axes.
  • Data is stored as a finite ordered list (sequence) of elements, and are known as vectors/tuples.

The number of elements inside a vector is termed as the dimensionality or dimension of a vector.

A vector can therefore be any collection of numerical data in a dataset. The vector (1,2,3,4) is known to be a four-dimensional (4D) vector, and even though it cannot be properly visualized in 3D space, knowing this will help you know that the data can be of several hundred dimensions based on the number of features in data.

Data How It Looks
Data – How It Looks

For example, bioinformatics datasets for genome sequences have hundreds of features. Similarly, text data when vectorized has a minimum of 3000 features.

3. Random Variables

A random variable can be defined as a functional mapping from the sample space of an event to the real number line.

Random Variable Mapping
Random Variable Mapping

A random variable in Linear algebra may be either continuous or discrete.

Discrete random variables take on a countable number of distinct values. Consider an experiment where three times a coin is flipped.

If X indicates the number of times the heads of the coin appear, then X is a discrete random variable that can only be 0, 1, 2, 3 (from no heads to all heads in three consecutive coin tosses). No other value for X is feasible.

Continuous random variables can represent any value within a specified range or interval and can take on an infinite number of possible values. An example of a continuous random variable would be an experiment that involves measuring the amount of rainfall in a city over a year or the average height of a random group of 25 people.

Linear Equations

The subject of algebra arose from studying equations.

For example, we might want to find all the real numbers x such that x = x2 − 1.

To solve, we could rewrite our equation as x2 −x−6 = 0 and then factor its left-hand side. This would tell us that (x − 3)(x + 2) = 0. So we would conclude that either x = 3 or x = −2 since either x − 3 or x + 2 has to be zero.

However, finding the roots of a polynomial is a nonlinear problem, whereas the topic to be studied here is the theory of linear equations.

The simplest linear equation is the equation ax = b. The letter x is the
variable, and a and b are fixed numbers. For example, consider 4x = 3.

The solution is x = 3/4.

Definition. If x1, x2, . . . xn are variables and a1, a2, . . . an and c are
fixed real numbers, then the equation:

a1x1 + a2x2 + · · · + anxn = c

is said to be a linear equation. The ai are the coefficients, the xi are the variables and c is the constant.

Some examples of linear equations are: (Solve them for your pactice)

  • Find the solution n to the equation n + 2 = 6
  • Solve the linear equation 3-a=2a.
  • Find c, if 5c - 2 = 33.
  • Find solution b to the equation b/3 = 3.
  • Solve an equation: m+10=3m.
  • If 5x+12=3x-24, determine the value of x.

Next, we’ll need a whole article for matrices specifically. Go to the second part of this tutorial here:

Linear Algebra 101 for Machine Learning – part 2/2

Book Recommendations to Learn Linear Algebra

  • Schaum’s Outline of Linear Algebra, Sixth Edition (Schaum’s Outlines) (If you buy ONE book, this is definitely it.)
  • Introduction to Linear Algebra, Fifth Edition (Gilbert Strang)
  • Linear Algebra and Its Applications by David Lay, Steven Lay
Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors