AI ProgrammingAcademy
Back to Samples

Optional Math: Linear Algebra for Professional

Prepare for Professional model predictions, embeddings and dimensionality reduction. Begin with Vectors and Matrices if these ideas are new. Optional support only—not a prerequisite for exams or certificates.

Learning Objectives

  • Calculate vector operations

    Add and scale vectors, find their length and interpret cosine similarity.

  • Multiply matrices

    Check dimensions and calculate a batch of model predictions.

  • Solve a small linear system

    Use elimination and recognize when a system has no unique solution.

  • Interpret eigenvectors

    Understand preserved directions and the connection to principal component analysis.

Core Concepts

1. Vectors, scaling and length

A vector is an ordered list of numbers. Add matching positions: [1, 2] + [3, 4] = [4, 6]. Multiplying by 2 gives [2, 4]. The Euclidean norm is the square root of the sum of squared entries; [3, 4] has length 5. Normalizing divides each entry by the length, producing a unit vector. The zero vector cannot be normalized.

from math import sqrt
u = [1, 2]
v = [3, 4]
print([a + b for a, b in zip(u, v)])
print([2 * a for a in u])
length = sqrt(sum(a * a for a in v))
print(length)
if length > 0:
    print([a / length for a in v])
# [4, 6]
# [2, 4]
# 5.0
# [0.6, 0.8]

2. Dot products and cosine similarity

The dot product adds pairwise products. Cosine similarity divides this result by the product of both vector lengths. For nonzero vectors it ranges from -1 to 1: 1 means the same direction, 0 means perpendicular, and -1 means opposite. Embedding search often compares directions this way. A similarity score is not a probability or proof that two passages mean the same thing.

from math import sqrt
u = [1, 0]
v = [1, 1]
dot = sum(a * b for a, b in zip(u, v))
nu = sqrt(sum(a * a for a in u))
nv = sqrt(sum(b * b for b in v))
if nu == 0 or nv == 0:
    raise ValueError("Cosine needs nonzero vectors")
print(round(dot / (nu * nv), 4))  # 0.7071

3. Matrix multiplication, step by step

An m-by-n matrix can multiply an n-by-p matrix, producing an m-by-p result. Each output entry is one row dotted with one column. Order matters: A times B generally differs from B times A. If X stores examples in rows and features in columns, X times a weight vector gives one prediction per example. Add a bias after each weighted sum.

X = [[1, 2], [3, 4]]
W = [[2, 0], [1, 3]]
columns = list(zip(*W))
result = [[sum(a * b for a, b in zip(row, col))
           for col in columns] for row in X]
print(result)  # [[4, 6], [10, 12]]
weights = [2, 1]
bias = 0.5
print([sum(x * w for x, w in zip(row, weights)) + bias
       for row in X])  # [4.5, 10.5]

Check dimensions first: These examples use known, compatible rectangular matrices. In your projects validate every row length before using zip, which silently stops at the shortest input.

4. Transpose and feature tables

Transposing swaps rows and columns. A 3-by-2 feature table becomes 2-by-3. This helps aggregate each feature across examples. Centering subtracts each feature mean. PCA starts from centered data so directions describe variation around the mean rather than distance from the origin.

X = [[1, 2], [3, 4], [5, 6]]
columns = list(zip(*X))
means = [sum(col) / len(col) for col in columns]
centered = [[x - mean for x, mean in zip(row, means)]
            for row in X]
print(means)     # [3.0, 4.0]
print(centered)  # [[-2.0, -2.0], [0.0, 0.0], [2.0, 2.0]]

5. Solve equations and recognize dependence

Solve x + y = 3 and 2x + y = 4 by subtracting the first equation from the second: x = 1, then y = 2. For a 2-by-2 matrix [[a, b], [c, d]], the determinant is ad - bc. A zero determinant means the equations cannot have a unique solution: they may be redundant or inconsistent. In real ML work use a numerical solver rather than explicitly calculating a matrix inverse; near-dependent columns can make results unstable.

a, b, c, d = 1, 1, 2, 1
e, f = 3, 4
determinant = a * d - b * c
if determinant == 0:
    raise ValueError("No unique solution")
x = (e * d - b * f) / determinant
y = (a * f - e * c) / determinant
print(x, y)  # 1.0 2.0

6. Eigenvectors and why PCA uses them

An eigenvector is a nonzero vector whose direction is preserved by a matrix: A v = lambda v. The eigenvalue lambda tells you the scaling, including a direction flip when negative. For [[2, 0], [0, 1]], [1, 0] is an eigenvector with eigenvalue 2. PCA finds eigenvectors of a covariance matrix (or uses singular value decomposition); the largest eigenvalues identify directions containing the most variance. High variance does not automatically mean a feature is useful for your target.

A = [[2, 0], [0, 1]]
v = [1, 0]
av = [sum(a * b for a, b in zip(row, v)) for row in A]
print(av)               # [2, 0]
print([2 * x for x in v])  # [2, 0]

Continue in Professional: Use this preparation before the Professional Linear Algebra for AI and deeper machine-learning math lessons. It supports understanding embeddings and model training; Docker and deployment do not require eigenvector calculations.

Practice Exercise

Predict a batch and compare embedding directions

Use feature rows [1, 2] and [3, 1], weights [2, -1] and bias 0.5. Calculate both predictions. Then calculate cosine similarity between [3, 4] and [4, 3]. Check equal dimensions and nonzero lengths. Expected predictions: [0.5, 5.5]; expected similarity: 0.96.
from math import sqrt
rows = [[1, 2], [3, 1]]
weights = [2, -1]
bias = 0.5
u, v = [3, 4], [4, 3]
# Validate dimensions, calculate predictions and cosine similarity
# Print predictions, then similarity rounded to two decimals

Concept Quiz

Test your understanding with this quiz. Select the best answer for each question.

1. A 4-by-3 matrix multiplies a 3-by-2 matrix. What is the result shape?

2. What is the Euclidean length of [3, 4]?

3. What does cosine similarity 1 say about two nonzero vectors?

4. What does a zero determinant tell you about a 2-by-2 linear system?

5. What makes v an eigenvector of A?

Are there any issues that Customer Support should be aware of?

Help optimize our AI programming lessons to ensure student success!

Sign in to send a comment to Customer Support.

Hide Icons