AI ProgrammingAcademy
Back to Samples

Linear Algebra for AI: Basic Level

Understand the numbers, lists, and grids behind AI.

About This Lesson

Linear algebra is the study of number lists and rules that change them. AI uses these lists to represent information and these rules to learn patterns. Attention helps a chatbot choose which words to focus on. LoRA adapts a model by learning a small change to a large table of weights. Start small and build one step at a time.

Free preview — no account needed. This is the Basic level of a Mathematical Foundations lesson from the AI Professional Series. The Intermediate and Advanced levels, with 20 more practice exercises and a Concept Quiz, are part of AI Professional. Optional practice only: it does not affect course totals, exams or certificates.

See AI Professional plans

This is the Basic level. There are 10 practice exercises, each with a hint and a full worked solution.

Basic Level: Learning Objectives

Start here if math feels unfamiliar. Learn to describe, combine, measure, and compare lists of numbers.

  • Tell a single number from a vector.
  • Add, subtract, and scale vectors.
  • Calculate vector length and a dot product.

Basic Level: Core Concepts

1. Scalars and vectors

A scalar is one number. A vector is an ordered list of numbers. Each number in the list is called a component. Order matters because each position describes a particular feature, or measured property.

In everyday words: One item's price is a scalar. A shopping list of quantities is a vector, with one position per item.

Worked example: A robot records 3 steps east and 4 steps north. Represent this movement and count its components.

  1. Put the east measurement first: 3.
  2. Put the north measurement second: 4.
  3. Write the ordered list [3, 4].
  4. Count the entries: 2.

Answer: The vector is [3, 4]. It has 2 components; each component is a scalar.

Why it matters in AI: AI often represents one item as a vector. Its positions might hold age, height, or other measured properties.

2. Vector addition and subtraction

Add vectors by adding numbers in matching positions. Subtract them by subtracting matching positions in the same order. The vectors must have the same number of components, and matching positions should describe the same feature.

In everyday words: Combine two shopping lists by adding the quantities for each item. Subtract what you already own to find what is still needed.

[a, b] + [c, d] = [a + c, b + d]; [a, b] − [c, d] = [a − c, b − d]

Read it as: Add or subtract the first entries, then add or subtract the second entries.

Worked example: Add and subtract u = [5, 2] and v = [1, 3].

  1. For the sum, calculate 5 + 1 = 6 and 2 + 3 = 5.
  2. For u minus v, calculate 5 − 1 = 4 and 2 − 3 = −1.

Answer: u + v = [6, 5]; u − v = [4, −1].

Why it matters in AI: AI uses vector subtraction to describe differences between predictions and targets. Addition combines updates with existing values.

3. Multiplying a vector by a number

Scaling means multiplying every component by the same scalar. A positive scalar stretches or shrinks a vector without reversing it. A negative scalar also reverses its direction. Multiplying by zero gives a vector of zeros.

In everyday words: Doubling a recipe doubles every ingredient, not just the first one.

s × [a, b] = [s × a, s × b]

Read it as: Multiply each entry by the scalar s.

Worked example: Scale [2, −3] by 3 and then, separately, by −1.

  1. Multiply by 3: 3 × 2 = 6 and 3 × (−3) = −9.
  2. Multiply the original vector by −1: (−1) × 2 = −2 and (−1) × (−3) = 3.

Answer: The results are [6, −9] and [−2, 3].

Why it matters in AI: A learning rate is a scalar that controls the size of a model update. Scaling makes the update larger or smaller.

4. Vector length, also called a norm

A norm measures a vector's size. Here we use the Euclidean norm, which is straight-line distance from the all-zero vector. A square multiplies a number by itself. A square root reverses squaring and is nonnegative here.

In everyday words: Walking 3 blocks east and 4 north leaves you 5 blocks from your starting point in a straight line.

||v|| = √(v₁² + v₂² + … + vₙ²)

Read it as: Square every component, add the squares, and take the square root. The small labels mark positions in the vector.

Worked example: Find the length of [3, 4].

  1. Square each entry: 3² = 9 and 4² = 16.
  2. Add the squares: 9 + 16 = 25.
  3. Take the nonnegative square root: √25 = 5.

Answer: The norm is 5.

Why it matters in AI: AI uses norms to measure the size of errors and updates. A large norm means a large overall size.

5. Dot product

The dot product combines two equal-length vectors into one scalar. Multiply matching components, then add the products. It can act as a weighted total: one vector holds values and the other holds how strongly each value counts.

In everyday words: Multiply each item's quantity by its price, then add the costs to get a bill.

u · v = u₁ × v₁ + u₂ × v₂ + … + uₙ × vₙ

Read it as: Multiply the entries at each matching position and add all those products.

Worked example: Find [2, 3] · [4, 1].

  1. Multiply the first pair: 2 × 4 = 8.
  2. Multiply the second pair: 3 × 1 = 3.
  3. Add the products: 8 + 3 = 11.

Answer: The dot product is 11.

import numpy as np
u = np.array([2, 3])
v = np.array([4, 1])
print(np.dot(u, v))  # 11

Why it matters in AI: A neuron, a small calculation unit in a neural network, uses a dot product to combine its inputs with learned weights.

Basic Level: Practice Exercises

Try each one on paper first. Use the hint if you get stuck, then check the full solution.

  1. Exercise 1. Which is a scalar, 7 or [7, 2]? How many components does [7, 2] have?

  2. Exercise 2. Calculate [1, 2] + [3, 4].

  3. Exercise 3. Calculate [5, 3] − [2, 4].

  4. Exercise 4. Calculate 3 × [2, 1, 0].

  5. Exercise 5. Calculate −2 × [1, −3].

  6. Exercise 6. Calculate 2 × [1, 3] + [4, −1].

  7. Exercise 7. Find the norm of [0, −5].

  8. Exercise 8. Find the norm of [1, 2, 2].

  9. Exercise 9. Calculate [2, −1, 3] · [1, 4, 2].

  10. Exercise 10. Let u = [1, 2] and v = [3, 1]. Find w = 2 × u − v, then find w · v and ||w||.

Hide Icons