AI ProgrammingAcademy
Back to Samples

Optional Math: Change and Optimization

Get an intuitive first look at slopes and model improvement, without starting with formal calculus. Optional support only—not a prerequisite for exams or certificates.

Learning Objectives

  • Calculate a rate of change

    Divide the change in output by the change in input.

  • Understand loss

    Treat loss as a measure of how far a prediction is from a target.

  • Follow one optimization step

    See why step size and direction matter when improving a model.

Core Concepts

1. Change divided by change

If distance rises from 10 km to 30 km over 2 hours, the average speed is (30 - 10) / 2 = 10 km per hour. A graph slope uses the same idea: change in output divided by change in input. An input change of zero cannot be used as a divisor.

distance_change = 30 - 10
time_change = 2
print(distance_change / time_change)  # 10.0

2. A derivative is a local slope

A straight line has the same slope everywhere; a curve does not. For f(x) = x², the slope at x = 3 is 6. A nearby numerical estimate uses a small step h: (f(x + h) - f(x)) / h. With h = 0.01, the estimate is about 6.01. Smaller steps can help, but extremely small steps introduce rounding issues.

x = 3
h = 0.01
approximate_slope = ((x + h) ** 2 - x ** 2) / h
print(round(approximate_slope, 2))  # 6.01

3. Loss measures error

Suppose a toy model predicts a single number w and the target is 3. Define loss as (w - 3)². At w = 5 the loss is 4; at w = 3 it is zero. Squaring makes errors nonnegative. Real models use many examples and other loss functions too.

w = 5
loss = (w - 3) ** 2
print(loss)  # 4

4. Take a small downhill step

For this particular loss, the derivative is 2 × (w - 3). Gradient descent updates w by subtracting learning_rate × derivative. Starting at w = 5 with rate 0.1 gives derivative 4 and new w = 4.6, which reduces loss to 2.56. A large step can overshoot; complex losses need not have one global minimum.

w = 5.0
learning_rate = 0.1
gradient = 2 * (w - 3)
w = w - learning_rate * gradient
print(round(w, 2))           # 4.6
print(round((w - 3)**2, 2))  # 2.56

When this helps: Return here before Professional calculus and model training. You do not need to master derivatives before starting Foundations programming.

Practice Exercise

Improve a toy prediction

Start with w = 7 and target = 3. Use learning rate 0.25 and gradient 2 × (w - target). Print the loss before the update, the updated w and the new loss.
w = 7.0
target = 3.0
learning_rate = 0.25
# Calculate loss, update w, then calculate the new loss

Concept Quiz

Test your understanding with this quiz. Select the best answer for each question.

1. An output increases by 12 while the input increases by 3. What is the average rate of change?

2. Why can a learning rate that is too large cause problems?

Are there any issues that Customer Support should be aware of?

Help optimize our AI programming lessons to ensure student success!

Sign in to send a comment to Customer Support.

Hide Icons