AI ProgrammingAcademy
Back to Samples

Optional Math: Calculus for Professional

Learn derivatives, the chain rule, partial derivatives and gradient descent with small worked examples before Professional model training. Optional support only—not a prerequisite for exams or certificates.

Learning Objectives

  • Differentiate simple functions

    Use constant, sum and power rules and check a derivative numerically.

  • Apply the chain rule

    Trace how a change travels through a composed prediction and loss.

  • Calculate partial derivatives

    Find the gradient with respect to a model weight and bias.

  • Connect gradients to training

    Update parameters and verify whether loss decreases.

  • Interpret integration

    Distinguish accumulated quantities from local rates of change.

Core Concepts

1. From average slope to derivative

Average slope is the change in output divided by the change in input. A derivative is the limiting local slope as the input step approaches zero. For f(x) = x squared, f-prime(x) = 2x, so at x = 3 the slope is 6. A central numerical difference compares f(x+h) and f(x-h). It is a check, not an exact symbolic proof; excessively small steps can suffer rounding error.

def f(x):
    return x ** 2
x, h = 3.0, 0.001
approximation = (f(x + h) - f(x - h)) / (2 * h)
print(round(approximation, 4))  # 6.0

2. Derivative rules you can use

A constant has derivative zero. The derivative of x to the power n is n times x to the power (n-1). Differentiate a sum term by term; constant multipliers stay. For f(x) = 3x squared + 2x + 5, the derivative is 6x + 2. At x = 2 this is 14. A positive derivative means increasing locally, not that the function is positive.

def f(x):
    return 3 * x ** 2 + 2 * x + 5
def derivative(x):
    return 6 * x + 2
print(f(2))           # 21
print(derivative(2))  # 14

3. Chain rule: follow the dependency

For a composition f(g(x)), multiply the outer derivative evaluated at g(x) by the inner derivative. Let u = 2x + 1 and L = u squared. Then dL/du = 2u and du/dx = 2, so dL/dx = 4(2x + 1). At x = 3 it is 28. Backpropagation applies this same rule through a model to measure how each parameter changes the final loss.

x = 3
u = 2 * x + 1
dL_du = 2 * u
du_dx = 2
print(dL_du * du_dx)  # 28

4. Partial derivatives and a model gradient

For prediction p = wx + b and loss L = (p-y) squared, treat x and y as fixed training data. When differentiating with respect to w, hold b fixed: dL/dw = 2(p-y)x. When differentiating with respect to b, hold w fixed: dL/db = 2(p-y). Together they form the gradient [dL/dw, dL/db]. For x=2, y=5, w=1, b=0, prediction is 2, error is -3, and the gradient is [-12, -6].

x, y = 2.0, 5.0
w, b = 1.0, 0.0
error = w * x + b - y
dw = 2 * error * x
db = 2 * error
print(error ** 2)  # 9.0
print(dw, db)      # -12.0 -6.0

5. Gradient descent and batch averages

Update every parameter by subtracting learning rate times its gradient. Calculate all gradients at the old parameter values, then update together. With rate 0.1 in the previous example, w becomes 2.2 and b becomes 0.6, giving prediction 5 and loss zero on that one example. This is not proof of generalization. For a batch, mean squared error averages the squared errors, and each gradient is averaged over the same examples. Large steps may diverge; use validation data and monitor loss.

x, y = 2.0, 5.0
w, b = 1.0, 0.0
rate = 0.1
error = w * x + b - y
dw, db = 2 * error * x, 2 * error
w, b = w - rate * dw, b - rate * db
print(round(w, 2), round(b, 2))  # 2.2 0.6
print(round((w * x + b - y) ** 2, 4))  # 0.0

What calculus does not promise: A zero gradient need not be a minimum: it can be a maximum or saddle point. Non-convex neural-network training is not guaranteed to find a global optimum.

6. Integrals accumulate rather than differentiate

Differentiation measures a local rate; integration accumulates a rate over an interval. At constant speed 3 metres per second for 4 seconds, the distance is 12 metres. An antiderivative of x is x squared divided by 2, so the integral of x from 0 to 2 is 2. Continuous probability densities also use area: probability is an integral over an interval, not the density value at one point. Basic model gradient training relies primarily on derivatives, not hand-calculated integrals.

speed, duration = 3, 4
print(speed * duration)  # 12
def antiderivative(x):
    return x ** 2 / 2
print(antiderivative(2) - antiderivative(0))  # 2.0

Continue in Professional: Use this lesson before Calculus and Optimization and the deeper machine-learning chain-rule and training lessons. Cloud deployment itself does not require differentiating models by hand.

Practice Exercise

Train one batch and verify the weight gradient

Use examples (x=1, y=3) and (x=2, y=5), starting w=1 and b=0. Calculate mean squared error and the average gradients. Update both parameters with learning rate 0.1. Use a central difference to check the original weight gradient. Expected loss: 6.5 before and 0.65 after; parameters: w=1.8 and b=0.5; original weight gradient: -8.
examples = [(1.0, 3.0), (2.0, 5.0)]
w, b = 1.0, 0.0
rate = 0.1
# Define mean squared error
# Calculate both average gradients before updating
# Check dw with a central difference, then update and print results

Concept Quiz

Test your understanding with this quiz. Select the best answer for each question.

1. What is the derivative of 3x squared + 2x + 5?

2. For L = (2x + 1) squared, what is dL/dx at x=3?

3. When calculating a partial derivative with respect to w, what happens to b?

4. A gradient is -8 and learning rate is 0.1. How does gradient descent change the parameter?

5. Does a zero gradient guarantee a minimum?

6. What does integrating a speed over time give?

Are there any issues that Customer Support should be aware of?

Help optimize our AI programming lessons to ensure student success!

Sign in to send a comment to Customer Support.

Hide Icons