Learn derivatives, the chain rule, partial derivatives and gradient descent with small worked examples before Professional model training. Optional support only—not a prerequisite for exams or certificates.
Use constant, sum and power rules and check a derivative numerically.
Trace how a change travels through a composed prediction and loss.
Find the gradient with respect to a model weight and bias.
Update parameters and verify whether loss decreases.
Distinguish accumulated quantities from local rates of change.
Average slope is the change in output divided by the change in input. A derivative is the limiting local slope as the input step approaches zero. For f(x) = x squared, f-prime(x) = 2x, so at x = 3 the slope is 6. A central numerical difference compares f(x+h) and f(x-h). It is a check, not an exact symbolic proof; excessively small steps can suffer rounding error.
def f(x):
return x ** 2
x, h = 3.0, 0.001
approximation = (f(x + h) - f(x - h)) / (2 * h)
print(round(approximation, 4)) # 6.0A constant has derivative zero. The derivative of x to the power n is n times x to the power (n-1). Differentiate a sum term by term; constant multipliers stay. For f(x) = 3x squared + 2x + 5, the derivative is 6x + 2. At x = 2 this is 14. A positive derivative means increasing locally, not that the function is positive.
def f(x):
return 3 * x ** 2 + 2 * x + 5
def derivative(x):
return 6 * x + 2
print(f(2)) # 21
print(derivative(2)) # 14For a composition f(g(x)), multiply the outer derivative evaluated at g(x) by the inner derivative. Let u = 2x + 1 and L = u squared. Then dL/du = 2u and du/dx = 2, so dL/dx = 4(2x + 1). At x = 3 it is 28. Backpropagation applies this same rule through a model to measure how each parameter changes the final loss.
x = 3
u = 2 * x + 1
dL_du = 2 * u
du_dx = 2
print(dL_du * du_dx) # 28For prediction p = wx + b and loss L = (p-y) squared, treat x and y as fixed training data. When differentiating with respect to w, hold b fixed: dL/dw = 2(p-y)x. When differentiating with respect to b, hold w fixed: dL/db = 2(p-y). Together they form the gradient [dL/dw, dL/db]. For x=2, y=5, w=1, b=0, prediction is 2, error is -3, and the gradient is [-12, -6].
x, y = 2.0, 5.0
w, b = 1.0, 0.0
error = w * x + b - y
dw = 2 * error * x
db = 2 * error
print(error ** 2) # 9.0
print(dw, db) # -12.0 -6.0Update every parameter by subtracting learning rate times its gradient. Calculate all gradients at the old parameter values, then update together. With rate 0.1 in the previous example, w becomes 2.2 and b becomes 0.6, giving prediction 5 and loss zero on that one example. This is not proof of generalization. For a batch, mean squared error averages the squared errors, and each gradient is averaged over the same examples. Large steps may diverge; use validation data and monitor loss.
x, y = 2.0, 5.0
w, b = 1.0, 0.0
rate = 0.1
error = w * x + b - y
dw, db = 2 * error * x, 2 * error
w, b = w - rate * dw, b - rate * db
print(round(w, 2), round(b, 2)) # 2.2 0.6
print(round((w * x + b - y) ** 2, 4)) # 0.0What calculus does not promise: A zero gradient need not be a minimum: it can be a maximum or saddle point. Non-convex neural-network training is not guaranteed to find a global optimum.
Differentiation measures a local rate; integration accumulates a rate over an interval. At constant speed 3 metres per second for 4 seconds, the distance is 12 metres. An antiderivative of x is x squared divided by 2, so the integral of x from 0 to 2 is 2. Continuous probability densities also use area: probability is an integral over an interval, not the density value at one point. Basic model gradient training relies primarily on derivatives, not hand-calculated integrals.
speed, duration = 3, 4
print(speed * duration) # 12
def antiderivative(x):
return x ** 2 / 2
print(antiderivative(2) - antiderivative(0)) # 2.0Continue in Professional: Use this lesson before Calculus and Optimization and the deeper machine-learning chain-rule and training lessons. Cloud deployment itself does not require differentiating models by hand.
examples = [(1.0, 3.0), (2.0, 5.0)] w, b = 1.0, 0.0 rate = 0.1 # Define mean squared error # Calculate both average gradients before updating # Check dw with a central difference, then update and print results
Test your understanding with this quiz. Select the best answer for each question.
Help optimize our AI programming lessons to ensure student success!
Sign in to send a comment to Customer Support.