The backward pass

This is the chapter the whole course is named for. We know each weight's gradient is a product of local slopes (the chain rule). Backpropagation is the clever way to compute all of them at once: run the network forward to get the loss, then sweep the gradient backward through the layers, reusing work at every step. One forward pass, one backward pass, and every weight knows how to improve.

Scroll to run it — forward, then back.

Why backward, and why it's fast

The direction matters. The chain rule for a deep weight ends with ∂L/∂a — a term about the layers near the loss. So if you compute gradients starting from the loss and moving backward, each layer can hand the layer before it exactly the piece it needs. Nothing is recomputed.

• Each layer receives the gradient of the loss with respect to its output.
• It uses that (plus its own local slopes) to compute the gradient for its weights.
• And it produces the gradient with respect to its input — which is precisely what the previous layer needs. Pass it back and repeat.

What you're left with

After the backward pass, every weight in the network is tagged with a number: its ∂L/∂w, the slope telling it which way (and how hard) to move to reduce the loss. The hard part — figuring out the direction — is done.

All that remains is to actually take the step, and do it again and again. Next: take a step, repeat.