The training loop

You have all three pieces: forward (guess), bce (how wrong), and backward (which way to nudge each weight). Training just runs them in a loop and takes a small step downhill each time. This is gradient descent, and it's the same loop the PyTorch course ran — except here nothing is hidden.

Forward, loss, backward, nudge

Each weight gets moved a little against its gradient — the direction that lowers the loss. How big a step is the learning rate. Add the loop to net.py:

lr = 0.5
for epoch in range(3001):
  z1, a1, a2 = forward(X)                 # guess
  loss = bce(a2, y)                       # how wrong
  dW1, db1, dW2, db2 = backward(X, y, z1, a1, a2)   # which way

  W1 -= lr * dW1;  b1 -= lr * db1         # nudge every weight downhill
  W2 -= lr * dW2;  b2 -= lr * db2

  if epoch % 500 == 0:
      acc = ((a2 > 0.5) == y).mean()
      print(f"epoch {epoch:4d}  loss {loss:.4f}  acc {acc:.2%}")

That's the entire training algorithm — four operations, repeated. W -= lr * dW is the gradient-descent update: subtract a fraction of the gradient from each weight. Do it a few thousand times and random noise becomes a working classifier.

Run it

python net.py

epoch    0  loss 0.6947  acc 59.50%
epoch  500  loss 0.0663  acc 98.75%
epoch 1000  loss 0.0458  acc 99.25%
epoch 1500  loss 0.0372  acc 99.75%
epoch 2000  loss 0.0320  acc 99.75%
epoch 2500  loss 0.0285  acc 99.75%
epoch 3000  loss 0.0258  acc 99.75%

Watch the two numbers move in opposite directions: the loss falls from 0.69 (a coin flip) toward zero, and accuracy climbs from a hair above chance to 99.75%. The network taught itself to carve the checkerboard that no straight line could — using only the gradients you derived by hand last chapter.

It works. In the final chapter we'll see the curve it learned, and connect what you built back to the frameworks.