Setup & a problem to solve

This is a hands-on course, and the mirror image of Your First PyTorch Project. There, one line — loss.backward() — computed all the gradients and you took it on faith. Here there's no framework and no magic: you write the forward pass, the loss, and backpropagation, in nothing but NumPy. By the end you'll know exactly what that one line was doing.

Setup is refreshingly short: one dependency.

Environment

Make a virtual environment and install NumPy:

python3 -m venv .venv
source .venv/bin/activate
pip install numpy

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install numpy

A problem a straight line can't solve

To prove our network actually learns something, we need a problem that a trivial model can't fake. The classic is XOR: label a point 1 when its two coordinates share a sign (both positive or both negative), and 0 otherwise. That paints the plane like a checkerboard — and no single straight line can separate the two classes. A network with no hidden layer is exactly a straight line, so this problem forces us to build a real one.

Create net.py and generate the data:

import numpy as np

rng = np.random.default_rng(0)   # seeded, so your numbers match this course

N = 400
X = rng.uniform(-1, 1, size=(N, 2))                 # 400 points in a square
y = ((X[:, 0] * X[:, 1]) > 0).astype(float)         # 1 if same sign, else 0
y = y.reshape(-1, 1)                                # column vector, shape (400, 1)

print("X:", X.shape, " y:", y.shape)
print("fraction labelled 1:", y.mean())

python net.py

X: (400, 2)  y: (400, 1)
fraction labelled 1: 0.4875

Roughly half the points are in each class, split across the four quadrants in a checkerboard. If you drew it, you couldn't cut the 1s from the 0s with any single straight line — you'd need a curve. Teaching the network to bend that curve is the whole game. Next, we build the network that will draw it.