Train once, run forever

You hear "training" a lot when people talk about AI — the months, the giant GPU clusters, the eye-watering bills. But that's only half the story, and arguably the smaller half. The other half is inference: every single time the finished model actually runs to answer a question, recognize a face, or write a sentence.

If training is the model going to school, inference is it showing up to work — every day, for every user, forever.

Scroll to see the two phases side by side.

Two very different jobs

The words get muddled, so here's the clean split:

• Training — learning the weights. You show the model data, it makes predictions, you measure how wrong it is, and you nudge every weight to do better (gradient descent). Slow, done once, needs enormous compute, and produces one artifact: a fixed set of weights.
• Inference — using those weights. A new input comes in, flows through the frozen model, and an answer comes out. Fast, cheap per run — but run billions of times, for every prompt and tap and request.

Why this distinction is the whole course

Almost everything practical about deploying AI — how fast it feels, what it costs, whether it runs on your phone or in a data center — is about inference, not training. Training is a research-and-engineering event; inference is the product.

For the rest of the course we'll zoom into that running model: what a single pass actually computes, how a chatbot streams its answer, the numbers that make a deployment good or bad, and the tricks that make it cheap. Next: the forward pass — what actually happens in one run.