a visual course

What Inference
Really Is

(AI Inference)

What actually happens when an AI model runs — not how it's trained, but how it's used. In about half an hour.

A free, visual course on inference — the moment a trained AI model actually runs to give an answer. Training vs inference, the forward pass, an LLM's token-by-token loop, latency vs throughput, and how inference is made cheap.

in the real world

Every chatbot reply, phone face-unlock, fraud check and feed refresh is inference — and it's where almost all of AI's real-world compute (and cost) actually goes.

Start with the two phases →

Part 1 — The two phases

1

Train once, run forever

Training teaches the model; inference is every time you actually use it — and that's most of the work.

Part 2 — Inside a running model

2

The forward pass

Inference is one trip through the network: push the input through frozen weights and read the output.

3

One token at a time

For a chatbot, inference is a loop: read the prompt, then generate the answer one token at a time.

4

Latency & throughput

Time to first token, tokens per second, and batching many users — the numbers that define a good deployment.

Part 3 — Making it cheap

5

Faster, smaller, cheaper

Quantization, distillation, caching and batching shrink the cost — because inference runs a billion times.

Part 4 — Out in the world

6

Where inference runs

Every chatbot reply, face unlock, fraud check and feed refresh is inference — in the cloud or on your device.