Right-size the model

Nobody hires a senior architect to change a lightbulb. Yet that's exactly what most AI products do: route every request — "extract this email address", "is this spam?" — to the biggest, smartest, most expensive model on the menu.

Every provider sells a ladder of models: a frontier tier for the hardest reasoning, a mid tier for everyday work, and a small tier that's fast and cheap. The price gaps are not subtle — 5–10× per token between the small tier and the frontier. Same API, same code; one line — the model name — changes the bill by an order of magnitude.

Scroll to route some traffic properly.

Match the tier to the task

The uncomfortable truth behind this lever: most production requests are easy. Classification, extraction, formatting, routing, short summaries — small models handle these reliably. The frontier model's edge only shows up on tasks with genuine depth: multi-step reasoning, subtle judgment, hard code.

A practical way to decide:

• Start small, escalate on failure. Try the cheap model first; if its answer fails a check (or the user retries), re-run on a bigger one. This "cascade" pattern gets frontier quality on the ~10% of requests that need it.
• Route by task type. If you can label requests up front — "extraction" vs "analysis" — send each category to its tier. Many products run a tiny, cheap classifier as the router.
• Distill what repeats. For one narrow, high-volume task, a small model fine-tuned on the big model's outputs often matches it — the distillation idea from the inference course.

The quality guardrail

Right-sizing is not "use the cheapest model everywhere" — a wrong answer is infinitely more expensive than a correct one at 10× the token price. The discipline is to measure: keep a small test set per task, and when a cheaper model passes it, take the savings. When it doesn't, pay up happily.

Model choice sets the rate you pay. The next two chapters cut the volume — starting with the strange fact that you keep re-buying the same tokens.