Trim, cap, batch

The last three levers are the unglamorous ones — no clever infrastructure, just discipline. They're also the ones you can apply this afternoon.

Put the prompt on a diet

Prompts accumulate flab: three paragraphs of personality, rules for situations that never occur, examples nobody trimmed after launch. Every one of those tokens rides along on every call, forever — and if it sits mid-prompt, it can't even be cached away cheaply when the parts around it change.

Scroll to see the diet, the cap, and the night bus.

Cap the expensive direction

Remember from Chapter 1: output tokens cost ~5× input. Yet models love to elaborate — restating your question, hedging, adding a cheerful summary. For machine-consumed output, kill all of it:

• Ask for the shape you need: "reply with only the label", "return JSON matching this schema", "max 150 words". (This is prompt engineering doing double duty as cost engineering.)
• Set the API's output limit (max_tokens) as a hard backstop so a runaway answer can't cost 20× what you expected.

CHATTY:  "Great question! Based on my analysis of this
       ticket, I believe the most appropriate category
       would likely be 'billing', because…"   (~120 tokens)

CAPPED:  billing                                  (1 token)

Take the night bus

Not every request needs an answer in two seconds. Nightly report generation, summarizing a backlog of documents, re-tagging your archive, running evals — none of it cares about latency. Providers sell exactly this tradeoff: batch APIs take a big file of requests, run them asynchronously (typically within an hour, up to a day), and charge 50% of the normal price for every token.

Half off, for changing when the work runs. It's the easiest discount in this course — the only skill required is noticing which of your workloads can wait.

That's all five levers: cheaper models, cached repeats, trimmed prompts, capped outputs, batched jobs. The final chapter is about making them stick — and where they pay off in the real world.