The context tax

Here's the fact that surprises almost everyone: the model remembers nothing between calls. When a chatbot "remembers" your name from five messages ago, it's because the app quietly re-sent all five messages — your words and the model's replies — inside the new request. The model re-reads the whole conversation, every single turn.

Now connect that to Chapter 1: everything you send is billed as input tokens. Re-sent history isn't free context. It's a purchase. Again. And again.

Scroll to watch three innocent chat turns compound.

Where the tax bites hardest

Do the math on a long conversation and it's worse than linear — each turn pays for all previous turns, so total conversation cost grows roughly with the square of its length. The same tax shows up wherever context piles up:

• Long system prompts — a 3,000-token system prompt rides along on every call your product ever makes.
• RAG pipelines — retrieved documents get stuffed into the prompt each turn, often thousands of tokens per question.
• Agents — an agent takes dozens of steps per task, and each step resends the growing transcript of everything before it. This is why agent workloads are the most expensive thing in AI today.

Taming it

You can't abolish the tax — the model genuinely needs context to answer well (that's the context window doing its job). But you can stop paying it stupidly:

• Trim what you carry. Summarize or drop old turns instead of resending a novel; most apps cap history at the last N messages.
• Retrieve less, better. In RAG, three relevant paragraphs beat thirty mediocre ones — for quality and cost.
• Watch tokens-per-request over time. If it climbs as sessions age, the context tax is eating you.

And the single biggest mitigation — making the repeated part nearly free — gets its own chapter shortly. First, though, the biggest lever of all: which model you're paying for in the first place.