There is a comforting story the market keeps telling itself: bigger context windows will solve agent reliability. If a model can ingest 200K, 500K, or 2M tokens, then surely it can keep the whole task in its head and reason its way through the mess.
That story breaks the minute you ship.
In production, the failure mode is rarely, "the model needed more room." More often it's this: the model had too much room, too much stale state, too many verbose tool results, and too little structure. The issue is not capacity. The issue is context design.
That is why context engineering matters more than context size.
Bigger Windows, Smaller Margins
Anthropic's recent context engineering guidance makes the core point plainly: context is finite, and the value of each additional token drops as the window fills. They call out context rot, the pattern where models get less useful signal from long contexts as more material accumulates.
That matters because most agent systems still default to the laziest possible architecture: append everything.
Conversation history? Keep it. Tool output? Keep it. Search results? Keep them too. Internal summaries, plans, partial drafts, scraped pages, stack traces, retries, logs, all of it gets stuffed back into the next call.
This feels safe because nothing is lost. In practice, it creates three expensive problems:
Latency and cost compound. Every unnecessary token gets re-read, re-billed, and re-delayed.
Signal density collapses. The model spends more attention budget sorting noise from relevance.
Failure gets harder to debug. Once everything is in context, nothing is clearly responsible.
A large window buys headroom. It does not buy judgment.
The Benchmark Trap
Long-context benchmarks helped create the illusion that bigger windows are enough. If a model can find the needle in a haystack, the thinking goes, it should be able to reason over a large working set just fine.
But Chroma's research on context rot points at the flaw in that assumption: simple retrieval benchmarks can overstate real capability. Controlled tests are not the same as production workloads. Real agents do not just retrieve. They compare, prioritize, revise, ignore, and synthesize across messy information with uneven importance.
That is where systems start to drift.
The danger is subtle because the model often does not fail catastrophically. It fails plausibly. It latches onto stale context, overweights an irrelevant tool result, or carries forward an outdated plan. The result looks coherent enough to pass a cursory glance, while the underlying reasoning has already degraded.
That is a much more expensive problem than hitting a hard token limit.
Context Is a View, Not a Dump
One of the clearest framing shifts came from Google's work on production multi-agent systems: context should be treated as a compiled view over richer state, not as a giant running log.
That framing is useful because it forces a systems question: what should this model invocation see right now?
Not everything the system has ever seen. Not every tool result it ever produced. Just the smallest high-signal view required for the current step.
Once you adopt that model, a lot of design decisions become clearer:
Storage and presentation should be separated. Durable state can live outside the prompt.
Transformations should be explicit. Summaries, filters, and handoff artifacts should be deliberate, not accidental.
Scope should be the default. Each call gets the minimum viable context, not the maximum available context.
This is the shift from prompt stuffing to context architecture.
The Three Levers That Actually Matter
Anthropic's breakdown is especially useful because it distinguishes tools that are often blurred together:
Compaction keeps a long-running interaction alive by distilling prior context into a smaller, high-fidelity working summary.
Tool-result clearing removes bulky outputs that can be re-fetched later, while preserving the fact that the tool was used.
Memory moves durable knowledge into structured external storage so the system does not need to drag it around in active context.
These are not interchangeable tricks. They solve different failure modes.
Compaction helps when the conversation itself gets long. Clearing helps when tool usage bloats the window. Memory helps when knowledge must persist across sessions without polluting every invocation.
Used together, they do something more important than token reduction: they preserve attention for the decision at hand.
Why Subagents Work So Well
This is also why the subagent pattern keeps showing up in strong agent systems.
A single generalist agent with one giant context has to do everything at once: plan, search, inspect, compare, summarize, and synthesize. That is convenient to demo, but fragile to operate.
Subagents let you split the work into bounded contexts. A retrieval agent can read a noisy corpus and return only a structured brief. A planner can reason over goals and constraints without wading through raw logs. A synthesis step can operate on compacted artifacts instead of the entire trail of intermediate work.
The win is not just efficiency. It is cognitive hygiene.
Good agent systems are increasingly less like one genius staring at a wall of text, and more like a disciplined team passing around clean notes.
What Builders Should Do Next
If you are building agents today, the practical takeaway is simple: stop treating token capacity as the main scaling variable.
Instead:
Measure context quality, not just context size. Track when accuracy or task completion starts falling as prompts grow.
Compact aggressively. Do not wait until the window is nearly full to summarize.
Clear re-fetchable tool output. Keeping every blob in prompt history is lazy and expensive.
Move durable knowledge into memory. If something matters across sessions, it should not live only inside chat history.
Use scoped subagents. If a task can be isolated, isolate it.
The builders who internalize this early will have a real advantage. Everyone else will keep paying more for agents that look informed but reason worse.
The Real Constraint
The real bottleneck in agent engineering is not whether a model can hold more tokens. It is whether your system can maintain a clean, relevant, decision-ready view of state as work unfolds.
Bigger context windows are useful. But they are not the architecture.
They are just more room to be sloppy.
And sloppy context is still sloppy at 2 million tokens.