A long-running AI agent often fails not because of the model, but because its working context grows without control at every step, important intermediate results are lost, or outdated assumptions are carried forward. A large context window merely postpones the problem. Reliable agents require a deliberate state and context architecture.
OpenAI published a recent example on 29 July 2026: In an ARC-AGI-3 harness, the model lost track of earlier actions and its reasoning. With retained reasoning and context compaction, the reported score on the public set increased by 188 percent while using six times fewer output tokens. This is a provider experiment, not general proof of productivity. It does, however, clearly show that the harness and context management are part of the system performance being measured.
Why Is a Large Context Window Not Enough?
An agent rarely works with only a conversation. It accumulates search results, tool outputs, plans, failed attempts, intermediate results and approvals. If all of this content is passed on without filtering, costs and latency increase. At the same time, it becomes harder for the model to distinguish relevant signals from historical noise.
The opposite problem is overly aggressive reduction. If the agent loses earlier decisions, tested hypotheses or the status of a tool, it starts again. The result is duplicated work, contradictory steps and additional costs.
Separate Four Types of State
A robust architecture separates at least four layers:
- Conversation context for the current interaction.
- Working state containing the plan, open items and validated intermediate results.
- Company knowledge from authoritative sources.
- Audit data covering tool calls, approvals, errors and results.
These layers have different lifecycles and permissions. Audit data does not automatically belong in the next model call. Company knowledge should not be treated as a freely editable agent note. The working state must remain compact without losing important decisions.
What Context Compaction Must Preserve
Context compaction should not simply summarise the latest messages. Effective compaction preserves:
- the task's objective and acceptance criteria,
- confirmed facts with sources,
- decisions already made,
- failed approaches and why they failed,
- open risks and the next permitted actions.
Raw tool data can often be replaced with structured results. For example, an agent does not need the entire response from a ticketing system if the ticket ID, status, owner and next due date are sufficient.
When Should an Agent Stop Instead of Compacting Again?
Compaction can conceal information loss. The workflow therefore needs stop signals. The agent should escalate to a human when sources conflict, an approval is missing, the state can no longer be reconstructed safely, or repeated compaction produces the same uncertainty.
Budget and time also matter. An agent that has made no measurable progress after twenty steps does not need another summary; it needs a defined termination condition.
How Do You Test Context Management?
A standard response test is not enough. Test tasks over many steps and check specifically:
- Does the agent retain earlier decisions correctly?
- Does it repeat work that has already been completed?
- Does it recognise outdated assumptions?
- Do sources and approvals remain linked to the result?
- How do cost and quality change after compaction?
Compare variants on the same set of tasks. Only then can you see whether fewer tokens genuinely mean lower costs or whether the amount of rework increases.
The Practical Consequence
For long-running agents, context management is an operational function. It requires data models, permissions, versioning, evaluation and stop signals. If you only swap the model, you are often measuring the harness just as much as the model itself.
Sources
- OpenAI, How two settings tripled our ARC-AGI-3 scores: https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/
- OpenAI on X, 29 July 2026: https://x.com/OpenAI/status/2082616636989952217
- OpenAI on X, Retained Reasoning and Context Compaction: https://x.com/OpenAI/status/2082616640144048433
- NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
Next Step
Choose an agent workflow with at least ten steps. Map which information is retained, compacted, discarded or logged separately after each step. Then test two context strategies against the same tasks and measure quality, repetition, costs and human rework.