New models appear faster than most enterprises can evaluate them properly. A public benchmark can help with initial screening. It says little about whether the model understands your documents, uses your tools correctly, or stays within the permitted error rate.
On 29 July 2026, OpenAI itself demonstrated how strongly a harness can affect measured performance. According to the company, Retained Reasoning and Context Compaction significantly improved the result in an ARC-AGI-3 test. The practical lesson is not to enable a particular API setting everywhere. It is that the model, harness, context management, and assessment method together form the system being tested.
Test the workflow, not just the answer
A production AI workflow includes retrieval, the model, tools, rules, and human review. The evaluation set should therefore exercise the entire path.
For a support agent, a good text score is not enough. The relevant criteria include the correct source, permitted customer access, correct tool selection, draft quality, and approval status.
Combine real and synthetic cases
Start with anonymised real tasks. Add synthetic cases for rare edge conditions and deliberate attacks. Hold back some cases so that optimisation does not target only the familiar test set.
For each case, document the expected result, acceptable variations, and disqualifying errors.
Define disqualifying errors first
Averages can hide critical failures. Define errors that are unacceptable regardless of the overall score:
- access to unauthorised sources,
- fabricated evidence,
- an unauthorised tool action,
- the wrong amount or recipient,
- failure to escalate when uncertain.
A model with a better average but a new disqualifying error is not an improvement for that route.
Combine automated and human assessment
Automated checks are suitable for schemas, sources, exact fields, and known rules. Model-based evaluators can scale, but they must be calibrated against human judgement.
Subject-matter experts are required to assess tone, practical usefulness, and risky edge cases. Make assessment criteria specific enough that two reviewers are likely to reach similar decisions.
Include cost and latency
Measure tokens, tool calls, retries, cache hits, runtime, and human correction time. Compare cost per accepted result rather than price per million tokens.
A more capable model can be more economical if it reduces loops and rework. A smaller model can win when it performs a narrow task reliably.
Shadow testing, approval, and rollback
Run a new model first in shadow mode or against recorded cases. Follow this with a limited pilot and monitoring. Keep the existing route available for a rapid rollback.
Version the model, prompt, tools, retrieval, and evaluation set together. Otherwise, you will not be able to explain a difference later.
Sources
- OpenAI Evals: https://platform.openai.com/docs/guides/evals
- Anthropic, Define success criteria and build evaluations: https://docs.anthropic.com/en/docs/test-and-evaluate/define-success
- NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
- OpenAI on X, 29 July 2026: https://x.com/OpenAI/status/2082616636989952217
Next step
For one workflow, build a set of twenty real cases, five edge cases, and three disqualifying errors. Measure the old and new models across the same complete system path before changing the production route.