2026-07-15
GPT-5.6 Sol vs Claude Fable 5: The Routing Test Before the Benchmark
A 40-case admission matrix separates hard model-contract routes from the workload evaluation still needed to choose between GPT-5.6 Sol and Claude Fable 5.
Frontier model routing
GPT-5.6 Sol and Claude Fable 5 are real frontier models. A universal winner between them is not. Their official contracts settle a few routing questions immediately; everything else belongs in a pinned workload evaluation.
The previous version of this article treated launch claims and independent signals as if they formed one comparable benchmark. They do not. Provider evaluations use different prompts, harnesses, tools, effort settings, corpora, stopping rules, and graders. A model can lead a published chart and still be the wrong production route for a particular agent.
This rewrite freezes the public provider contracts as retrieved on 21 July 2026, then runs a deterministic 40-case admission matrix before either model would receive a prompt. The matrix does not call OpenAI or Anthropic. It decides whether documentation alone can route a workload, whether the comparison is invalid, or whether a controlled head-to-head evaluation is still required.
e25c097d937c95aed0cde637a13ea8fff12f1fec10d93f351980b53174d723c5.The old winner sentence is the first thing to delete
That is an admission problem before it is a model problem.
“Which model should run your AI agent?” sounds like one question. It hides at least five: Can the provider accept the request? Does the required tool surface exist? Does the workload fit the budget? Does the safety behavior preserve the intended task? Which model completes the actual job more reliably under the same harness?
The first four may eliminate a candidate before quality is measured. The fifth cannot be answered by comparing two launch posts. A production decision needs the same task corpus, prompt, tools, permissions, retry policy, grader, and success threshold on both sides.
This distinction matters most for autonomous agents. A one-shot answer benchmark can ignore recovery, partial progress, tool misuse, repeated side effects, context growth, and the cost of a failed long run. Those are not secondary metrics once a model can change code or operate a browser.
Provider contracts, frozen on July 21
Capacity, price, and tools are contracts. Quality is still an experiment.
OpenAI documents gpt-5.6-sol as its frontier reasoning model. The unsuffixed gpt-5.6 alias routes to Sol. Its current model page lists a 1,050,000-token context window, 922,000-token maximum input, 128,000-token maximum output, text and image input, and text output. The page also enumerates Responses, Chat Completions, and Batch support plus hosted tools including web search, file search, code interpreter, hosted shell, apply_patch, computer use, MCP, and tool search.
Anthropic documents claude-fable-5 as its most capable widely released model. Its models overview lists a 1,000,000-token context window, 128,000-token maximum output, text and image input, adaptive thinking that is always on, and comparatively slower latency. The identifier is a pinned snapshot rather than an evergreen pointer.
gpt-5.6-sol1.05M context; 922k maximum input; 128k maximum output.
Standard token price: $5 input and $30 output per million.
claude-fable-51M context; 128k maximum output; adaptive thinking always on.
Standard token price:
Those values are useful, but they are not a capability score. OpenAI also states that requests above 272,000 input tokens price the full request at twice the input rate and 1.5 times the output rate. Anthropic's current overview calls Fable slower. Both details can change a routing calculation without proving that one model writes better code.
Testing the admission contract before one model call
The local fixture starts with verified model IDs, current provider documentation, confirmed access to both providers, a pinned repository-agent workload, text-and-image input, text output, a 180,000-token request, a 24,000-token output allowance, and a five-repeat evaluation protocol. One field changes per negative case.
The decision order is intentionally dull. Validate the comparison contract. Verify both model IDs and both documentation snapshots. Confirm access symmetry. Check modalities and token limits. Apply hard price and required-tool constraints. Only then inspect whether prompt, harness, corpus, rubric, and repeat count are stable enough for a quality comparison.
| Mutation | Decision | Why it is not a benchmark |
|---|---|---|
| Unknown model ID or unverified provider page | BLOCK_MODEL_ID or BLOCK_UNVERIFIED_PROVIDER_CONTRACT | The compared product is not pinned to a public contract. |
| Only one provider is accessible | BLOCK_ACCESS_ASYMMETRY | A failed or unavailable route cannot be scored against a live one. |
| Audio input or output is required | BLOCK_MODALITY_MISMATCH | The documented contracts compared here cover text/image input and text output. |
| Input is above 922k and input plus reserved output remains within 1M | ROUTE_CLAUDE_FABLE_DOCUMENTED_CONTEXT | The request exceeds Sol's maximum input while the complete request budget still fits Fable's documented context. |
| Budget admits Sol's standard rate but not Fable's | ROUTE_GPT_SOL_DOCUMENTED_PRICE | A hard price ceiling decides before a quality claim is needed. |
| The harness requires a named OpenAI-hosted tool | ROUTE_GPT_SOL_DOCUMENTED_TOOL | The route follows an exact provider-native dependency, not a general intelligence claim. |
| Prompt, harness, corpus, rubric, or repeats drift | BLOCK_EVAL_PROTOCOL | The test changed between contestants and cannot support a winner. |
| Both contracts fit and the test is pinned | RUN_HEAD_TO_HEAD_EVAL | Documentation does not settle task success. |
Invalid comparisons should fail loudly.
The fixture is an admission test, not a provider benchmark. It checks whether a comparison may proceed. It does not measure model quality, latency, tool correctness, or customer outcomes.
Price can decide without a benchmark
Cost per accepted task is the number that survives contact with production.
At standard direct-API rates, Sol's documented input and output prices are lower than Fable's. If a workload has a non-negotiable ceiling below
That shortcut has a boundary. A 300,000-token Sol request crosses OpenAI's long-context threshold, so the entire request receives the stated 2× input and 1.5× output multipliers. Cached input, cache writes, Batch, Flex, priority processing, provider-cloud prices, and tool charges can change the bill again. A realistic cost row uses the actual request distribution, not the headline price.
Failed runs belong in the denominator. A cheaper token price can lose if the agent repeats expensive tool calls, consumes more output, or needs a human restart. Conversely, paying more for a model that produces no measurable success gain is just expensive routing. The evaluation must report cost per accepted task, not only cost per token.
The context edge needs an output budget
Limits are not leadership.
The marketing shorthand says both models have roughly a million-token context. The request contracts are more precise. Sol advertises a 1.05M context window but caps input at 922k, reserving room for output. Fable lists a 1M context and 128k maximum output; that million is the combined context budget, not a million-token input allowance. With the fixture's 24k output reservation, Fable can admit inputs from 922,001 through 976,000 tokens while exceeding Sol's documented maximum input. A one-million-token input leaves no output room and is blocked.
That is a legitimate direct route. It is also a narrow one. Most repository agents should not treat “fits in context” as a design goal. Sending every file increases cost, latency, stale evidence, prompt-injection surface, and the chance that important constraints disappear inside irrelevant material. Retrieval, scoped tools, summaries, and durable task state remain useful even with a million tokens.
A tool name is not a task score
Measure the artifact.
Sol's model page explicitly lists a broad hosted tool surface. That can settle a route when the application contract requires one exact provider-native mechanism. A harness built around OpenAI hosted shell and apply_patch, for example, is not portable merely because another model can emit function calls.
The reverse inference is invalid. The OpenAI page listing a tool does not prove that Fable lacks an equivalent integration, and an Anthropic launch post praising long-running agents does not prove that Sol fails them. Tool availability, tool-call syntax, execution environment, approval boundaries, and recovery semantics must be compared as application contracts.
For a portable evaluation, expose equivalent capabilities behind one adapter and log every call. For a provider-native evaluation, admit that the platform is part of the contestant. Both tests are useful; combining them produces an uninterpretable score.
Failure modes: safety behavior belongs in the fixture
Fallback is an observable route, not an implementation footnote.
Fable's launch and redeployment notes make its safety tradeoff unusually visible. Anthropic says some requests are handled by Opus 4.8 when classifiers trigger, and that the updated classifier increases false positives for benign coding and debugging while targeting a reported cyber bypass. That behavior may be the correct product decision. It also means an evaluation must record which model actually served a case and whether fallback was acceptable.
OpenAI likewise documents real-time cyber and biology safeguards for GPT-5.6 that may block a request or pause generation while classifiers review output. “No safety intervention can ever occur” is therefore not a satisfiable guarantee for either route. The fixture blocks that requirement instead of quietly pretending it can be met.
A defensive-security corpus should separate an appropriate refusal, an erroneous refusal, a silent fallback, an incomplete patch, and a harmful completion. Counting every refusal as failure rewards unsafe behavior; counting every refusal as success hides unusable automation.
A repeated run cannot move the goalposts
No winner without the same work.
A useful head-to-head test starts after the admission gate. Freeze a dated model ID or documented pinned identifier. Freeze the system prompt, user task, repository commit, fixtures, tool adapter, network policy, approval policy, time budget, retry budget, and grader. Then run each case several times. The local contract requires at least three repeats and uses five by default.
Score the final artifact, not the model's explanation of it. For a coding agent, that means tests, type checks, lint, build, security invariants, file-scope compliance, diff review, and required documentation. Record wall-clock time, input and output tokens, cache use, tool calls, retries, failures, and human interventions. Preserve partial work so a model that almost finishes does not look identical to one that corrupts the repository.
{
"taskCommit": "pinned",
"prompt": "pinned",
"toolContract": "equivalent-or-declared-native",
"repeats": 5,
"pass": "tests + policy + artifact review",
"report": ["successRate", "costPerPass", "p50Latency", "p95Latency", "interventions"]
}Randomize run order and keep graders blind to the provider when possible. If one model needs a different prompt, publish that as a separate optimized configuration rather than editing the prompt during the contest. A fair identical-prompt test and a best-available-configuration test answer different questions.
Troubleshooting a router that learned folklore
Constraints choose candidates. Evidence earns promotion.
A production router can remain small. Reject requests outside both contracts. Route Fable when the input genuinely needs more than Sol's 922k maximum and the input plus reserved output remains within Fable's 1M context. Route Sol when a hard standard-rate ceiling excludes Fable or when an exact OpenAI-native tool is part of the product contract. Send everything else through the workload evaluation.
Promotion should require a measured margin and a rollback path. Keep a shadow sample on the alternative route. Pin the evidence date. Re-run after model, prompt, tool, harness, price, or safeguard changes. A routing decision that cannot name the fixture and source snapshot that produced it is folklore.
The honest verdict is less dramatic than “model A wins.” Sol has a documented price advantage and a clearly enumerated hosted-tool surface. Fable has a documented input-admission band above Sol's maximum input and an explicit long-running-agent product position. Below those hard boundaries, the right model is the one that passes more of your jobs at an acceptable total cost and latency under a repeatable test.
Sources and the untested boundary
- OpenAI GPT-5.6 Sol model page — model ID, context and input/output limits, modalities, endpoints, features, supported tools, and long-context pricing note.
- OpenAI GPT-5.6 guide — family aliases, reasoning modes, programmatic tool calling, multi-agent beta, safeguards, and evaluation guidance.
- OpenAI API pricing — standard, cached-input, cache-write, output, Batch, Flex, and priority rates retrieved on 21 July 2026.
- Anthropic Claude models overview — Fable model ID, pricing, context and output limits, latency class, thinking mode, availability, and versioning contract.
- Anthropic Fable 5 launch and redeployment note — provider capability claims, safeguards, fallback behavior, interruption, restoration, and updated false-positive tradeoff.
- Local deterministic admission fixture — 40 cases, 40 matches, 18 decision states, seven direct constraint routes, five head-to-head requirements, and 28 blocks or holds. SHA-256:
e25c097d937c95aed0cde637a13ea8fff12f1fec10d93f351980b53174d723c5.
The retained experiment stops before inference. It does not include an OpenAI or Anthropic API call, provider credential, customer repository, live browser, tool execution, latency sample, billing event, safety-classifier event, human grader, or statistically powered benchmark. Its claim is narrower and testable: a routing system can reject malformed comparisons and resolve documented hard constraints before spending money on a model contest. The remaining quality decision still requires real workload evidence.