Fable 5 vs. GPT-5.6 Sol: which AI wins on NP-hard problems?

Key Takeaways

• Claude Fable 5 outperformed GPT-5.6 Sol by roughly 1,900 points on average in a fiber-network NP-hard optimization benchmark
• Fable 5 showed far tighter consistency, staying within a 319-point range while Sol spanned nearly 2,000 points
• The /goal mode won 4 of 6 trials but made both models' mean scores worse due to occasional large regressions

A new independent benchmark pits Anthropic's Claude Fable 5 against OpenAI's GPT-5.6 Sol on an unpublished NP-hard fiber-network optimization problem. The result: Fable 5 wins decisively on both raw score and consistency, while the much-hyped /goal mode in both Claude Code and Codex proved unreliable.

Charles Azam, the researcher who ran the test, spent a week years ago writing C++ to solve this same problem. That gives him a useful human baseline against which to judge the AI results. His verdict on Fable 5 is unequivocal.

“Fable 5 was an absolute beast on this benchmark. It produced the best solution overall, and its consistency is unlike anything I have seen from a model on this problem. This is pure raw intelligence. Incredible.”

— Charles Azam

What problem did they solve?

KIRO is a fiber-network design problem Azam worked on as an engineering student in 2018. Given directed distance matrices for Grenoble, Nice, and Paris, a solver must connect distribution points and terminals using loops and short chains while respecting structural constraints. The objective is total cable length. Lower is better.

How hard is it? Azam offers a lower bound: Paris alone has 532 terminals and 11 distribution hubs. Even ignoring ordering and branches, there are 11^532 possible hub assignments. A tighter bound using 19 loops of 28 terminals each yields roughly 10^1223 configurations. Brute force is not an option.

Head-to-head results

Azam ran three matched 30-minute sessions for both Fable 5 and Sol, each in plain mode and with /goal enabled. The raw scores tell the story.

Negative deltas mean /goal found a better solution. Goal mode won four of six trials. But the means reveal the catch.

Fable 5's plain mean was 32,386. Its /goal mean was 33,145, an average 759-point regression. Sol showed the same pattern: plain mean 34,261, /goal mean 35,129, an 868-point regression. Both models typically gained a small edge from /goal and occasionally suffered a catastrophic miss. The large misses dominated the average.

More striking is the consistency gap. Fable 5 plain stayed inside a 319-point range across three runs. Sol plain spanned 1,958 points. For production workloads where you cannot cherry-pick the best run, that variance matters.

Why doesn't /goal just make models try harder?

Azam dug into the implementations. Claude Code and Codex both expose a /goal command, but they work differently under the hood.

Claude Code treats /goal as a session-scoped stop hook. After each main-model turn, a small evaluator model (Haiku by default) reads the goal condition and the conversation, then returns yes or no. A no triggers another turn; a yes clears the goal. The evaluator cannot use tools or inspect files. It can only judge evidence that appeared in the transcript. That can catch an early exit, but it cannot tell whether another ten million solver iterations would help.

Codex treats a goal as persisted thread state with lifecycle tools. The architecture differs, but the fundamental limit is the same: neither system can peer inside the search dynamics of the optimization itself.

“/goal is not a generic 'try harder' switch. It changes the control loop and the search path. Sometimes that finds a better basin. Sometimes it gives a bad idea more time to mature.”

— Charles Azam

What this means for AI-assisted optimization

This benchmark is narrow by design: one problem family, 30-minute budgets, maximum reasoning settings. It does not tell you which model is better for code review, customer support, or content generation. But it does test something hard to game: raw combinatorial intelligence on a novel problem the models have never seen.

On that dimension, Fable 5 is ahead. Its plain mode beat Sol's plain mode by about 1,875 points on average. Its /goal mode beat Sol's /goal mode by 1,984 points. And it did so with far less variance.

The /goal disappointment is equally instructive. Features that promise to make an AI "try harder" sound appealing. In practice, they change the search path in ways the user cannot predict. Sometimes that helps. Sometimes it lets a bad heuristic dig itself deeper. On this benchmark, playing it safe with plain mode was the better bet.

Frequently asked questions

Frequently Asked Questions

What is an NP-hard problem?

NP-hard problems are a class of computational challenges where no known algorithm can find the optimal solution in polynomial time as the input scales. Real-world examples include routing, scheduling, and network design. The fiber-network problem in this benchmark belongs to that class.

Is Fable 5 better than GPT-5.6 Sol for all tasks?

No. This benchmark tested a single NP-hard optimization domain. Performance on coding, summarization, or other tasks may differ. Treat this as one signal, not a verdict on overall capability.

Should I use /goal mode in Claude Code or Codex?

It depends on the task. For well-defined stopping conditions, /goal can prevent early exits. For open-ended optimization, the benchmark suggests it can introduce high variance. Test on your own workloads before relying on it in production.

Where can I see the full benchmark data?

Azam published code, prompts, result tables, and trajectory notes in the CLIArena repository linked from his blog post.

Source: Hacker News: Best