Both Top Models Tied at 88.6% — Then the Harder Benchmark Failed a Third of the Time
On SWE-bench Verified, Opus 4.8 and GPT-5.5 are now in a dead heat at ~88.6%. On SWE-bench Pro — the messier, multi-file version — the leader scores 69.2%. The benchmark that saturated tells you nothing; the one that didn't tells you where your agent breaks.
NeuroX AI · June 16, 2026
On the benchmark everyone quotes, the race is over. Claude Opus 4.8 and GPT-5.5 now sit at 88.6% and 88.7% on SWE-bench Verified — a 0.1-point gap, a statistical tie. Read the headline and you'd conclude coding agents are basically solved.
Then look at SWE-bench Pro, the harder variant built from messier, multi-file, real-world tasks. The leader — Opus 4.8 — scores 69.2%, against GPT-5.5's 58.6%. Same models, same week. Nearly a third of the harder problems still fail.
That gap is the whole story. Verified saturated because its problems are clean: isolated, well-scoped, the kind of fix you can land without understanding the system around it. Pro didn't saturate because its problems look like your backlog — changes that touch five files, depend on context the prompt never carries, and break in ways that don't throw an error.
The lesson for anyone shipping agents: the benchmark that flattened out tells you nothing about production. The one that didn't is measuring the exact 30% where real work lives — and where an unsupervised agent will confidently hand you a wrong answer.
You don't close that gap with a better model. Both leaders already cleared the easy half. You close it with the harness around the model: scoped tasks, independent verification, a definition of done a human can check.