4× Faster Tokens Bought 30% Faster Agents. The Rest Went to Orchestration.
NVIDIA shipped a model that generates tokens 4× faster than its size class. Running 10,000 real agent tasks, it finished only 30% sooner. The gap is the part of your agent that isn't inference.
NeuroX AI · August 13, 2026
NVIDIA released Nemotron 3.5 Lightning on August 11 — a 30B mixture-of-experts model with 3B active parameters, distilled from Nemotron 3 Ultra, claiming up to 4× the output speed of similar-sized models. Then it ran 10,000 agent tasks on PinchBench at 86% accuracy and finished 30% faster than Qwen3.6 35B, not 300%.
NVIDIA says the quiet part in its own post: "Agent efficiency ultimately comes down to how quickly a model completes useful work and not simply how fast it generates tokens." Between those two numbers sits everything your agent does that isn't decoding — tool calls, retrieval round-trips, result validation, subagent handoffs, retries. Quadruple the token rate and that layer doesn't move.
The same week, LangChain benchmarked NVIDIA's NeMo Switchyard router across 145 multi-step tasks averaging 6.3 model calls each. Claude Opus 4.8 was needed on 7% of turns — and took 68% of the bill. Escalation routing cut cost 74% and gave up six points of accuracy, 86.0% down to 80.0%. Note the line item nobody budgets for: the judge model deciding when to escalate consumed 21.2% of routed spend. Their break-even needed 5.9% of calls offloaded before routing paid for itself at all.
Both results point the same direction. The model is no longer the thing you tune. The loop around it is.