Nvidia SoL‑Pi harness optimization cuts coding agent token usage nearly 50%

Nvidia’s SoL-Pi system cuts coding agent token usage nearly in half by optimizing the harness

For teams running multi-step, agentic pipelines, token bills are more than an accounting headache, they shape architecture choices, CI cadence, and whether new automation actually pays back. Nvidia’s SoL‑Pi work focuses on a practical lever most organizations haven’t tuned aggressively: the harness, the control layer that mediates model calls, tools, and verification. The reported results are large enough to justify pilots, but they come with clear trade-offs and open engineering questions.

Quick glossary (first time readers)

Harness, the control code between an LLM agent and its environment: state handling, tool invocation, verification and retry logic, and how context is managed.
Pi harness (Pi), the baseline harness used in Nvidia’s comparisons (the name used in reporting of the study).
Codex / Claude Code, other existing harness or framework baselines cited in the comparison.
GPT‑5.6 Sol and Opus 5, the two LLMs referenced: SoL‑Pi was optimized on GPT‑5.6 Sol and then tested unchanged on Opus 5 (as reported).
EdgeBench, the held‑out public benchmark used for final evaluation (51 tasks total; a subset held out from search).
Prompt cache, reusing previously sent prompts/context to avoid resending large contexts; an important production cost factor.

What Nvidia did and why that matters

Instead of changing models or serving stacks, Nvidia researchers used an automated “research agent” to watch execution traces of coding agents, propose harness edits, and validate candidates inside sandboxed environments. The idea is simple: change how you call the model and what you resend between steps, and you can cut token volume without touching the model itself.

Jonathan Kemper reported these experiments for The Decoder (Sept 26, 2026), summarizing the Nvidia paper referenced there (arXiv:2609.20519v1). According to that reporting, the search explored 152 directions across 535 executable environments, ran more than 3, 000 experiments, and produced over 60, 000 agent, environment interactions across 495 GitHub-derived tasks and 40 synthetic test cases. The authors held a 51-task public benchmark (EdgeBench) partly out of the search loop to limit overfitting (11 tasks for one-time validation; 40 fully held out), per The Decoder’s write-up.

The headline results (reported)

On the held-out EdgeBench tasks, SoL‑Pi variants reduced token usage by roughly 44.7-49% (reported point estimates from The Decoder summarizing Nvidia’s paper).
The most aggressive variant (all four discovered mechanisms enabled) used 49% fewer tokens while reaching 93.7% of Pi’s score on EdgeBench (as reported).
A performance‑prioritized single‑mechanism variant reportedly beat Pi’s score by 5.3% while still saving tokens.
The authors’ cost estimates (based on current API prices cited in the report) put hourly savings at $8.75, $13.50/hour versus native Codex and Claude Code harnesses, and $4.36, $5.71/hour versus the Pi harness (reported figures; see notes below on assumptions).
Portability: a harness optimized on GPT‑5.6 Sol, when applied unchanged to Opus 5, reportedly retained 94.3% of Pi’s performance with similar token savings, though the mechanisms “fired less often” on Opus 5 (per The Decoder’s summary).

Not every benchmark favored the changes. On Terminal‑Bench 4 (63 CPU tasks) SoL‑Pi solved 15 tasks versus 18 for Pi/Codex but still achieved roughly 25% lower total cost; on a Lean 4 set tied to IMO 2026 problems SoL‑Pi solved 3 of 6 problems with the lowest cost per solved problem (all results as reported). The authors also describe swarm experiments where 20 SoL‑Pi workers reduced costs by 26.8% versus a comparable Pi swarm; one example run reportedly fell from $1, 339 to $894.

Note on the numbers above: these are the point estimates reported by Jonathan Kemper in The Decoder summarizing the Nvidia paper. The report does not publish distributional statistics (means vs medians, standard deviations, per‑task variance) in the summary; consult the paper and supplemental materials for full statistical detail and the authors’ raw cost spreadsheet if you need production‑grade validation.

The four mechanisms SoL‑Pi surfaced, and practical notes

The automated search found four concrete harness-level mechanisms that reduce wasted LM calls and shrink context. Below each mechanism is condensed into what it is, an illustrative example, a common failure mode, and a practical mitigation.

Action Fusion, Merge consecutive steps into a single composite action to avoid an intermediate LM call.

Illustrative example: before: an “edit” LM call (2, 000 tokens) followed by a “test” LM call (1, 200 tokens); after fusion: a single 2, 300‑token call, fewer round trips and lower total tokens. (This example is illustrative; actual token counts vary by task.)

Failure mode: fused prompts can become too long or lose modular clarity, making debugging harder.

Mitigation: set prompt‑length guards and fall back to unfused steps when fusion would exceed a length threshold or reduce repeatability.
Online Context Compact, Trim or drop accumulated planning notes or scaffolded context that provably won’t be needed in future steps.

Failure mode: compaction can accidentally remove a detail that later is needed for verification or debugging.

Mitigation: conservative heuristics (time‑based or dependency‑based retention), keep a compressed archival copy, and require verification checks before any destructive compaction.
ObservationPack, Archive large tool outputs (like full test logs) and forward only short summaries; reinsert the full output on demand.

Failure mode: summaries can omit subtle signals that matter for root-cause reasoning.

Mitigation: use summary confidence indicators, inexpensive checks that detect missing error patterns, and quick retrieval paths to the archived full output when anomalies arise.
Evidence‑Preserving Reducer, Route huge logs or failing-test dumps to a cheaper model to extract key findings, then verify automatically that the reducer didn’t drop critical evidence.

Failure mode: a cheap reducer misses a critical clue and verification fails to detect that omission.

Mitigation: dual‑path verification: when the reducer reports a shortfall or low confidence, re-run the full context through the base model for confirmation; log all fallbacks for audit.

These mechanisms are practical engineering patterns: avoid repeated round trips, stop resending long outputs, and offload obvious summarization to cheaper models with verification. The discipline is to trade marginal fidelity for large, predictable token reductions while retaining auditable fallbacks.

“SoL‑Pi cuts coding agents’ token usage by up to 49 percent with little change in performance by optimizing the control layer between the model and its environment.”, summary phrase reported by Jonathan Kemper in The Decoder (Sept 26, 2026), summarizing the Nvidia paper.

Key methodological choices that lend credibility

Scale: The search reportedly examined 152 directions across 535 executable environments with >3, 000 runs and >60k interactions, a nontrivial engineering investment (reported by The Decoder).
Held‑out evaluation: The team deliberately separated search from final evaluation and kept 40 EdgeBench tasks fully out of the search loop; per The Decoder, “The held‑out evaluation happens only after the harness is frozen and doesn’t feed back into the search process.” That design reduces a common form of overfitting in automated optimization.
Portability test: the harness optimized on GPT‑5.6 Sol was applied unchanged to Opus 5; reported retention of performance (94.3% of Pi) suggests partial transferability but also trajectory sensitivity (mechanisms fired less often on Opus 5).

Where the trade‑offs and open risks are

Prompt‑cache economics: trimming context can reduce prompt‑cache hits over time; a harness that saves tokens now could change cache dynamics and increase costs later. Measure cache hit rates pre/post changes.
Model‑specific tuning: optimization trajectories matter. A harness tuned to one model’s planning patterns may be less active and less beneficial on another.
Information loss and safety: compressing logs and routing data to cheaper models risks dropping critical evidence. Nvidia’s Evidence‑Preserving Reducer includes verification in the reported design, but production teams need concrete failure rates and examples before trusting automatic compaction.
Domain sensitivity: large token savings on one benchmark (EdgeBench) did not uniformly translate to better task coverage on others (Terminal‑Bench 4), so expect variability by task type.

Practical experiment checklist for teams

Run a small pilot that mirrors your production mix: suggested minimum 50 distinct tasks or 500+ runs to surface cache and distribution effects.
Collect these metrics for each run: total tokens, input tokens, output tokens, success/solve flag, time‑to‑solution, and prompt‑cache hit rate.
Use a held‑out set of tasks that are never used during search or tuning; reserve at least 20-30% of your evaluation pool as fully blind.
Require per‑task variance reporting (mean, median, standard deviation) and cost assumptions: request the exact price table and tokenization model used to compute $/hour savings.
Instrument verification and fallback paths: record how often summarizers/reducers fall back to full traces and surface those runs for manual inspection.
If you operate multiple models, run a mixed‑model optimization or at least validate an optimized harness across every model you deploy.

Key questions, honest answers

Can harness‑level optimization materially reduce token bills?

Yes, Jonathan Kemper’s reporting in The Decoder summarizes Nvidia’s paper as finding token reductions of roughly 44.7-49% on the held‑out EdgeBench suite, with the best variant using 49% fewer tokens while reaching 93.7% of Pi’s score. Validate those numbers against the paper and supplements before assuming the same in production.
Do those savings come with accuracy loss?

Sometimes, the most aggressive token‑saving variant had a modest performance drop to 93.7% of Pi on EdgeBench; a different configuration reportedly improved score by 5.3% while still saving tokens. The outcome depends on which mechanisms are enabled and the task domain.
Will an optimized harness transfer across models?

Partially, the harness optimized on GPT‑5.6 Sol, when applied unchanged to Opus 5, reportedly retained 94.3% of Pi’s performance with similar savings, but mechanisms triggered less often on Opus 5. Optimization can be trajectory‑specific, so multi‑model tuning is prudent.
Are the dollar‑savings robust?

Not guaranteed, the reported $/hour savings ($8.75, $13.50 vs Codex/Claude Code; $4.36, $5.71 vs Pi) are authors’ estimates based on API prices cited in The Decoder. Real savings depend on your model mix, tokenization, cache behavior, orchestration costs, and contractual pricing.
Is automated harness search safe to run in production?

Proceed cautiously, the research ran candidates in isolated environments and used held‑out evaluation; production deployments should replicate those safety boundaries, enforce verification, and require human review before changing harnesses that invoke tooling or deployment steps.

What to watch next

Will the authors release code, harness candidates, and the cost spreadsheets used to compute $/hour? Public artifacts make reproducing and auditing the results tractable.
How often do verification fallbacks trigger in long runs and what are the false‑negative rates for the Evidence‑Preserving Reducer? That determines operational risk for compressed logs.
How prompt‑cache hit rates and long‑running fleet economics change over weeks or months after aggressive context trimming, measure longitudinal effects before wide rollout.

Final perspective and next steps

SoL‑Pi reframes a simple operational insight as a systems problem: expensive agentic workflows aren’t only about model speed or per‑token price, they’re about how many tokens you force your models to generate and resend. The reported token reductions (44.7-49% on a held‑out benchmark, per The Decoder summary of Nvidia’s paper) are large enough to warrant pilots for organizations with substantial agentic volume.

But don’t flip production toggles based solely on headline percentages. Before adopting any automatically discovered harness changes, obtain the paper and supplements, request the authors’ cost assumptions and code, run blind held‑out validation on your workload mix, instrument verification fallbacks, and keep human‑in‑the‑loop gates for changes that alter tooling calls. If your token bills have ballooned because of multi‑step agents, harness‑level optimization deserves a disciplined experiment, it can save money, but it needs the same observability and safety discipline you’d apply to any change that touches CI/CD or deployment tooling.