Cognition SWE‑2 claims Fable parity at 64% lower inference cost — weights closed, runs only in Devin

Cognition says SWE‑2 matches top coding models at 64% lower inference cost, but you can’t run it yourself (Cognition, summarized by MarkTechPost, Sept 12, 2026)

Cognition reports that SWE‑2, a coding model post‑trained with reinforcement‑learning from Moonshot AI’s open Kimi K3 (2.8 trillion parameters), scores 50.0% on FrontierCode 1.1 Main, within one point of Fable 5.1, while cutting inference cost by 64% (per Cognition, summarized by MarkTechPost). The catch: SWE‑2’s weights aren’t being released and the model currently runs only inside Cognition’s Devin product (Desktop and CLI today, Web and Fusion rolling out), so all cross‑model comparisons come from Cognition’s evaluation harness.

TL;DR

SWE‑2 is an RL post‑trained coding model built from Kimi K3 that introduces selectable effort modes (medium/high/max) trained in a single RL run (Cognition reported).
Cognition reports FrontierCode 1.1 Main = 50.0% for SWE‑2, “within 1 point” of Fable 5.1 at 64% lower cost, but those are vendor‑run comparisons on Cognition’s harness (per Cognition / MarkTechPost).
Technical levers include a cost‑aware reward (R = S − λC), length‑weighted baselines, rollout/serving optimizations, and quantization‑aware training (Cognition reported).
Action for buyers: demand the evaluation harness, USD‑per‑inference numbers per effort mode, training compute costs, and a short controlled pilot in your environment before procurement.

What SWE‑2 is, in plain terms

SWE‑2 is Cognition’s most capable coding model to date. The company started from Moonshot AI’s open Kimi K3 (2.8T parameters) and ran a large‑scale reinforcement‑learning post‑training pass that trains three selectable reasoning‑effort levels in one RL run, explicitly penalizes inference cost in the reward, and applies a set of training and serving optimizations Cognition says reduce both latency and dollar cost (per Cognition).

“R = S minus lambda times C”

Cognition uses this shorthand for a reward that subtracts a cost penalty from a binary success signal. Cognition explains S is a binary success indicator, C mixes inference cost (USD) and rollout time, and each effort level gets its own λ set from the local slope of the base model’s Pareto curve (Cognition reported). Cognition also uses a length‑weighted reward baseline (sum(R×L)/sum(L)) to stabilize gradients across rollouts.

Key technical levers (one‑line explanations)

Pareto‑informed cost penalties, λ is chosen from the base model’s local Pareto slope so the reward trades solve rate and cost in a locally tuned way (Cognition reported).
Length‑weighted baseline, rewards averaged by rollout token length to reduce gradient variance tied to long rollouts (Cognition reported).
Multi‑effort RL in one run, medium/high/max effort modes trained together, each with its own cost penalty so the model can be asked to be cheap or push harder (Cognition reported).
Prefill delayer, batching trick that raises tokens‑per‑minute per GPU and TPS per request by an estimated 10-20% (Cognition reported).
DSpark speculative decoding, speculative decoding to accelerate rollouts (Cognition cited DSpark as a building block).
SpecForge retraining for the draft model, retrained draft increases accept lengths by ~15% before online training alongside the policy (Cognition reported).
NVFP4 / FP8 + quantization‑aware training, numeric kernels and QAT used to reduce memory and limit train/inference mismatch versus prior builds (Cognition reported).
Environment flywheel, tripled RL environments and a repair loop that reuses earlier checkpoints to patch verifier false positives/negatives (Cognition reported).

Performance claims, what Cognition reports (and the provenance)

Headline numbers are Cognition’s measurements (summarized by MarkTechPost): FrontierCode 1.1 Main = 50.0% for SWE‑2, “within 1 point” of Fable 5.1, at “64% lower cost” (Cognition reported, cross‑model comparisons are from Cognition’s harness). Cognition also reports SWE‑2 beats its K3 base on every reported row and leads Terminal‑Bench 2.1 while trailing Terminal‑Bench 4 by roughly 30 points versus Fable 5.1 / GPT‑6 Astra (Cognition reported).

Operational efficiency changes reported by Cognition (company numbers): mean steps per run fell from 127 (SWE‑1.7) to 53 (SWE‑2 medium), and median first real edit improved from 48 steps to 18. Cognition says SWE‑2 medium takes 58% fewer turns and costs 81% less than SWE‑1.7 medium (Cognition reported).

Two important provenance points up front: FrontierCode is Cognition’s own benchmark, and the rival‑model numbers in Cognition’s published tables are evaluated by Cognition’s harness. That makes the direction and mechanisms interesting, but independent replication on a neutral harness is required to confirm the cross‑model cost and parity claims.

Trustworthiness and safety checks (what Cognition reran)

Cognition reran two evaluations from their open trustworthiness study. On 145 politically sensitive prompts about China, SWE‑2 “passed” 98.0% overall: 99.8% in English, 95.2% in Simplified Chinese, and 99.1% in Traditional Chinese (Cognition reported). They also report a context‑dependent vulnerability test where no framing produced a statistically significant change for any model (Cognition reported).

These results are useful but narrow. High pass rates on a specific, known dataset are reassuring for that dataset. They do not prove robustness across adversarial or unseen framings. Cognition did not publish the full prompt set, pass/fail rubric detail, or the statistical test parameters in the material summarized for public readers.

Limits, open questions, and what Cognition hasn’t (yet) disclosed

Independent validation: Nearly all cross‑model comparisons come from Cognition’s harness. Independent, third‑party runs on the same harness are needed to confirm parity and the 64% cost claim (MarkTechPost).
Absolute cost figures: Cognition reports relative cost reductions (64%, 81%) but has not published absolute USD‑per‑inference numbers or the full cost formula that turns TPM/time into USD in the public summary (Cognition reported; absolute values absent).
Exact λ and reward scale: Cognition explains λ is set to a local Pareto slope but hasn’t published the numeric λ values, units, smoothing method, or whether λ is global or per‑task (Cognition described method but didn’t publish values).
Training compute and wall‑clock: Total RL GPU‑hours, instance types, and monetary training cost were not disclosed in the public summary (Cognition described methods but not compute totals).
Dataset and environment specifics: Cognition “tripled RL environments” and added instruction‑following overlays but did not publish the environment content breakdown, licensing, or verifier specs in the summarized material.
Metric definitions: Terms like “step, ” “turn, ” and “first real edit” are reported by Cognition but the exact measurement protocol and thresholds were not published in the public summary.
Generality beyond coding: Results are coding‑focused; whether the RL recipe transfers to non‑coding tasks is untested in the provided materials.

Business implications, what product and engineering leaders should care about

Three practical takeaways if you run developer productivity or automation teams:

Cost‑aware RL is now a production lever. Explicitly penalizing inference cost inside the reward (R = S − λC) gives you a real knob to trade accuracy for dollars. If validated, this makes per‑call budgeting a first‑class objective for coding agents (Cognition reported).
Selectable effort modes can improve UX and economics. Training multiple effort settings in one model lets you serve cheap, quick suggestions for routine edits and higher‑effort reasoning for critical flows without swapping models (Cognition reported).
Closed, product‑embedded models change procurement and reproducibility. Running SWE‑2 only inside Devin simplifies vendor control and safety gating for Cognition, but it prevents customers and researchers from running neutral head‑to‑head benchmarks or deploying the model on their own infrastructure.

Operational warning: faster or cheaper generation shifts costs to human review, CI, and long‑term maintenance. Cognition’s own community notes around large‑scale code generation (e.g., Kimi K3 demos of tens of thousands of lines) echo a recurring theme, scaling generation often moves the bottleneck into verification and maintenance capacity.

Due‑diligence checklist for buyers (ask vendors for these before signing)

Provide the exact evaluation harness and scripts used for FrontierCode and Terminal‑Bench comparisons (timeouts, tool access, seeds, and prompt templates).
Publish absolute USD‑per‑inference for each effort mode and an explanation of how C is computed (time‑to‑USD conversion, currency of pricing, and any amortized overheads).
Share total RL compute (GPU‑hours), instance types, and wall‑clock training time for the SWE‑2 RL run(s).
Share the Pareto curves for the base model and the numeric λ values or a reproducible algorithm to compute them (units and smoothing method included).
Supply a representative set of failure cases and the verifier rules used in the RL environments, plus licensing details for any data used.

Concrete internal tests to run during a pilot

Run SWE‑2 (via Devin) across 100 representative PRs covering your critical languages and CI workflows; measure reviewer time, suggestion accept rate, and post‑merge CI failure rate.
Measure tokens‑per‑minute, average suggestion size, and accept/modify/reject ratios per effort mode to quantify reviewer load changes.
Introduce adversarial or ambiguous prompts and record hallucination rates, security‑sensitive code generation, and license risk cases.
Request logs showing mean steps/turns for the same tasks so you can compare Cognition’s reported step reductions against your codebase reality.

Key questions, honest, short answers

Does SWE‑2 really match top competitors while costing 64% less?

Cognition reports SWE‑2 at 50.0% on FrontierCode 1.1 Main, within one point of Fable 5.1, at 64% lower cost, but these are vendor‑run comparisons on Cognition’s harness (per Cognition / MarkTechPost). Independent replication and absolute USD figures are not published publicly.
Can I download SWE‑2 weights or call it from my own API?

No. Cognition has not released weights or a standalone API; SWE‑2 runs only inside Devin (Desktop and CLI today, Web and Fusion rolling out), per Cognition.
What produced the reported cost wins?

Cognition attributes gains to Pareto‑informed cost penalties (R = S − λC), a length‑weighted baseline, joint multi‑effort RL, rollout/serving optimizations (prefill delayer, DSpark speculative decoding), SpecForge retraining for the draft model, and quantization‑aware kernels (NVFP4/FP8), all reported by Cognition.
Are the behavioral improvements meaningful for developer workflows?

Per Cognition, SWE‑2 takes fewer turns, edits sooner, and shows stronger verification behavior versus SWE‑1.7, changes likely to reduce reviewer friction, but you should validate on your codebase, CI, and security policies.
Should enterprises trust these results as‑is?

Treat them as promising vendor results. Ask for the harness, cost breakdowns, compute totals, λ values, and a short pilot with logs before committing significant workloads.

Final note, what to do next

SWE‑2 is a strong example of cost‑aware RL applied at scale and of the kinds of product choices vendors now make: optimize for operational dollars and ship models inside controlled agents. That combination can deliver real value for engineering teams, if the vendor‑reported numbers hold up under neutral evaluation and if buyers account for the human and CI costs that follow higher throughput.

If you’re evaluating SWE‑2 or similar coding agents, start by demanding the evaluation harness and absolute cost numbers, then run a focused 30‑ to 60‑day pilot against your representative PRs. The mechanics Cognition describes (Pareto‑informed λ, multi‑effort training, rollout optimizations) are plausible and worth testing, but procurement decisions should be driven by your own reproducible measurements, not vendor‑only tables.