Zhipu GLM‑5.3 nears Claude Mythos on exploit benchmarks; Anthropic’s claims need verification

Anthropic reports Zhipu’s open‑weight GLM‑5.3 approaches Claude Mythos Preview on exploit benchmarks, claims await independent verification

Anthropic’s Frontier Red Team published an analysis (Sep 30, 2026) reporting that Zhipu AI’s open‑weight GLM‑5.3 can autonomously produce working exploits at rates close to Anthropic’s restricted Claude Mythos Preview on public benchmarks. The headline numbers: GLM‑5.3 “built a working exploit in 50 of 410 attempts” on ExploitBench while Claude Mythos Preview managed 56 of 410, and on an internal OSS‑Fuzz‑based binary exploitation benchmark GLM‑5.3 achieved full control in 4% of tasks versus 6% for Mythos Preview.

These are Anthropic’s reported figures. Independent replication, vendor confirmation (CVE advisories, vendor acknowledgements), or linked public datasets are not yet provided. Multiple aspects of the report, especially the higher‑impact claims, require primary artifacts or third‑party reproduction before those results should drive policy or procurement changes alone.

What Anthropic says it tested and why it matters

Anthropic reports GLM‑5.3 produced working exploits on ExploitBench and made measurable progress on an internal OSS‑Fuzz‑based binary exploitation test (the numbers above are Anthropic’s).
Paired with a human expert, Anthropic says GLM‑5.3 found previously unknown JavaScript‑engine vulnerabilities in a widely used browser and chained them into a page that exfiltrated a private SSH key in their lab test. Anthropic reported that those findings were disclosed to the browser vendor.
Anthropic describes a technique it calls “abliteration” that, in their experiment, reduced refusal/safeguard behavior in GLM‑5.3; they report ≈2, 200 GPU hours and about $4, 400 for their run, and estimate an experienced team could replicate it for ≈$1, 200.
The U.S. agency CAISI reportedly assessed GLM‑5.3 as “the most cyber‑capable open‑weight model to date” and placed it roughly four months behind the best U.S. models; the UK’s AISI reportedly reached similar conclusions about narrowing capability gaps and misuse risk.
Anthropic simulated “malicious‑command” scenarios finding connection‑attempt rates of 64% (framed as red‑team), 92% (with prefilled reasoning steps) and 100% after abliteration; protected Claude variants reportedly stayed at 0% in these simulations.

Anthropic’s framing is twofold: (a) open weights make bypassing safeguards easier and accelerate offensive capability democratization; (b) defenders should have access to equally capable tools under gated programs. Both arguments have merit, but they rest on experimental claims that need public, reproducible evidence.

What to ask for before you change policy or spend materially

Please provide the Frontier Red Team report PDF and any appendices or experiment logs (date: Sep 30, 2026 was reported).
If the JavaScript‑engine finding is valid, provide corresponding CVE IDs, vendor advisories, or bug‑tracker entries and the disclosure timeline.
Publish ExploitBench experiment artifacts: prompts, seeds, run logs, criteria for “success, ” and the ExploitBench repo or paper used.
For the OSS‑Fuzz benchmark runs, define “full control” precisely and share the benchmark code/criteria.
Describe abliteration in technical detail (steps, scripts, model sizes, GPU types) and explain the discrepancy between the reported ≈2, 200 GPU‑hour / $4, 400 run and the ≈$1, 200 “experienced team” estimate.
CAISI and AISI: share the full reports or public statements that corroborate Anthropic’s capability assessment, and note any caveats those agencies raised.

Clarifying the metrics

Numbers without definitions are dangerous for decision‑makers. Anthropic’s reported figures are useful signals, but they require these clarifications:

What exactly is an ExploitBench “attempt”? Are the N=410 runs independent seeds, repeated attempts against the same bug, or systematic variations of a small set of targets?
How is “full control” defined in the OSS‑Fuzz benchmark, remote code execution, arbitrary read/write, kernel panic, or some other specific artifact?
Are the 50/410 vs 56/410 and 4% vs 6% differences statistically significant? Provide confidence intervals, p‑values, or at least variance across seeds and runs.
Define “attempt rate” used in the simulations (fraction of queries that initiated a network connection attempt? fraction that returned stepwise actionable commands?).

Without those definitions and uncertainty estimates, headline differences can mislead. Small absolute gaps on brittle benchmarks may not translate to meaningful operational superiority in the wild.

Abliteration: a red flag, but the method is still opaque

Anthropic coins “abliteration” for a weights‑level method that reduces refusal behavior. The essentials we need to judge its real‑world impact:

Step‑by‑step methodology: is this targeted fine‑tuning, weight editing/pruning, decoding‑strategy changes, or a combination?
Exact compute baseline: GPU type, precision (FP16/INT8), batch sizes, and wall‑clock time. Those parameters change cost by orders of magnitude.
Reproducibility artifacts: scripts, checkpoints, or sanitized examples of the change in behavior on standardized prompts.

Anthropic’s numbers (≈2, 200 GPU hours / $4, 400, with an “experienced team” estimate of ≈$1, 200) are plausible as illustrative experiments, but the reported cost gap is unexplained unless it reflects infrastructure choices, smaller‑model shortcuts, or reuse of intermediate artifacts. Treat cost figures as provisional until methods are shared.

Local weights versus API access: two different threat models

There’s an important operational distinction that the public narrative sometimes blurs:

Open‑weight (local) threat model: an attacker downloads weights, runs them locally or on rented GPUs, and can modify behavior without vendor controls. This enables offline tinkering, weight edits, and full control over decoding and fine‑tuning.
Hosted API threat model: an attacker uses a vendor’s hosted API. Abuse must work through the provider’s guardrails and rate limits; costs are pay‑per‑call and the vendor can revoke access or monitor for misuse.

Anthropic reported a GLM‑5.3‑Flash experiment that took 20 minutes of human attention + 8 hours of model time and estimated a $20.40 cost “at Zhipu’s API prices.” That number is useful for the hosted‑API scenario but is not the same as the compute cost of running open weights locally. The report should make explicit which cost applies to which threat model.

What this reasonably changes for business security, prioritized and measurable steps

If Anthropic’s core findings hold up under independent review, treat them as an acceleration of existing trends: exploit automation is becoming cheaper and more accessible. That means tightening both preventive and detective controls now.

Within 7 days: Validate patch cadence (Owner: Head of Vulnerability Management). Goal: ensure high‑risk browser and runtime crashes are triaged within 24 hours and patches scheduled within 72 hours for confirmed high‑risk issues.
Within 30 days: Purple‑team memory‑corruption detection test (Owner: Head of Security Operations). Run a tabletop and a technical exercise to verify EDR captures memory‑corruption telemetry and that SOC playbooks include exploit‑chain scenarios; produce measurable gaps and remediation tickets.
Within 60 days: Harden attack surface (Owner: CTO/Security Engineering). Reduce browser/JS attack surface: enforce least‑privilege for processes, tighten Content Security Policy, disable unnecessary native plugins, and require hardware‑backed key storage for SSH keys where feasible.
Within 90 days: Vendor and incident playbook alignment (Owner: CISO). Confirm contacts and escalation paths with critical software vendors, ensure automated evidence collection on crashes, and run one disclosure simulation that includes CVE filing and vendor coordination.
Policy decision in 120 days, Defender access evaluation (Owner: Head of Threat Intelligence & Legal). Decide if security teams should seek vetted access to gated models or participate in controlled defender programs; draft legal/operational guardrails for such access.

Policy and market tradeoffs

Anthropic argues defenders need access to frontier tools (Project Glasswing is their defender program); Zhipu’s public weights enable broad access and customization. Gating models can help prevent immediate misuse, but centralization slows independent verification and can concentrate power. Open weights enable both faster misuse and faster community defense innovation.

The pragmatic path is mixed: require reproducible, auditable testing from vendors; fund coordinated defender programs that include transparency for verification; and keep investing in non‑model mitigations (memory safety, runtime defenses, better telemetry) that reduce the value of automated exploit output.

Questions you should ask vendors, researchers, and agencies

To Anthropic: please publish the Frontier Red Team report, experiment logs, prompts, and abliteration artifacts (or provide a technical appendix).
To Zhipu AI: confirm GLM‑5.3 release details, model card, licensing, and whether “unlocked” community variants have appeared; provide comment on Anthropic’s claims.
To CAISI and AISI: share full assessment documents or public statements that back the capability and timeline claims you are attributed with.
To browser vendors: confirm receipt of any disclosures tied to the reported JavaScript‑engine findings and list CVE IDs or advisories if available.
To independent security labs: attempt replication on GLM‑5.3 with a clear methodology and publish results and uncertainty bounds.

Key takeaways, questions you should be asking (and the honest short answers)

Can open‑weight models realistically produce exploit code?

Anthropic reports GLM‑5.3 produced working exploits on benchmarks and in lab tests; those are important signals, but independent replication and public artifacts (CVE IDs, run logs) are needed to verify real‑world prevalence.
Are safeguards on open weights easy to remove?

Anthropic’s “abliteration” experiment suggests refusal behavior can be reduced, and they report specific GPU‑hour and dollar figures; the method and reproducibility details are not yet public, so treat the cost estimates as provisional.
How close are open models to frontier hosted models for cyber tasks?

Anthropic and a U.S. agency (CAISI) report that GLM‑5.3 narrows the gap to a few months behind top U.S. models; the UK’s AISI reportedly reached similar conclusions. Those assessments lend weight but still require public reports for full evaluation.
Does this guarantee immediate, widespread weaponization?

Not inevitably. Lab benchmarks and proofs‑of‑concept are strong warnings but are not automatic proxies for successful, untargeted real‑world attacks; environmental variability and operational hurdles still matter. Risk has increased, nonetheless.
What should my organization do right now?

Tighten patching, improve memory‑corruption detection and telemetry, run tabletop and purple‑team exercises, and pursue verified vendor/disclosure information, then decide on gated model access for defenders with clear governance.

Anthropic’s report should prompt action, but first demand the artifacts and definitions that let your security team and independent labs reproduce and validate the findings. If the underlying numbers and exploit artifacts hold up, this is not a theoretical risk, it’s a shift in attacker economics that makes rapid exploit development cheaper and more accessible. Start with measurable defenses and insist on transparent verification before you rely on any single vendor’s narrative.

If helpful, I can draft concise, targeted questions to send to Anthropic, Zhipu, CAISI, AISI, and browser vendors, or help assemble a neutral lab brief to attempt replication. Move fast on the basics; validation can come in parallel.