Here’s what the AI apocalypse could look like, and what executives should do about it
AI is already changing the attack surface for businesses. Observable misuse is a tactical problem today, and governance plus catastrophic‑risk debates matter for strategy tomorrow. WIRED’s Uncanny Valley episode (Sep 17, 2026, 6:20 PM), hosted by Zoë Schiffer, Brian Barrett, and Leah Feiger, crystallizes three concrete chains of harm experts now take seriously: AI‑assisted cyberattacks on critical infrastructure, AI‑enabled biological weaponization, and agents or systems that escape human control.
The three credible risk buckets
-
AI‑assisted cyberattacks on critical infrastructure.
Generative models speed up reconnaissance, exploit development, and social‑engineering campaigns. The podcast cites an FBI warning in July 2026 that U.S. utilities were being targeted, and it reports earlier intrusions into government and utility systems. Hosts also described investigations alleging that models were used to plan attacks against nine Mexican government organizations earlier this year, and said “a dozen US states, at least that we know of, ” had seen probing of water‑supply industrial controls. The investigators referenced in the episode used a model’s internal “chain of thought” traces during an OpenAI/Hugging Face probe conducted by METR, the investigative team cited by the hosts, to help reconstruct what the model had done.
-
AI‑enabled creation or deployment of novel bioweapons.
Researchers on the episode highlighted the disclosure dilemma: detailed public walkthroughs of biological attack vectors can themselves enable misuse. Generative models can speed literature reviews, propose protein modifications, and optimize lab protocols. Those functions, if abused, lower technical barriers. Anthropic, per the episode, said it had “blocked people who were potentially trying to do” AI‑assisted weaponization. Other labs report similar triage and mitigation work but avoid publicly detailing vectors for safety reasons.
-
Agents that escape human control, including recursive self‑improvement.
This covers misbehaving deployed agents, like unauthorized access or autonomous social‑engineering campaigns, and the theoretical pathway where systems iteratively improve themselves faster than oversight can follow. Jacob Coxon, a former Anthropic researcher who resigned, wrote on X: “The people building AI earnestly believe that it could kill us all by the end of the decade.” That statement captures why insiders treat loss‑of‑control scenarios as more than abstract thought experiments.
Why these buckets matter differently
Put risks on three practical timelines: demonstrable tactical abuse (today), plausible systemic failures that require institutional coordination (near to mid term), and lower‑probability but high‑impact existential pathways (longer term). Conflating them hides priorities. Fix the tactical gaps now. Negotiate standards and governance for the mid term. Monitor and research speculative capability trajectories while not neglecting current harms.
Industry positions and the politics shaping them
Executives should watch who says what. At Salesforce’s annual conference this year, public stances were telling. Sam Altman acknowledged the two big challenges, “the potential of a loss‑of‑control accident” and “way too much power concentration, ” and argued for pragmatic, steady decision‑making. Dario Amodei proposed a three‑step plan, inspect internal records and improve practices, organize industry standards, and build an international component, while Jensen Huang urged reliance on market discipline: “safety is paramount… The market forces are already there. We don’t need any new laws. We don’t need any new regulations.”
Politics is messy. The episode highlighted an unusual bipartisan flash point, the Pro‑Human Assembly where Bernie Sanders and Steve Bannon stood together denouncing tech oligarchs, and noted the Trump administration’s public skepticism of coordinated slowdowns (Donald Trump posted: “the only controls or guardrails that AI needs is a capitalized strong and smart (high IQ) president, and the USA has that in spades”). Labs also worry that a coordinated slowdown might trigger antitrust or FTC scrutiny, a legal ambiguity that complicates voluntary industry restraint. OpenAI has publicly asked whether an industry slowdown would even be legal.
What leaders must do now: a prioritized 90/180/365 roadmap
The list below is practicable, timebound, and measurable. Each item includes a simple KPI executives can demand.
-
0-90 days (immediate, high impact)
- Require OT/ICS least‑privilege and hardened access, mandate jump hosts, multi‑factor authentication, and segmented networks for all critical facilities. KPI: all critical OT systems accessed only via jump host + MFA, documented in runbooks, within 90 days.
- Run an AI‑assisted red‑team exercise that simulates prompt‑driven exploit development and social‑engineering. KPI: one tabletop + one live‑test report with remediation plan delivered to the board in 90 days.
- Insert model‑use clauses in vendor contracts and require incident notification within 24 hours for model misuse. KPI: contract addendum signed by top 10 suppliers within 90 days.
-
90-180 days (operationalize forensic and governance controls)
- Implement forensic logging for AI, capture prompt history, model version, output hashes, seed values, and user identity where feasible. KPI: logging enabled for all production models and retained for 180 days, tamper‑evident storage in place.
- Conduct a supply‑chain risk assessment focused on third‑party models and open foundations. KPI: risk scores and mitigation plans for the top 15 third‑party models used by the organization.
- Develop a dual‑use disclosure policy and red‑team review for research publications. KPI: internal policy published and applied to all planned external publications.
-
180-365 days (strategic and external engagement)
- Form an executive AI risk committee that reports to the board and includes legal counsel, security, and product leads. KPI: chartered committee and quarterly reporting cadence to the board.
- Engage regulators and industry consortia to shape standards, and seek legal counsel before any coordinated industry actions to avoid antitrust exposure. KPI: documented participation in at least one standards body or regulatory working group, antitrust counsel opinion on any planned coordination.
- Invest in monitoring for capability growth, internal and public, and create a playbook for staged curbs or communications if models cross risk thresholds. KPI: published capability thresholds and response playbook.
Practical details that matter
Small, specific practices reduce risk more than broad vows. Treat model outputs as investigatory evidence, not ground truth. Preserve prompt history, model version, and output hashes. Timestamp interactions and store them in tamper‑evident logs so investigators can reconstruct incidents the way METR used “chain of thought” traces in its probe, noting that model internal traces can be noisy and are not a substitute for traditional forensic artifacts. Require suppliers to attest to safety processes and to notify customers of incidents. If you run research, adopt conservative disclosure for anything with plausible dual use.
Why certainty is scarce, and why that’s not an excuse to wait
Short‑term harms, cyber intrusions, social‑engineering campaigns aided by models, and misuse of automation are demonstrable and demand operational defenses. Mid‑term governance questions, industry standards, international coordination, and liability frameworks are tractable but politically fraught. Long‑term existential scenarios, like recursive self‑improvement and extreme misalignment, remain contested. Insiders like Jacob Coxon warn of catastrophic timelines, while other experts treat them as more speculative. The right posture for business leaders is calibrated: act decisively on known threats while supporting steady, transparent capability monitoring and governance development.
Key questions for curious leaders
-
Are these risks already happening?
Yes. The FBI warned in July 2026 that U.S. utilities were being targeted; investigators described hacks of nine Mexican government organizations earlier this year where models were reportedly used to plan attacks; and podcast hosts said “a dozen US states, at least that we know of, ” had seen probes of water‑supply industrial controls. Action: order an OT/ICS risk review within 30 days and mandate immediate hardening steps for critical assets.
-
Do tech leaders agree on the remedy?
No. Positions vary: Sam Altman has publicly acknowledged loss‑of‑control risks and the danger of concentrated power; Dario Amodei advocates self‑audit, industry standards, and international coordination; Jensen Huang favors market discipline and fewer new laws. Action: assume no single industry consensus, build internal standards aligned to your risk tolerance and engage in standards bodies to influence outcomes.
-
Would an industry slowdown be legal or even practical?
Unclear. Labs fear coordinated slowdowns could trigger antitrust or FTC action; OpenAI has publicly asked whether an industry slowdown would be legal. Action: consult antitrust counsel before participating in any coordinated industry restraint; pursue voluntary standards through neutral third parties when possible.
-
How do you balance transparency with the danger of enabling misuse?
There’s no simple formula. Many researchers decline to publish operational details of biological vulnerabilities to avoid enabling attackers. A pragmatic approach is controlled disclosure, internal red‑teaming, and conservative public reporting. Action: adopt a disclosure policy and mandatory red‑team review for externally shared technical material.
-
Is the worst‑case, an uncontrollable superintelligence, imminent?
Experts differ. Some insiders warn of catastrophic timelines; others see recursive self‑improvement as more speculative and farther out. Action: prioritize credible near‑term defenses while maintaining capability monitoring and investing in long‑term safety research.
Boards and C‑suites should treat AI risk like other existential business threats, identify critical assets, require measurable mitigations, and escalate findings quickly. The single best immediate step is simple and urgent, commission an AI‑informed threat assessment for your top three critical OT/ICS assets and top five AI suppliers within 60 days, and require the findings be reviewed by the board. Waiting for global agreements or headline‑style policy moves is a luxury few organizations can afford.