AI-generated CSAM surge: why boards must treat it as an operational risk

IWF: sharp rise in photorealistic AI-generated CSAM, and why boards should care

According to the Internet Watch Foundation (IWF), analysts assessed 6, 310 still images in the first half of 2026 that met the legal definition of child sexual abuse material (IWF H1 2026 report). Note: these figures cover still images only and the IWF’s public note does not fully disclose its detection or verification methodology. The H1 2026 total is reported to be about 40% higher than the organisation’s 2025 tally of more than 4, 500, and the IWF says it found more photorealistic material in H1 2026 than for the whole of 2025.

Photorealistic here means high-fidelity images that appear realistic. “AI-generated CSAM” in the IWF framing refers to synthetic or AI-produced still images that, in the organisation’s view, meet the legal threshold for child sexual abuse material. The IWF statement does not publish a full breakdown of how it distinguishes AI-generated from edited real images or which platforms hosted the content.

What else the IWF and other services reported

The IWF said the majority of images it assessed depicted girls and children aged “seven to 13.” The Report Remove service reported receiving 420 reports from children who believed images of themselves had been faked or manipulated in 2026, already exceeding its 2025 total of 397, according to the service. The IWF has previously warned of a separate surge in synthetic videos, but the 6, 310 figure cited above applies to still images only.

“It is incumbent on tech companies to build tools which cannot be abused this way, ” said Hannah Swirsky, head of policy at the IWF. “This problem has been created by technology, and if companies won’t build tools which are safe by design.”

On policy, the UK government reiterated that AI-generated CSAM is illegal under UK law. A government spokesperson said: “UK law is clear that child sexual abuse material is illegal, regardless of whether it is AI-generated or not. We are going further by banning AI models that are used to create this type of abuse material and produce sickening, hyper-realistic imagery that often contains the likeness of real children.” The spokesperson also said that developing a model designed to produce hyper-realistic CSAM can carry a criminal sentence of “up to five years.” Kanishka Narayan, the UK’s AI minister, described the policy stance this way: “nothing is off the table” and the guiding question for any proposal should be “Will it make the British people safer?”

Why this is an operational problem, not just a headline

This is an operational problem with immediate human, legal and reputational consequences. Three interlocking realities matter for business leaders:

Direct victim harm and reputational exposure. Synthetic CSAM can use the likenesses of real children and circulate widely. Platforms that fail to remove it quickly face regulatory scrutiny and public outrage.
Legal and regulatory risk. Regulators are signalling expectations for “safe by design” models and stronger platform duties. In the UK, public statements indicate criminal liability for creation, adaptation or distribution of models used for this content.
Technical escalation and moderation strain. Advances in generative models are improving realism faster than some detection systems can keep up. That creates moderation backlogs and increases false positives and false negatives.

Practical, measurable steps for executives (with timelines)

Executives don’t need to build models themselves, but they must own safety outcomes. Below are immediate, pragmatic actions with suggested ownership and timelines.

30 days, vendor audit (CPO/Head of Procurement). Require documented answers from each AI provider on training-data exclusions for minors, whether the model emits provenance metadata or watermarks, API access controls, and sample data lineage. Ask for signed attestation of content safeguards.
60-90 days, operational controls (CTO/Head of Product). Implement per-API-key rate limits, anomaly detection for high-volume generation, and human-in-the-loop gates for sexual-content requests. Recommended KPIs to track: median detection latency < 24 hours, median takedown time < 48 hours, and a target reduction in false negatives month-over-month.
90 days, tabletop and resourcing (Chief Legal/Head of Trust & Safety). Run a cross-functional incident tabletop focused on synthetic CSAM scenarios, involving legal, comms, law enforcement liaison, and outside NGOs (e.g., IWF). Confirm reporting workflows and mandatory notifications across jurisdictions.
120-180 days, privacy defaults and UX (Head of Product/Design). Push stronger default privacy settings for accounts with children: private profiles by default, “close friends” sharing, and clearer parental controls. Communicate changes to users and track adoption metrics.
Ongoing, moderation quality controls (Head of Trust & Safety). Combine automated triage with trained human reviewers, adopt dual-review for high-risk classifications, and maintain an appeals path to reduce wrongful takedowns. Track reviewer accuracy and time-to-resolution.

Sample language for vendor contracts: “Provider warrants that models and APIs will not be adapted or used to generate sexual content involving minors; Provider will implement and maintain provenance metadata or robust watermarking; Provider will notify Customer within 24 hours of any identified misuse vulnerabilities.” Make these clauses auditable and enforceable.

Where the public data is weak, and what to ask industry and government for

The IWF figures are stark. Several important methodological and scope questions remain unanswered in the public notes. How exactly does the IWF distinguish AI-generated from edited real imagery? Which platforms and channels host the majority of material? What share of images use the identifiable likenesses of real children? How many incidents reflect mass-generation via a small number of models versus organic, distributed misuse?

Priority asks businesses and policymakers should make now:

Standardized incident reporting: Platforms should publish anonymized, machine-readable reports that include content type (still/video), detection method, platform/channel, and removal timelines.
Shared, privacy-safe datasets for detection research: Shared, privacy-safe datasets that enable academic and vendor research into robust detection without re-exposing victims.
Legal clarity on liability: Clear rules that differentiate responsibilities for model providers, model distributors, and hosting platforms, plus guidance on cross-border evidence sharing and victim protections.

Hypothetical incident (illustrative)

Imagine a mid-sized app that exposes an image-generation API with permissive rate limits and a leaked developer key. Within hours, malicious actors mass-generate photorealistic images. Automated filters flag some content but miss others. User reports trickle in. Without a tabletop, the app lacks a law-enforcement contact and takes 72 hours to remove the worst content. The result: regulatory inquiry, brand damage, and avoidable harm to victims. That chain, permissive access, inadequate monitoring, slow escalation, is preventable with the controls listed above.

Prioritized asks for regulators and industry bodies

Mandate minimum transparency and reporting standards for synthetic CSAM incidents.
Require provenance metadata or watermarking standards for major generative models where technical feasibility exists.
Fund cross-sector detection research and establish secure sharing channels for anonymized examples to improve detection tools.

Key takeaways, questions you should ask, and short answers

Has AI-generated CSAM increased recently?

According to the Internet Watch Foundation (IWF), analysts assessed 6, 310 AI images in H1 2026 that met the legal definition of child sexual abuse, a reported figure higher than the IWF’s total for all of 2025. These figures apply to still images only.
Do the IWF figures include videos?

No, the 6, 310 figure refers to still images only. The IWF has previously reported surges in synthetic videos, but the new numbers cited are for images.
Are there legal penalties for creating or distributing models that generate this content?

The UK government has said AI-generated CSAM is illegal and that creating, adapting or distributing models used for this purpose can attract criminal liability; a government spokesperson cited a maximum sentence of “up to five years” for developing a model designed to produce hyper-realistic CSAM.
What immediate steps should companies take?

Within 30 days: audit AI vendors for exclusions and provenance support. Within 90 days: implement API access controls, human-in-the-loop gating, and a tabletop incident plan. Ongoing: combine automated detection with trained human review and coordinate with NGOs and law enforcement.

Generative AI delivers tangible business value, but these numbers are a reminder that without clear accountability, design constraints, and measurable operational controls, the technology can amplify severe harms. Boards should treat synthetic CSAM as a live operational risk: ask for transparent vendor attestations, demand incident reporting metrics, resource trust & safety teams, and press regulators for standardized reporting and legal clarity. The next 12 months will test whether industry and government move at the speed this harm requires.

For product teams: consider stronger default privacy settings and deliberate UX patterns that reduce accidental exposure for minors. Ensure cross-border playbooks account for data-protection rules and rapid evidence preservation.