🔗 Share this article

Technology companies and child safety organizations will be granted permission to evaluate whether AI tools can produce child exploitation images under recently introduced UK legislation.

Significant Rise in AI-Generated Harmful Content

The declaration came as revelations from a protection monitoring body showing that reports of AI-generated child sexual abuse material have more than doubled in the past year, growing from 199 in 2024 to 426 in 2025.

Updated Legal Framework

Under the amendments, the authorities will allow approved AI companies and child protection organizations to inspect AI systems – the underlying technology for chatbots and visual AI tools – and verify they have sufficient safeguards to stop them from creating images of child exploitation.

"Fundamentally about preventing abuse before it happens," declared Kanishka Narayan, adding: "Specialists, under rigorous conditions, can now identify the risk in AI models promptly."

Addressing Regulatory Obstacles

The changes have been implemented because it is against the law to produce and own CSAM, meaning that AI developers and other parties cannot create such images as part of a evaluation process. Until now, officials had to delay action until AI-generated CSAM was published online before addressing it.

This legislation is designed to averting that issue by helping to stop the creation of those materials at their origin.

Legislative Structure

The changes are being introduced by the government as modifications to the crime and policing bill, which is also implementing a ban on owning, producing or distributing AI models designed to generate child sexual abuse material.

Real-World Consequences

This week, the minister visited the London headquarters of a children's helpline and heard a mock-up call to counsellors involving a account of AI-based abuse. The interaction depicted a adolescent seeking help after facing extortion using a explicit AI-generated image of themselves, constructed using AI.

"When I hear about children experiencing blackmail online, it is a source of extreme anger in me and rightful concern amongst families," he stated.

Alarming Data

A leading online safety foundation reported that cases of AI-generated exploitation material – such as online pages that may include multiple files – had significantly increased so far this year.

Instances of the most severe material – the gravest form of abuse – increased from 2,621 visual files to 3,086.

• Female children were overwhelmingly victimized, accounting for 94% of illegal AI depictions in 2025
• Depictions of newborns to two-year-olds increased from five in 2024 to 92 in 2025

Sector Reaction

The legislative amendment could "constitute a crucial step to guarantee AI products are safe before they are launched," stated the chief executive of the online safety organization.

"AI tools have made it so survivors can be victimised all over again with just a few clicks, providing offenders the ability to create potentially endless quantities of sophisticated, lifelike child sexual abuse material," she added. "Material which further commodifies survivors' trauma, and makes children, particularly female children, less safe both online and offline."

Counseling Session Information

Childline also published information of counselling sessions where AI has been mentioned. AI-related risks discussed in the conversations include:

• Using AI to evaluate body size, body and looks
• Chatbots dissuading young people from talking to trusted adults about abuse
• Being bullied online with AI-generated material
• Digital blackmail using AI-manipulated pictures

Between April and September this year, Childline delivered 367 support sessions where AI, conversational AI and related topics were mentioned, significantly more as many as in the equivalent timeframe last year.

Fifty percent of the mentions of AI in the 2025 sessions were related to psychological wellbeing and wellness, including utilizing chatbots for support and AI therapy apps.