AI image-generators are being trained on child abuse, other paedophile content content, finds study

In a recent investigation, it has been revealed that widely-used AI image-generators conceal a troubling flaw — thousands of images depicting child sexual abuse.

The disturbing findings come from a report issued by the Stanford Internet Observatory, urging companies to address and rectify this alarming issue within the technology they have developed.

The report unveils that these AI systems, entrenched with images of child exploitation, not only generate explicit content featuring fake children but can also manipulate photos of fully clothed teenagers into something inappropriate.

Up until now, anti-abuse researchers, assumed that AI tools producing abusive imagery combined information from adult pornography and harmless images of children by picking them up from other places on the internet. However, the Stanford Internet Observatory discovered over 3,200 images of suspected child sexual abuse within the LAION AI database itself.

LAION, a massive index of online images and captions, is used to train prominent AI image-making models, such as Stable Diffusion.

In response to the report, LAION has temporarily removed its datasets. The organization emphasizes a zero-tolerance policy for illegal content and states that the removal is a precautionary measure to ensure the datasets’ safety before republishing them.

Although these problematic images constitute a fraction of LAION’s vast index of 5.8 billion images, the Stanford group argues that they likely impact the AI tools’ ability to generate harmful outputs.

Additionally, the report suggests that the presence of these images reinforces the prior abuse of real victims who may appear multiple times.

The report highlights the challenges in addressing this issue, attributing it to the rushed development and widespread accessibility of many generative AI projects due to intense competition in the field.

The Stanford Internet Observatory calls for more rigorous attention to prevent the inadvertent inclusion of illegal content in AI training datasets.

Stability AI, a prominent LAION user, acknowledges the issue and asserts that it has taken proactive steps to mitigate the risk of misuse. However, an older version of Stable Diffusion, identified as the most popular model for generating explicit imagery, remains in circulation.

The Stanford report urges drastic measures, including the removal of training sets derived from LAION and the disappearance of older versions of AI models associated with explicit content. It also calls on platforms like CivitAI and Hugging Face to implement better safeguards and reporting mechanisms to prevent the generation and distribution of abusive images.

In response to the findings, tech companies and child safety groups are urged to adopt measures similar to those used for tracking and taking down child abuse materials in videos and images. The report suggests assigning unique digital signatures or “hashes” to AI models to track and remove instances of misuse.

While the prevalence of AI-generated images among abusers is currently small, the Stanford report emphasizes the need for developers to ensure their datasets are free of abusive materials, and ongoing efforts to mitigate harmful uses as AI models circulate.

(With inputs from agencies)

AI image-generators are being trained on child abuse, other paedophile content content, finds studyRead More