Many school websites and social media feeds contain photographs of students in a variety of settings; from participating in lessons or sports to performing on stage or enjoying educational visits. This practice is often justified on the basis that such images showcase achievement and attract prospective families. Some have even said to this author, “It looks good with Ofsted.” But the dangers of this practice have been highlighted recently by the Internet Watch Foundation (IWF) and the National Crime Agency (NCA).
The IWF and NCA report an increase in criminals exploiting publicly available images of children to create realistic sexualised content using Artificial Intelligence. Analysts found 3,440 AI-generated videos of child sexual abuse in 2025, compared to just 13 in 2024. Most worryingly, IWF report that an unnamed UK secondary school was recently subjected to a blackmail attempt after criminals downloaded photos of children from the school’s website or social media accounts and then, using AI tools, turned them into child sexual abuse material. The criminals then demanded payment from the school to prevent the images from being shared online.
The IWF and NCA are recommending that educational institutions remove identifiable pictures of children from their websites and social media accounts.
AI Enabled Sextortion
Blackmailing people over intimate images, also known as sextortion, has a become increasingly prevalent in recent years following the trend to share the most intimate details online. Children, especially young girls, are particularly vulnerable. Sometimes they face pressure from boys to show their “commitment” to a relationship by sharing intimate images.
The Children’s Charity, The NSPCC, and The Report Remove service, which allows children in the UK to confidentially report sexual images and videos of themselves and remove them from the internet, have recently reported a sharp rise in children being blackmailed over sexual images. There have also been cases of British teenagers who have killed themselves after receiving extortion threats.
The advancement in Generative AI tools now mean that any image, no matter how innocent, can be sexualised. Readers may remember the controversy involving Grok; the AI companion built into X, Elon Musk’s social media platform. It began in May 2025 when users prompted Grok to alter photos of real women into sexualised images. By late 2025 it had escalated dramatically; users simply replied to public photos with requests like “put her in a bikini,” and Grok posted the generated images directly to X, publicly and instantly. Estimates suggest it produced around 4.4 million images in nine days, with 41 to 65 per cent sexualised. Some of those images involved children. X has since made changes to Grok to prevent abuse. More recently Meta was forced to withdraw its new AI tool Muse Image, following a public backlash. It could generate new photos using other people’s social media profile photoswithout telling them.
Data Protection
Publishing photos of children is also a data protection issue and so needs to comply with the UK GDPR. Like all processing of personal data it needs to be, amongst other things, fair, lawful and transparent. Data subjects, including children, have rights including the right to object and receive a copy of their data, including images, and ask for them to be deleted (subject to some exceptions).
The Information Commissioner’s Office guidance about photos in schools emphasises the importance of complying with the UK GDPR but needs an update to cover the dangers of AI. Most schools will have a privacy policy and a procedure for collecting consent from parents before publishing images of their children. However research by Northumbria University, and published by Defend Digital Me, a children’s rights campaign group, states that only 7% of education authorities who disclosed their schools image guidance (following FOI requests) mentioned that posting photographs on social media may pose a risk to children’s privacy. The research authors suggest that parents are therefore being asked to provide consent to photographs being shared online without being told of the risks that this may pose.
Raising Awareness
There is a clear need here to educate parents, children and schools about the dangers (as well as the legal issues) posed by AI when it comes to public images of children.
The Internet Watch Foundation and the National Crime Agency have produced a new guide for parents and carers which recommends amongst other things, reviewing privacy settings on apps, talking to their children, and knowing what to do if something goes wrong. This follows similar advice they issued to education professionals last year, on how to protect student images from AI manipulation.
In the Guardians of Data podcasts we delve deeper into the issues raised here:
- In Episode 2 we explore the Grok AI controversy.
- In Episode 6 we discuss the legal, ethical and societal issues around taking photographs in public for social media.
- In Episode 8 we analyse the Government’s plans for our children’s data, discuss children’s privacy in the internet age and the role Big Tech is playing in the collection storage and analysis of all our data.
Our GDPR Essentials E Learning course is ideal for school staff and education professionals who require foundational knowledge about GDPR compliance and the key risk areas. Click here to watch a preview.
See also our workshop: Working with Children’s Data
