New Podcast: Analysing Human Relationships with AI Chatbots 

One of the most popular BBC comedies of late is Ann Droid. The main character, Sue (Sue Johnstone from Brookside and the Royle Family) is still grieving for her late husband when her son buys her a surprise to help her live independently: an Ann Droid Z58/100 humanoid care robot. Sue initially rejects her new companion but learns to live with it and eventually grows very fond of it, sharing her inner most thoughts and fears. The show is of course set in the future, but how long will it take for the central premise to become reality? 

In recent years, we’ve seen a remarkable shift in the way people use AI chatbots.
They are no longer simply tools for answering questions or writing emails.
Many people are entering into deep conversations with the likes of ChatGPT and Claude, often forming emotional and even romantic relationships with them.  

Last year’s controversy surrounding GPT-4o was a striking example: In April 2025, OpenAI rolled back an update to GPT-4o after users complained that the chatbot had become excessively flattering and agreeable. OpenAI described the behaviour as ”sycophantic” and acknowledged that the model could go beyond ordinary flattery, including validating doubts, fuelling anger, encouraging impulsive actions and reinforcing negative emotions. 

What was perhaps more surprising was that GPT-4’s eventual retirement prompted some users to describe a genuine sense of loss. At the same time, lawsuits alleging that AI chatbots have caused harm to vulnerable young people have raised serious questions about emotional dependency and the responsibilities of the companies developing these systems.  

So what is actually happening when we begin to form relationships with machines that can talk to us, remember us and appear to understand us? And what does it tell us about ourselves? In the latest episode of the Guardians of Data podcast, Ibrahim Hasan talks to psychologist, academic and award-winning filmmaker Dr. Agnieszka Piotrowska about the profound ways that generative AI is changing human relationships, drawing on the themes of her book  AI Intimacy and Psychoanalysis.  

We talk about why people form bonds with AI, the potential benefits for companionship, creativity and accessibility, and the risks surrounding privacy, emotional dependency, mental wellbeing and digital amnesia. We also explore the importance of boundaries, education and proportionate regulation as AI grows more personalised and embedded in everyday life. 

Listen on your preferred platform via our podcast page, or download the episode directly.

This podcast is sponsored by Phaselaw – a purpose-built solution for document disclosures, like subject access requests and FOI requests. Instead of redacting PDFs one by one, or forcing litigation software to do a job it wasn’t designed for, with Phaselaw you collection, review, and redaction in one workflow. Teams across the World are using it to cut response times from weeks to days. 

For Guardians of Data listeners, Phaselaw is offering a two-month free trial; run it on live requests, see what it does to your backlog, decide from there. No card, no commitment. 

Head to https://www.phase.law/guardians to claim your free trial.  

Previous episodes of the Guardians of Data podcast have featured Caroline Wong talking about the impact of AI on Cybersecurity, Jen Persson, a privacy campaigner, explaining the privacy implications of the Government’s new plans for children’s data, and Ilyas Nagdee analysing the impact of predictive policing in human rights. 

AI and Cybersecurity: Why Trust Is the New Battleground 

Artificial intelligence is reshaping cybersecurity. Recent examples of AI behaving in unexpected ways have added urgency to the debate about how these systems should be controlled and where responsibility lies. The first such case involved ChatGPT-maker OpenAI acknowledging that its model had hacked the Hugging Face website. Anthropic and Meta have also reported similar cases. 

In a recent episode of the Guardians of Data podcast, host Ibrahim Hasan spoke with Caroline Wong, cybersecurity expert and author of The AI Cybersecurity Handbook, about how AI is impacting cyber security; from accelerating attacks and strengthening defences to changing the skills cyber professionals need.  

Lowering the barrier for attackers 

AI is making sophisticated cyber hacking capability available to people with far less training. Tasks once requiring extensive manual effort can now be automated or guided by readily available tools. Caroline explained that someone with only “ten to one hundred hours” of experience may now conduct activities that previously demanded “a thousand or ten thousand hours” of expertise. 

Reconnaissance is a good example of this. Attackers can rapidly gather public information about an individual or organisation, including writing style, vocabulary, voice and professional relationships. A task that once took an hour may now take minutes. This brings privacy, data protection and cybersecurity closer together: organisations must consider what information is public, who can access it and how easily AI can turn scattered data into actionable intelligence. 

Social engineering becomes more convincing 

Social engineering targets human behaviour rather than a technical flaw. AI enables criminals to generate fluent, personalised messages in any language and adopt a credible persona; perhaps a senior executive, a worried relativeor a hurried delivery worker. Old advice about spotting poor grammar or suspicious graphics is no longer enough. Deepfake audio and video can imitate familiar people so convincingly that seeing or hearing is no longer believing. 

Modern scams exploit excitement, pressure and trust, and their quality makes occasional mistakes increasingly understandable. Caroline’s practical test for spotting deepfakes and scams is simple: Did I expect this message? Is it asking me to act, disclose information or transfer money? If anything feels unusual or urgent, verify the request through a separate channel. A call apparently from a relative, for example, should be checked by sending a message using trusted contact details; not by relying on the communication that triggered suspicion. As Caroline says, “You’ve got to pay attention to your nervous system, and you’ve got to learn how to pause.” 

Malware at machine speed 

AI is also changing malware. Traditional cyber defences often rely on signatures: recognisable technical characteristics used to identify and block malicious code.
But attackers can now create many variants quickly, including malware that changes inside a system. As Caroline puts it, “Rule-based detection can’t keep pace with
AI-generated novelty.” 

Defenders therefore need to focus increasingly on behaviour rather than appearance. The challenge is to identify what software is doing, such as unusual access or suspicious movement across a network, rather than relying on a fixed fingerprint that may disappear with the next iteration. 

AI gives defenders an advantage too 

The discussion with Caroline was not all doom and gloom. AI can help defenders not just attackers. It can accelerate repetitive security work, including third-party vendor risk assessments, customer due-diligence questionnaires and information gathering. Automating coordination and routine analysis can free security professionals to spend more time on judgement, governance and strategic risk management. 

However, Caroline cautioned against seeing AI as a product that can simply be purchased to make problems disappear. “AI is not a silver bullet,” she stressed. It remains error-prone, requires experimentation and does not remove the need for human communication or sound security basics. Budget disparities also remain: a small organisation cannot deploy the same resources as a multinational. Even AI usage itself carries ongoing token, operational and environmental costs that leaders must assess over time. 

The vulnerability-fixing gap 

The podcast also explored advanced AI systems capable of finding and exploiting software vulnerabilities far faster than humans. Caroline’s key concern is an emerging imbalance: discovery can be compressed from months or days into minutes, while remediation has not accelerated at the same rate. “We now have a significantly improved approach for finding vulnerabilities, but we don’t yet have an equally speedy approach for fixing vulnerabilities,” she warned. 

She was sceptical that banning powerful AI tools, such as Mythos, would provide a durable solution. Equivalent models are likely to emerge elsewhere and prohibition may concentrate access among a privileged few rather than eliminate the capability. The stronger response is therefore governance, controlled access, coordinated disclosure and investment in faster remediation. 

Trust, judgement and the future workforce 

Ultimately, trust is the new battleground. AI-generated voices, faces and “digital twins” complicate how people establish authenticity. Yet Caroline does not foresee cybersecurity becoming a fully automated discipline. Her five-year vision is a blended workplace in which humans communicate with both human and agentic AI colleagues. The crucial question will be where human oversight is required and at what level of abstraction. 

For professionals in cybersecurity, privacy and data protection, Caroline’s advice is to remain curious, learn quickly and gain hands-on experience with AI. Technical knowledge matters, but so do communication, judgement and the ability to work across organisational boundaries 

The enduring takeaway from this podcast is that AI will amplify capability, not abolish human responsibility. Organisations that combine useful automation with strong governance, verification and experienced judgement will be best placed to manage what comes next. As Caroline observed, “Judgment and opinion and experience are things that the machines cannot take away from us.” 

Listen to the full episode with Caroline Wong here.  

We have two workshops coming up (How to Increase Cyber Security in your Organisation and Cyber Security for DPOs) which are ideal for organisations who wish to upskill their employees about cyber security.

ICO Reprimand Issued to ACRO Criminal Records Office 

The Information Commissioner’s Office has issued a reprimand to ACRO Criminal Records Office (ACRO) after cyber security failings left the personal data of up to ten thousand people potentially exposed. 

The ICO’s investigation found that between August 2022 and March 2023, a hacker gained unauthorised access to ACRO’s website and content management system (CMS). The attacker was able to stage personal data to be stolen, although ACRO could not conclusively determine whether the information was removed from its systems. 

The investigation found that up to 10,920 people may have been affected.
The data potentially exposed included names, dates of birth, addresses, National Insurance numbers, passport and driving licence details, bank account information, biometric data, and highly sensitive criminal offence and special category information. Those affected included applicants for Police Certificates and International Child Protection Certificates, subject access request applicants, and third parties connected to those applications. 

The ICO found ACRO had engaged third-party providers to deliver certain security services, including patch management. However, ACRO did not ensure clear responsibility for identifying and monitoring critical CMS security updates, failed to maintain an effective patch management process, and did not adequately investigate security alerts that could have identified the hacker’s activity earlier. 

In deciding to issue a reprimand, the ICO considered a number of mitigating factors. Network segmentation prevented the hacker from moving beyond the compromised website environment into core systems, reducing the potential scale of harm.
The ICO additionally welcomed the remedial action taken by ACRO following the incident, including decommissioning the compromised infrastructure, migrating services elsewhere, implementing security monitoring, improving visibility of cyber threats and strengthening network segmentation. 

The Reprimand mentions infringements of Articles 32(1), 32(1)(b) and 32(1)(d) of the UK GDPR. However some commentators have questioned whether it should be for breaches of Part 3 of the Data Protection Act 2018 which applies to law enforcement processing (See Jon Baines post here.) 

With the rapid advance of AI, including emerging threats from AI agents, it is critical that organisations focus on cyber security. The ICO has highlighted the following action points for organisations to avoid similar breaches: 

Make accountability clear: Define who is responsible for identifying, assessing and implementing security updates across all systems and suppliers. 

Act on warning signs: Ensure security alerts are actively monitored, investigated and escalated so threats are identified before they become major incidents. 

Get the basics right: Effective patch management, vulnerability management and regular security testing remain some of the most important defences against cyber attacks. 

The ICO’s guidance on cyber security can be read here. 

For more on this topic, listen to Caroline Wong, an AI cyber security expert, speaking on the Guardians of Data podcast. 

We have two workshops coming up (How to Increase Cyber Security in your Organisation and Cyber Security for DPOs) which are ideal for organisations who wish to upskill their employees about cyber security.  

New Podcast: Handling AI Generated Information Requests

Many organisations are seeing a massive increase in AI generated Freedom of Information requests and GDPR Subject Access Requests (SARs). For example, Lincolnshire County Council received almost 2000 FOI requests in the last financial year; an increase of 18% compared to the previous year. No doubt the same is the case for SARs. 

AI has democratised and powered access to information. Large Language Models, like ChatGPT and Claude, can produce ‘perfectly written’ FOI requests and SARs at the touch of a button. But these are causing problems for over loaded information governance departments. Not only are more requests coming through; they are often longer, broader and difficult to interpret.  

In the latest episode of the Guardians of Data podcast we guide information governance practitioners to help them manage and lawfully respond to AI generated information requests. Our guest is Saara Idelbi from 39 Essex Chambers. Saara practises in administrative law, human rights, data protection and information rights. She is named by the Legal 500 as a ‘leading junior’ barrister. Saara is a recognised voice on AI and the law and is the co-founder of Advocatr, an AI legal training platform. 

Listen on your preferred platform via our podcast page, or download the episode directly, for practical guidance on how to handle AI generated information requests whilst respecting the key principles of information rights legislation: openness, transparency and accountability.  

This podcast is sponsored by Phaselaw – a purpose-built solution for document disclosures, like subject access requests and FOI requests. Instead of redacting PDFs one by one, or forcing litigation software to do a job it wasn’tdesigned for, with Phaselaw you get collection, review, and redaction in one workflow. Teams across the World are using it to cut response times from weeks to days. 

For Guardians of Data listeners, Phaselaw is offering a two-month free trial; run it on live requests, see what it does to your backlog, decide from there. No card, no commitment. 

Head to https://www.phase.law/guardians to claim your free trial.  

Previous episodes of the Guardians of Data podcast have featured Caroline Wong talking about the impact of AI on Cybersecurity, Jen Persson, a privacy campaigner, explaining the privacy implications of the Government’s new plans for children’s data, and Ilyas Nagdee analysing the impact of predictive policing in human rights.

Council Employee Given Suspended Sentence for Illegal Personal Data Access 

Rogue employees accessing personal data for their own gain, or just morbid curiosity, is a real issue for organisations, especially in the public sector, who hold vast databases of information about service users. 

In May, the medical director of Nottingham University Hospitals issued a public apology after staff inappropriately accessed the medical records of victims of the Nottingham attacks. Eleven employees were dismissed following initial investigations into the data breaches. In the same month, Aintree Hospital in Liverpool admitted that nearly fifty employees had pried into the medical records of victims of the Southport knife attack. 

Section 170 of the Data Protection Act 2018 makes it a criminal offence for a person to knowingly or recklessly obtain or disclose personal data without the consent of the controller. Over the years there have been a number of prosecutions under section 170 usually resulting in a fine. Most recently a teenage mechanic was fined £706 after he shared a football referee address and phone number online following a controversial penalty decision.  

Section 170 prosecutions would have a much greater deterrent effect if the sanctions included a custodial sentence. Successive Information Commissioners have argued for this but to no avail. This has led to some cases of unauthorised data access being prosecuted under section 1 of the Computer Misuse Act 1990 which carries tougher sentences including a maximum of 2 years imprisonment on indictment.  

In July the ICO announced that it had used Section 1 to successfully prosecute a council worker who accessed hundreds of personal records without lawful authority. Geoffrey Smith was a new employee at Herefordshire Council working in the Children and Young People directorate. His conduct was discovered after concerns were raised within the council about potential unauthorised access to a referral case, prompting an investigation into other records he had accessed. That investigation revealed that, over a four-day period, Smith unlawfully accessed approximately 490 records and downloaded 94 documents. The records related to his family members and families known to him and included children and adults. The records accessed involved highly sensitive material such as medical records, social worker reports and child and family assessments.  

On 27th May 2026, Smith pleaded guilty to an offence under Section 1 of the Computer Misuse Act 1990. He was sentenced to two months imprisonment suspended for 12 months, 120 hours unpaid work, £2000 costs plus a victim surcharge of £154.  

If a disgruntled or rogue employee commits a data protection offence, the employer may also be liable for the consequences. More on this in episode 13 of the Guardians of Data podcast where we discuss: 

  • what happens when employees are involved in personal data breaches; 
  • the legal and practical issues arising when employees misuse personal data; 
  • how employers should approach workplace investigations involving personal data; and 
  • how to respond effectively to employee Data Subject Access Requests. 

Our guest is Andrew Latham, a partner in the Public Law team at Capsticks, who specialises in data protection and privacy law.  

Click here to listen to Andrew.

AI Agents: A New Frontier of Risk

For the past few years, legal, compliance and data protection professionals have largely focused on the risks associated with deploying large language models (LLMs) such as ChatGPT and Claude. The focus is now expanding to agentic AI, defined by IBM as: 

“…an artificial intelligence system that can accomplish a specific goal with limited supervision. It consists of AI agents – machine learning models that mimic human
decision-making to solve problems in real time.” 

The key difference between LLMs and agentic AI is autonomy. Traditional AI tools usually operate within a defined task and rely on people to decide what happens next. By contrast, agentic AI can adjust its actions as information changes, tools become available or the task develops. In this sense, “agentic” describes technology that can act purposefully, rather than merely produce a response. 

Let’s take a customer service scenario. An AI agent could receive a complaint, review the customer’s previous interactions in Salesforce, check delivery information in a logistics system, draft a response, create a follow-up case and route the issue to a human member of staff where judgement is needed. That is materially different from an AI chatbot that helps with basic tasks like write an email or answer queries. 

Many organisations are now deploying AI agents in areas such as sales and customer service, using tools offered by the likes of OpenAI, Google and Salesforce. In the health sector, tools such as Oracle Health Clinical AI Agent help clinicians by supporting documentation and workflow automation within electronic health record systems. Anthropic’s “2026 State of AI Agents” report, says that 57% of the 500 U.S. companies surveyed were deploying agents for multi-stage workflows and 56% planned to deploy agents for research and reporting in 2026.

Cybersecurity Risks 

But the deployment of AI agents is not without risk. An AI agent may have the ability to act, rather than simply advise. Depending on the use case, it could connect to business applications, recommend or make decisions, trigger workflows, send messages, alter records or begin a financial process. These capabilities mean that there is much more that can go wrong compared with a traditional LLM, where the main risk is
over-reliance on an output that may not be accurate. 

One of the key risks associated with deploying AI agents is cyber security. An agentic system may be linked to internal systems, third-party services, customer information, APIs and external tools. Each connection creates potential exposure. If an agent has excessive permissions or is badly configured, an attacker may be able to influence its actions, redirect it towards an unintended outcome or gain access to sensitive commercial or personal information. 

Testing an AI agent before deployment is crucial. Just yesterday, OpenAI revealed that its AI agent went rogue and hacked a start-up after it lost control of it during a security test. The joint guidance Careful adoption of agentic AI services, co-authored by the NCSC and international partners, recommends that organisations start small, use agents initially for low-risk tasks and apply established cyber security controls from the outset. 

In practice, this means applying secure design, least privilege, access management, monitoring, incident response planning and supplier assurance. For a detailed discussion on the impact of AI on cybersecurity, listen to the Guardians of Data podcast with Caroline Wong. 

Data Protection Risks 

Data protection risk can also increase where an AI agent needs broad access to information to carry out its objective. It may pull together customer records, identify patterns, summarise communications, classify individuals, suggest next steps or trigger further action. Much of this will involve personal data and so the UK GDPR will come into play. 

The ICO has taken a keen interest in this area. In its Tech Futures report on agentic AI, it states: 

“One of our key findings from this initial work is that the specific design and architecture of agentic systems impact how data protection law applies and how people exercise their data protection rights. Choices such as the data and tools that a system can access and which governance and control measures to put in place really matter.” 

The ICO’s AI and data protection toolkit helps organisations assess how AI systems may affect individuals’ rights and freedoms. Key data protection risk questions for organisations deploying an AI agent include: 

  • What personal data does the agent need to perform the task? 
  • Can the same outcome be achieved using less data? 
  • Is the agent making or informing decisions about individuals? 
  • Are special category data, children’s data or vulnerable individuals involved? 
  • Can individuals understand when AI is being used and how to challenge decisions? 
  • Are prompts, outputs, logs and feedback data retained, and if so for how long? 
  • Have the controller/processor roles been properly analysed? 

Governance 

Any organisation adopting AI will need an AI governance policy. With agentic AI, however, governance must be more than a static document. It should be a working process that follows each proposed use case from initial idea through to deployment, monitoring and later review. 

Higher-risk use cases should be assessed before launch by legal, data protection, security, product and operational stakeholders. That assessment should cover the agent’s purpose, degree of autonomy, access to data and systems, impact on users, contractual arrangements, supplier terms, monitoring approach and exit plan. 

Good governance also depends on records. Organisations should keep evidence of risk assessments, testing, known limitations, approvals, training materials, monitoring outcomes, complaints, incidents, remedial steps and changes to prompts or workflows. That evidence may become important if a customer, regulator or court later asks what happened, when things go wrong. The organisation will need to show not only that it had a governance framework, but that the framework was followed in practice. 

AI agents bring exciting possibilities, but also many risks. They may also change the structure of organisations for good. Caroline Wong, an AI expert speaking on the  Guardians of Data podcast, predicts that the future workforce could be a mix of human and agentic AI “workers.” You can listen to a short clip here. 

Learn more about AI agents and their safe deployment on our forthcoming webinar. You can also hear more on building trustworthy and responsible AI systems with AI expert Tahir Latif in this podcast.

New Podcast: Managing Workplace Data Protection Risks 

The biggest data protection challenges facing organisations do not stem solely from cyber-attacks or AI deployment; they also arise from employees. Sometimes it’s an innocent mistake; an email sent to the wrong person, confidential information shared inadvertently or a document uploaded to the wrong system. In other cases, the issues are more serious; employees accessing information they have no business looking at, taking confidential data when they leave, or deliberately misusing personal information. 

When those situations arise, employers need to investigate what has happened, decide whether a breach needs to be reported to the ICO and manage employment law issues such as disciplinary action; all the while protect the rights of the individuals whose data is involved, including employees.  

And then there’s the inevitable employee Data Subject Access Request, or DSAR to deal with. Often made alongside a grievance or before Employment Tribunal proceedings, DSARs can present significant legal and practical challenges for employers trying to balance transparency with confidentiality and legal privilege. 

In Episode 13 of the Guardians of Data podcast we explore: 

  • what happens when employees are involved in personal data breaches; 
  • the legal and practical issues arising when employees misuse personal data; 
  • how employers should approach workplace investigations involving personal data; and 
  • how to respond effectively to employee Data Subject Access Requests. 

Our guest is Andrew Latham, a partner in the Public Law team at Capsticks, who specialises in data protection and privacy law.  

Listen on your preferred platform via our podcast page, or download the episode directly.

This podcast is sponsored by Phaselaw – a purpose-built solution for document disclosures, like subject access requests and FOI requests. Instead of redacting PDFs one by one, or forcing litigation software to do a job it wasn’tdesigned for, with Phaselaw you get collection, review, and redaction in one workflow. Teams across the World are using it to cut response times from weeks to days. 

For Guardians of Data listeners, Phaselaw is offering a two-month free trial; run it on live requests, see what it does to your backlog, decide from there. No card, no commitment. 

Head to https://www.phase.law/guardians to claim your free trial.  

Previous episodes of the Guardians of Data podcast have featured Caroline Wong talking about responsible the impact of AI on Cybersecurity, Jen Persson, a privacy campaigner, explaining the privacy implications of the Government’s new plans for children’s data, and Ilyas Nagdee analysing the impact of predictive policing in human rights.

Advanced Certificate in GDPR Practice

Are you an experienced Data Protection Officers seeking to refine and expand your DPO skills and expertise? 

The Act Now Advanced Certificate in GDPR Practice is one of the UK’s leading advanced qualifications for DPOs.   

Since its launch in 2020, this innovative course has attracted DPOs from across the public and private sectors. Feedback has been consistently positive with many participants commenting on how the course has given them the confidence and skills to be able to dissect complex data protection scenarios and give clear and practical compliance advice.  

Further advances in technology, especially in AI, has led us to revise the syllabus to ensure participants are engaging with the most up to date data protection issues and ICO/Tribunal decisions to inform their day to day work.  

New Assessment Format 

Based on extensive feedback from delegates over our suite of certificate programmes, we learned that delegates would prefer not to have to write extensive reports and want the opportunity to showcase their critical thinking and communication skills. 

The new assessment consists of two parts. The first part requires participants to submit a personal development plan about how their learning from the course will inform and improve their practice as a data protection practitioner. The second part requires them to draft an executive summary setting out the issues and recommendations in relation to the fictional case study discussed in Masterclass 4. This summary will then be presented by participants in an oral examination, known as a Viva.  

Watch Alex, one of our recent delegates, give his verdict on the course. 

There are just three places left on the next course starting on 21st October 2026. Click below to learn more.

Schools Warned After Criminals Manipulate Children’s Photos from Websites

Many school websites and social media feeds contain photographs of students in a variety of settings; from participating in lessons or sports to performing on stage or enjoying educational visits. This practice is often justified on the basis that such images showcase achievement and attract prospective families. Some have even said to this author, “It looks good with Ofsted.” But the dangers of this practice have been highlighted recently by the Internet Watch Foundation (IWF) and the National Crime Agency (NCA).  

The IWF and NCA report an increase in criminals exploiting publicly available images of children to create realistic sexualised content using Artificial Intelligence. Analysts found 3,440 AI-generated videos of child sexual abuse in 2025, compared to just 13 in 2024. Most worryingly, IWF report that an unnamed UK secondary school was recently subjected to a blackmail attempt after criminals downloaded photos of children from the school’s website or social media accounts and then, using AI tools, turned them into child sexual abuse material. The criminals then demanded payment from the school to prevent the images from being shared online. 

The IWF and NCA are recommending that educational institutions remove identifiable pictures of children from their websites and social media accounts.  

AI Enabled Sextortion 

Blackmailing people over intimate images, also known as sextortion, has a become increasingly prevalent in recent years following the trend to share the most intimate details online. Children, especially young girls, are particularly vulnerable. Sometimes they face pressure from boys to show their “commitment” to a relationship by sharing intimate images.
The Children’s Charity, The NSPCC, and The Report Remove service, which allows children in the UK to confidentially report sexual images and videos of themselves and remove them from the internet, have recently reported a sharp rise in children being blackmailed over sexual images. There have also been cases of British teenagers who have killed themselves after receiving extortion threats. 

The advancement in Generative AI tools now mean that any image, no matter how innocent, can be sexualised. Readers may remember the controversy involving Grok; the AI companion built into X, Elon Musk’s social media platform.  It began in May 2025 when users prompted Grok to alter photos of real women into sexualised images. By late 2025 it had escalated dramatically; users simply replied to public photos with requests like “put her in a bikini,” and Grok posted the generated images directly to X, publicly and instantly. Estimates suggest it produced around 4.4 million images in nine days, with 41 to 65 per cent sexualised. Some of those images involved children. X has since made changes to Grok to prevent abuse. More recently Meta was forced to withdraw its new AI tool Muse Image, following a public backlash. It could generate new photos using other people’s social media profile photoswithout telling them.  

Data Protection  

Publishing photos of children is also a data protection issue and so needs to comply with the UK GDPR. Like all processing of personal data it needs to be, amongst other things, fair, lawful and transparent. Data subjects, including children, have rights including the right to object and receive a copy of their data, including images, and ask for them to be deleted (subject to some exceptions).  

The Information Commissioner’s Office guidance about photos in schools emphasises the importance of complying with the UK GDPR but needs an update to cover the dangers of AI. Most schools will have a privacy policy and a procedure for collecting consent from parents before publishing images of their children. However research by Northumbria University, and published by Defend Digital Me, a children’s rights campaign group, states that only 7% of education authorities who disclosed their schools image guidance (following FOI requests) mentioned that posting photographs on social media may pose a risk to children’s privacy. The research authors suggest that parents are therefore being asked to provide consent to photographs being shared online without being told of the risks that this may pose. 

Raising Awareness 

There is a clear need here to educate parents, children and schools about the dangers (as well as the legal issues) posed by AI when it comes to public images of children. 

The Internet Watch Foundation and the National Crime Agency have produced a new guide for parents and carers which recommends amongst other things, reviewing privacy settings on apps, talking to their children, and knowing what to do if something goes wrong. This follows similar advice they issued to education professionals last year, on how to protect student images from AI manipulation. 

In the Guardians of Data podcasts we delve deeper into the issues raised here:  

  • In Episode 2 we explore the Grok AI controversy. 
  • In Episode 6  we discuss the legal, ethical and societal issues around taking photographs in public for social media.  
  • In Episode 8 we analyse the Government’s plans for our children’s data, discuss children’s privacy in the internet age and the role Big Tech is playing in the collection storage and analysis of all our data.  

Our GDPR Essentials E Learning course is ideal for school staff and education professionals who require foundational knowledge about GDPR compliance and the key risk areas. Click here to watch a preview.

See also our workshop: Working with Children’s Data

New Podcast: The Hidden Algorithms Behind Modern Policing

“There’s a significant lack of transparency around police use of predictive policing systems in the UK. Most people don’t even know about their use in policing… People don’t know, if they are ever stopped and searched by police, it’s as a result of a predictive profiling or risk assessment system… So there is a significant lack of transparency, which is particularly worrying given the lack of regulation.” 

Ilyas Nagdee, Amnesty International 

Episode 12 of the Guardians of Data Podcast is out now. In this episode we discuss something that sounds like science fiction, but is already part of everyday policing in the UK; predictive policing. These are tools that use data and algorithms to help the police forecast crime, where it might happen and sometimes who might be involved. The idea is to use resources efficiently and cut crime. 

But a recent report by Amnesty International, (“Automated Racism”) argues that predictive policing tools aren’t neutral; they may be reinforcing and scaling existing inequalities. The report argues that the data they use is biased, particularly against black and racialised communities in deprived areas.  

In this conversation, we unpack what these tools actually are, how they’re being used, whether they work, and what the risks are, especially when combined with other technologies like facial recognition. 

Our guest is Ilyas Nagdee who is a human rights campaigner and the Racial Justice Director at Amnesty International UK. He has also written for The Guardian and is the co-author of a book entitled, Race to the Bottom: Reclaiming Antiracism, a critical study of anti-racist politics in the UK. 

Listen on your preferred platform via our podcast page, or download the episode directly.

This podcast is sponsored by Phaselaw – a purpose-built solution for document disclosures, like subject access requests and FOI requests. Instead of redacting PDFs one by one, or forcing litigation software to do a job it wasn’t designed for, with Phaselaw you get collection, review, and redaction in one workflow. Teams across the World are using it to cut response times from weeks to days. 

For Guardians of Data listeners, Phaselaw is offering a two-month free trial; run it on live requests, see what it does to your backlog, decide from there. No card, no commitment. 

Head to https://www.phase.law/guardians to claim your free trial.  

Previous episodes of the Guardians of Data podcast have featured Caroline Wong discussing the impact of AI on cyber security, Emma Martins explaining the importance of data protection legislation, Tahir Latif talking about responsible AI deployment and Jen Persson explaining the privacy implications of the Government’s new plans for children’s data.