Tag: Cybersecurity

AI Agents: A New Frontier of Risk

For the past few years, legal, compliance and data protection professionals have largely focused on the risks associated with deploying large language models (LLMs) such as ChatGPT and Claude. The focus is now expanding to agentic AI, defined by IBM as: 

“…an artificial intelligence system that can accomplish a specific goal with limited supervision. It consists of AI agents – machine learning models that mimic human
decision-making to solve problems in real time.” 

The key difference between LLMs and agentic AI is autonomy. Traditional AI tools usually operate within a defined task and rely on people to decide what happens next. By contrast, agentic AI can adjust its actions as information changes, tools become available or the task develops. In this sense, “agentic” describes technology that can act purposefully, rather than merely produce a response. 

Let’s take a customer service scenario. An AI agent could receive a complaint, review the customer’s previous interactions in Salesforce, check delivery information in a logistics system, draft a response, create a follow-up case and route the issue to a human member of staff where judgement is needed. That is materially different from an AI chatbot that helps with basic tasks like write an email or answer queries. 

Many organisations are now deploying AI agents in areas such as sales and customer service, using tools offered by the likes of OpenAI, Google and Salesforce. In the health sector, tools such as Oracle Health Clinical AI Agent help clinicians by supporting documentation and workflow automation within electronic health record systems. Anthropic’s “2026 State of AI Agents” report, says that 57% of the 500 U.S. companies surveyed were deploying agents for multi-stage workflows and 56% planned to deploy agents for research and reporting in 2026.

Cybersecurity Risks 

But the deployment of AI agents is not without risk. An AI agent may have the ability to act, rather than simply advise. Depending on the use case, it could connect to business applications, recommend or make decisions, trigger workflows, send messages, alter records or begin a financial process. These capabilities mean that there is much more that can go wrong compared with a traditional LLM, where the main risk is
over-reliance on an output that may not be accurate. 

One of the key risks associated with deploying AI agents is cyber security. An agentic system may be linked to internal systems, third-party services, customer information, APIs and external tools. Each connection creates potential exposure. If an agent has excessive permissions or is badly configured, an attacker may be able to influence its actions, redirect it towards an unintended outcome or gain access to sensitive commercial or personal information. 

Testing an AI agent before deployment is crucial. Just yesterday, OpenAI revealed that its AI agent went rogue and hacked a start-up after it lost control of it during a security test. The joint guidance Careful adoption of agentic AI services, co-authored by the NCSC and international partners, recommends that organisations start small, use agents initially for low-risk tasks and apply established cyber security controls from the outset. 

In practice, this means applying secure design, least privilege, access management, monitoring, incident response planning and supplier assurance. For a detailed discussion on the impact of AI on cybersecurity, listen to the Guardians of Data podcast with Caroline Wong. 

Data Protection Risks 

Data protection risk can also increase where an AI agent needs broad access to information to carry out its objective. It may pull together customer records, identify patterns, summarise communications, classify individuals, suggest next steps or trigger further action. Much of this will involve personal data and so the UK GDPR will come into play. 

The ICO has taken a keen interest in this area. In its Tech Futures report on agentic AI, it states: 

“One of our key findings from this initial work is that the specific design and architecture of agentic systems impact how data protection law applies and how people exercise their data protection rights. Choices such as the data and tools that a system can access and which governance and control measures to put in place really matter.” 

The ICO’s AI and data protection toolkit helps organisations assess how AI systems may affect individuals’ rights and freedoms. Key data protection risk questions for organisations deploying an AI agent include: 

  • What personal data does the agent need to perform the task? 
  • Can the same outcome be achieved using less data? 
  • Is the agent making or informing decisions about individuals? 
  • Are special category data, children’s data or vulnerable individuals involved? 
  • Can individuals understand when AI is being used and how to challenge decisions? 
  • Are prompts, outputs, logs and feedback data retained, and if so for how long? 
  • Have the controller/processor roles been properly analysed? 

Governance 

Any organisation adopting AI will need an AI governance policy. With agentic AI, however, governance must be more than a static document. It should be a working process that follows each proposed use case from initial idea through to deployment, monitoring and later review. 

Higher-risk use cases should be assessed before launch by legal, data protection, security, product and operational stakeholders. That assessment should cover the agent’s purpose, degree of autonomy, access to data and systems, impact on users, contractual arrangements, supplier terms, monitoring approach and exit plan. 

Good governance also depends on records. Organisations should keep evidence of risk assessments, testing, known limitations, approvals, training materials, monitoring outcomes, complaints, incidents, remedial steps and changes to prompts or workflows. That evidence may become important if a customer, regulator or court later asks what happened, when things go wrong. The organisation will need to show not only that it had a governance framework, but that the framework was followed in practice. 

AI agents bring exciting possibilities, but also many risks. They may also change the structure of organisations for good. Caroline Wong, an AI expert speaking on the  Guardians of Data podcast, predicts that the future workforce could be a mix of human and agentic AI “workers.” You can listen to a short clip here. 

Learn more about AI agents and their safe deployment on our forthcoming webinar. You can also hear more on building trustworthy and responsible AI systems with AI expert Tahir Latif in this podcast.

New Podcast: The Impact of AI on Cybersecurity  

“Today, it’s actually very, very easy for attackers to take a piece of malware and effectively launch one hundred different versions all at once.” 

Caroline Wong, Author and Cybersecurity Expert 

Episode 11 of the Guardians of Data Podcast is out now. In this episode we discuss how AI is reshaping trust, identity, cybersecurity, and organisational accountability.  

In recent weeks, governments, regulators and cyber security professionals have been gripped by the emergence of Mythos, the powerful AI model developed by Anthropic. Touted as capable of identifying software vulnerabilities at a level that rivals some of the world’s most skilled human researchers, the model has generated excitement, concern and intense debate.   

Against this backdrop, our guest on this podcast is an internationally renowned cybersecurity leader, educator and technology strategist. Caroline Wong is Chief Strategy Officer at Axari and the author of The AI Cybersecurity Handbook.  

In this conversation, Caroline explains how cybercriminals are using AI to launch sophisticate cyber-attacks. We also discuss how organisations can use the same technology to strengthen their cyber defences.  

But this conversation goes beyond the technical. We discuss why trust is becoming the central battleground in cybersecurity, how deepfakes and AI-generated content are reshaping the way we verify information, and why human judgment remains critical despite rapid advances in automation. We also take a closer look at Mythos itself and what it means for the future of cybersecurity.  

Whether you’re a privacy practitioner, cybersecurity professional or simply interested in understanding how AI is transforming the digital world around us, this is a conversation packed with practical insights and thought-provoking ideas.   

Listen on your preferred platform via our podcast page, or download the episode directly.

This podcast is sponsored by Phaselaw – a purpose-built solution for document disclosures, like subject access requests and FOI requests. Instead of redacting PDFs one by one, or forcing litigation software to do a job it wasn’t designed for, with Phaselaw you get collection, review, and redaction in one workflow. Teams across the World are using it to cut response times from weeks to days. 

For Guardians of Data listeners, Phaselaw is offering a two-month free trial; run it on live requests, see what it does to your backlog, decide from there. No card, no commitment. 

Head to https://www.phase.law/guardians to claim your free trial.  

Previous episodes of the Guardians of Data podcast have featured Tahir Latif talking about responsible AI deployment, Jen Persson, a privacy campaigner, explaining the privacy implications of the Government’s new plans for children’s data, Naomi Mathews and Ibrahim Hasan explaining the law on filming people in public for social media and Olu Odeniyi analysing recent cyber breaches and discussing the lessons learnt.

Transport for London Cyber Attack 

Transport for London (TfL) is currently dealing with a cyber attack that has targeted its computer systems. Sources within TfL have revealed that staff have been encouraged to work from home where possible, as the attack primarily affects the transport provider’s back-office systems at its corporate headquarters. TfL is collaborating closely with the National Crime Agency and the National Cyber Security Centre to respond to the incident. 

Shashi Verma, TfL’s Chief Technology Officer, said: 

“We have implemented several measures to address an ongoing cybersecurity incident within our internal systems. The security of our systems and customer data is of utmost importance, and we are continuously assessing the situation throughout this incident.”  

Mr Verma emphasised that, although a complete assessment is still underway, there is no current evidence of customer data being compromised. If it turns out that any personal data has been compromised, whether employee or customer data,  of course TfL will need to consider reporting the matter to the Information Commissioner’s Office (ICO) as a personal data breach under Article 33 of the UK GDPR. As a statutory body, failure to do so could lead to TfL being fined up to £8.7 million. If the ICO investigates and finds a breach of the DP Principles (e.g. security) this could rise to £17.5 million. 

Back in the day major cyber incidents involving personal data were sure to be the subject of an ICO fine. In 2018, British Airways and  Marriott International were fined £20 million and  £18.4 million respectively. More recently the ICO has issued more reprimands in line with its policy on public sector enforcement. It recently issued a reprimand to the Electoral Commission following the discovery that unspecified “hostile actors” had managed to gain access to copies of the electoral registers, from August 2021. On 26th June 2024, the ICO announced that it will now review the two-year trial before making a decision on the public sector approach in the autumn.  

This is not the first cyber attack on a major public service provider in the capital.  Last month the ICO announced that it had issued a GDPR Notice of Intent of £6.09 million to an NHS IT supplier. This comes after its findings that the company failed to adequately protect the personal data of 82,946 individuals in breach of Article 32 of the UK GDPR.  As a key IT and software provider for the NHS and other healthcare organisations across the country, Advanced often holds role of Data Processor for many of its clients. The breach in question occurred during a ransomware attack in August 2022. Hackers exploited a vulnerability through a customer account that lacked multi-factor authentication, gaining access to multiple health and care systems operated by Advanced. The compromised data included phone numbers, medical records, and even details on how to access the homes of 890 individuals receiving at-home care. 

We have two workshops coming up (How to Increase Cyber Security in your Organisation and Cyber Security for DPOs) which are ideal for organisations who wish to up skill their employees about cyber security. See also our Managing Personal Data Breaches Workshop.