Tag: Cybersecurity

AI and Cybersecurity: Why Trust Is the New Battleground 

Artificial intelligence is reshaping cybersecurity. Recent examples of AI behaving in unexpected ways have added urgency to the debate about how these systems should be controlled and where responsibility lies. The first such case involved ChatGPT-maker OpenAI acknowledging that its model had hacked the Hugging Face website. Anthropic and Meta have also reported similar cases. 

In a recent episode of the Guardians of Data podcast, host Ibrahim Hasan spoke with Caroline Wong, cybersecurity expert and author of The AI Cybersecurity Handbook, about how AI is impacting cyber security; from accelerating attacks and strengthening defences to changing the skills cyber professionals need.  

Lowering the barrier for attackers 

AI is making sophisticated cyber hacking capability available to people with far less training. Tasks once requiring extensive manual effort can now be automated or guided by readily available tools. Caroline explained that someone with only “ten to one hundred hours” of experience may now conduct activities that previously demanded “a thousand or ten thousand hours” of expertise. 

Reconnaissance is a good example of this. Attackers can rapidly gather public information about an individual or organisation, including writing style, vocabulary, voice and professional relationships. A task that once took an hour may now take minutes. This brings privacy, data protection and cybersecurity closer together: organisations must consider what information is public, who can access it and how easily AI can turn scattered data into actionable intelligence. 

Social engineering becomes more convincing 

Social engineering targets human behaviour rather than a technical flaw. AI enables criminals to generate fluent, personalised messages in any language and adopt a credible persona; perhaps a senior executive, a worried relativeor a hurried delivery worker. Old advice about spotting poor grammar or suspicious graphics is no longer enough. Deepfake audio and video can imitate familiar people so convincingly that seeing or hearing is no longer believing. 

Modern scams exploit excitement, pressure and trust, and their quality makes occasional mistakes increasingly understandable. Caroline’s practical test for spotting deepfakes and scams is simple: Did I expect this message? Is it asking me to act, disclose information or transfer money? If anything feels unusual or urgent, verify the request through a separate channel. A call apparently from a relative, for example, should be checked by sending a message using trusted contact details; not by relying on the communication that triggered suspicion. As Caroline says, “You’ve got to pay attention to your nervous system, and you’ve got to learn how to pause.” 

Malware at machine speed 

AI is also changing malware. Traditional cyber defences often rely on signatures: recognisable technical characteristics used to identify and block malicious code.
But attackers can now create many variants quickly, including malware that changes inside a system. As Caroline puts it, “Rule-based detection can’t keep pace with
AI-generated novelty.” 

Defenders therefore need to focus increasingly on behaviour rather than appearance. The challenge is to identify what software is doing, such as unusual access or suspicious movement across a network, rather than relying on a fixed fingerprint that may disappear with the next iteration. 

AI gives defenders an advantage too 

The discussion with Caroline was not all doom and gloom. AI can help defenders not just attackers. It can accelerate repetitive security work, including third-party vendor risk assessments, customer due-diligence questionnaires and information gathering. Automating coordination and routine analysis can free security professionals to spend more time on judgement, governance and strategic risk management. 

However, Caroline cautioned against seeing AI as a product that can simply be purchased to make problems disappear. “AI is not a silver bullet,” she stressed. It remains error-prone, requires experimentation and does not remove the need for human communication or sound security basics. Budget disparities also remain: a small organisation cannot deploy the same resources as a multinational. Even AI usage itself carries ongoing token, operational and environmental costs that leaders must assess over time. 

The vulnerability-fixing gap 

The podcast also explored advanced AI systems capable of finding and exploiting software vulnerabilities far faster than humans. Caroline’s key concern is an emerging imbalance: discovery can be compressed from months or days into minutes, while remediation has not accelerated at the same rate. “We now have a significantly improved approach for finding vulnerabilities, but we don’t yet have an equally speedy approach for fixing vulnerabilities,” she warned. 

She was sceptical that banning powerful AI tools, such as Mythos, would provide a durable solution. Equivalent models are likely to emerge elsewhere and prohibition may concentrate access among a privileged few rather than eliminate the capability. The stronger response is therefore governance, controlled access, coordinated disclosure and investment in faster remediation. 

Trust, judgement and the future workforce 

Ultimately, trust is the new battleground. AI-generated voices, faces and “digital twins” complicate how people establish authenticity. Yet Caroline does not foresee cybersecurity becoming a fully automated discipline. Her five-year vision is a blended workplace in which humans communicate with both human and agentic AI colleagues. The crucial question will be where human oversight is required and at what level of abstraction. 

For professionals in cybersecurity, privacy and data protection, Caroline’s advice is to remain curious, learn quickly and gain hands-on experience with AI. Technical knowledge matters, but so do communication, judgement and the ability to work across organisational boundaries 

The enduring takeaway from this podcast is that AI will amplify capability, not abolish human responsibility. Organisations that combine useful automation with strong governance, verification and experienced judgement will be best placed to manage what comes next. As Caroline observed, “Judgment and opinion and experience are things that the machines cannot take away from us.” 

Listen to the full episode with Caroline Wong here.  

We have two workshops coming up (How to Increase Cyber Security in your Organisation and Cyber Security for DPOs) which are ideal for organisations who wish to upskill their employees about cyber security.

AI Agents: A New Frontier of Risk

For the past few years, legal, compliance and data protection professionals have largely focused on the risks associated with deploying large language models (LLMs) such as ChatGPT and Claude. The focus is now expanding to agentic AI, defined by IBM as: 

“…an artificial intelligence system that can accomplish a specific goal with limited supervision. It consists of AI agents – machine learning models that mimic human
decision-making to solve problems in real time.” 

The key difference between LLMs and agentic AI is autonomy. Traditional AI tools usually operate within a defined task and rely on people to decide what happens next. By contrast, agentic AI can adjust its actions as information changes, tools become available or the task develops. In this sense, “agentic” describes technology that can act purposefully, rather than merely produce a response. 

Let’s take a customer service scenario. An AI agent could receive a complaint, review the customer’s previous interactions in Salesforce, check delivery information in a logistics system, draft a response, create a follow-up case and route the issue to a human member of staff where judgement is needed. That is materially different from an AI chatbot that helps with basic tasks like write an email or answer queries. 

Many organisations are now deploying AI agents in areas such as sales and customer service, using tools offered by the likes of OpenAI, Google and Salesforce. In the health sector, tools such as Oracle Health Clinical AI Agent help clinicians by supporting documentation and workflow automation within electronic health record systems. Anthropic’s “2026 State of AI Agents” report, says that 57% of the 500 U.S. companies surveyed were deploying agents for multi-stage workflows and 56% planned to deploy agents for research and reporting in 2026.

Cybersecurity Risks 

But the deployment of AI agents is not without risk. An AI agent may have the ability to act, rather than simply advise. Depending on the use case, it could connect to business applications, recommend or make decisions, trigger workflows, send messages, alter records or begin a financial process. These capabilities mean that there is much more that can go wrong compared with a traditional LLM, where the main risk is
over-reliance on an output that may not be accurate. 

One of the key risks associated with deploying AI agents is cyber security. An agentic system may be linked to internal systems, third-party services, customer information, APIs and external tools. Each connection creates potential exposure. If an agent has excessive permissions or is badly configured, an attacker may be able to influence its actions, redirect it towards an unintended outcome or gain access to sensitive commercial or personal information. 

Testing an AI agent before deployment is crucial. Just yesterday, OpenAI revealed that its AI agent went rogue and hacked a start-up after it lost control of it during a security test. The joint guidance Careful adoption of agentic AI services, co-authored by the NCSC and international partners, recommends that organisations start small, use agents initially for low-risk tasks and apply established cyber security controls from the outset. 

In practice, this means applying secure design, least privilege, access management, monitoring, incident response planning and supplier assurance. For a detailed discussion on the impact of AI on cybersecurity, listen to the Guardians of Data podcast with Caroline Wong. 

Data Protection Risks 

Data protection risk can also increase where an AI agent needs broad access to information to carry out its objective. It may pull together customer records, identify patterns, summarise communications, classify individuals, suggest next steps or trigger further action. Much of this will involve personal data and so the UK GDPR will come into play. 

The ICO has taken a keen interest in this area. In its Tech Futures report on agentic AI, it states: 

“One of our key findings from this initial work is that the specific design and architecture of agentic systems impact how data protection law applies and how people exercise their data protection rights. Choices such as the data and tools that a system can access and which governance and control measures to put in place really matter.” 

The ICO’s AI and data protection toolkit helps organisations assess how AI systems may affect individuals’ rights and freedoms. Key data protection risk questions for organisations deploying an AI agent include: 

  • What personal data does the agent need to perform the task? 
  • Can the same outcome be achieved using less data? 
  • Is the agent making or informing decisions about individuals? 
  • Are special category data, children’s data or vulnerable individuals involved? 
  • Can individuals understand when AI is being used and how to challenge decisions? 
  • Are prompts, outputs, logs and feedback data retained, and if so for how long? 
  • Have the controller/processor roles been properly analysed? 

Governance 

Any organisation adopting AI will need an AI governance policy. With agentic AI, however, governance must be more than a static document. It should be a working process that follows each proposed use case from initial idea through to deployment, monitoring and later review. 

Higher-risk use cases should be assessed before launch by legal, data protection, security, product and operational stakeholders. That assessment should cover the agent’s purpose, degree of autonomy, access to data and systems, impact on users, contractual arrangements, supplier terms, monitoring approach and exit plan. 

Good governance also depends on records. Organisations should keep evidence of risk assessments, testing, known limitations, approvals, training materials, monitoring outcomes, complaints, incidents, remedial steps and changes to prompts or workflows. That evidence may become important if a customer, regulator or court later asks what happened, when things go wrong. The organisation will need to show not only that it had a governance framework, but that the framework was followed in practice. 

AI agents bring exciting possibilities, but also many risks. They may also change the structure of organisations for good. Caroline Wong, an AI expert speaking on the  Guardians of Data podcast, predicts that the future workforce could be a mix of human and agentic AI “workers.” You can listen to a short clip here. 

Learn more about AI agents and their safe deployment on our forthcoming webinar. You can also hear more on building trustworthy and responsible AI systems with AI expert Tahir Latif in this podcast.

New Podcast: The Impact of AI on Cybersecurity  

“Today, it’s actually very, very easy for attackers to take a piece of malware and effectively launch one hundred different versions all at once.” 

Caroline Wong, Author and Cybersecurity Expert 

Episode 11 of the Guardians of Data Podcast is out now. In this episode we discuss how AI is reshaping trust, identity, cybersecurity, and organisational accountability.  

In recent weeks, governments, regulators and cyber security professionals have been gripped by the emergence of Mythos, the powerful AI model developed by Anthropic. Touted as capable of identifying software vulnerabilities at a level that rivals some of the world’s most skilled human researchers, the model has generated excitement, concern and intense debate.   

Against this backdrop, our guest on this podcast is an internationally renowned cybersecurity leader, educator and technology strategist. Caroline Wong is Chief Strategy Officer at Axari and the author of The AI Cybersecurity Handbook.  

In this conversation, Caroline explains how cybercriminals are using AI to launch sophisticate cyber-attacks. We also discuss how organisations can use the same technology to strengthen their cyber defences.  

But this conversation goes beyond the technical. We discuss why trust is becoming the central battleground in cybersecurity, how deepfakes and AI-generated content are reshaping the way we verify information, and why human judgment remains critical despite rapid advances in automation. We also take a closer look at Mythos itself and what it means for the future of cybersecurity.  

Whether you’re a privacy practitioner, cybersecurity professional or simply interested in understanding how AI is transforming the digital world around us, this is a conversation packed with practical insights and thought-provoking ideas.   

Listen on your preferred platform via our podcast page, or download the episode directly.

This podcast is sponsored by Phaselaw – a purpose-built solution for document disclosures, like subject access requests and FOI requests. Instead of redacting PDFs one by one, or forcing litigation software to do a job it wasn’t designed for, with Phaselaw you get collection, review, and redaction in one workflow. Teams across the World are using it to cut response times from weeks to days. 

For Guardians of Data listeners, Phaselaw is offering a two-month free trial; run it on live requests, see what it does to your backlog, decide from there. No card, no commitment. 

Head to https://www.phase.law/guardians to claim your free trial.  

Previous episodes of the Guardians of Data podcast have featured Tahir Latif talking about responsible AI deployment, Jen Persson, a privacy campaigner, explaining the privacy implications of the Government’s new plans for children’s data, Naomi Mathews and Ibrahim Hasan explaining the law on filming people in public for social media and Olu Odeniyi analysing recent cyber breaches and discussing the lessons learnt.

Transport for London Cyber Attack 

Transport for London (TfL) is currently dealing with a cyber attack that has targeted its computer systems. Sources within TfL have revealed that staff have been encouraged to work from home where possible, as the attack primarily affects the transport provider’s back-office systems at its corporate headquarters. TfL is collaborating closely with the National Crime Agency and the National Cyber Security Centre to respond to the incident. 

Shashi Verma, TfL’s Chief Technology Officer, said: 

“We have implemented several measures to address an ongoing cybersecurity incident within our internal systems. The security of our systems and customer data is of utmost importance, and we are continuously assessing the situation throughout this incident.”  

Mr Verma emphasised that, although a complete assessment is still underway, there is no current evidence of customer data being compromised. If it turns out that any personal data has been compromised, whether employee or customer data,  of course TfL will need to consider reporting the matter to the Information Commissioner’s Office (ICO) as a personal data breach under Article 33 of the UK GDPR. As a statutory body, failure to do so could lead to TfL being fined up to £8.7 million. If the ICO investigates and finds a breach of the DP Principles (e.g. security) this could rise to £17.5 million. 

Back in the day major cyber incidents involving personal data were sure to be the subject of an ICO fine. In 2018, British Airways and  Marriott International were fined £20 million and  £18.4 million respectively. More recently the ICO has issued more reprimands in line with its policy on public sector enforcement. It recently issued a reprimand to the Electoral Commission following the discovery that unspecified “hostile actors” had managed to gain access to copies of the electoral registers, from August 2021. On 26th June 2024, the ICO announced that it will now review the two-year trial before making a decision on the public sector approach in the autumn.  

This is not the first cyber attack on a major public service provider in the capital.  Last month the ICO announced that it had issued a GDPR Notice of Intent of £6.09 million to an NHS IT supplier. This comes after its findings that the company failed to adequately protect the personal data of 82,946 individuals in breach of Article 32 of the UK GDPR.  As a key IT and software provider for the NHS and other healthcare organisations across the country, Advanced often holds role of Data Processor for many of its clients. The breach in question occurred during a ransomware attack in August 2022. Hackers exploited a vulnerability through a customer account that lacked multi-factor authentication, gaining access to multiple health and care systems operated by Advanced. The compromised data included phone numbers, medical records, and even details on how to access the homes of 890 individuals receiving at-home care. 

We have two workshops coming up (How to Increase Cyber Security in your Organisation and Cyber Security for DPOs) which are ideal for organisations who wish to up skill their employees about cyber security. See also our Managing Personal Data Breaches Workshop.