Category: AI

Records Management and AI 

In 2025 Artificial Intelligence (AI) will continue to redefine the way we live, work, and interact. From improving healthcare outcomes to optimising supply chains, AI projects hold the promise of unprecedented advancements. Accuracy, explainability, transparency are often cited, amongst others, as key concepts in discussions about successful implementation of AI projects. However, one critical component is sometimes overlooked: good records management.  

Data Integrity and Quality 

The foundation of any AI system, especially Generative AI, is data. AI algorithms rely on vast amounts of data to learn, make predictions, and generate insights. Therefore, the accuracy, completeness, and reliability of this data are paramount. Good records management ensures that data is systematically collected, organised, and maintained throughout its lifecycle. By implementing rigorous records management practices, organisations can avoid the pitfalls of incomplete or inaccurate data, which can lead to flawed AI models and unreliable outcomes. 

In the healthcare sector, AI models are increasingly used to diagnose diseases and recommend treatment plans. The accuracy of these models depends on the quality of medical records and patient data. Poor records management can result in missing or erroneous data, potentially jeopardising patient safety and leading to incorrect diagnoses. Conversely, well-managed records provide a robust dataset for training AI algorithms, enhancing their accuracy and reliability. 

Compliance with Legal and Regulatory Requirements 

Good records management practices are essential for ensuring compliance with legal and regulatory requirements. AI projects often involve the collection and processing of personal data. The GDPR impose stringent requirements on how organisations handle this data. By maintaining accurate and up-to-date records of data collection, usage, storage, and disposal, organisations can demonstrate their commitment to data protection and privacy. Additionally, effective records management enables organisations to respond promptly to data access requests, audits, and inquiries, further enhancing compliance and transparency. 

Data Security and Risk Management 

Data breaches and cyber-attacks are significant threats to AI projects, as they can compromise the integrity and confidentiality of sensitive information. Good records management practices play a crucial role in mitigating these risks. By implementing robust data governance frameworks, organizations can establish clear protocols for data access, storage, and protection. 

Effective records management involves the use of encryption, access controls, and regular audits to safeguard data against unauthorized access and breaches. In the event of a security incident, well-managed records provide a clear trail of data activity, enabling organisations to quickly identify and address vulnerabilities. This proactive approach to data security not only protects the organisation’s assets but also fosters trust among stakeholders. 

Facilitating Data Integration and Interoperability 

AI projects often require the integration of data from multiple sources, including internal databases, external partners, and public datasets. Good records management practices facilitate seamless data integration and interoperability, ensuring that data from diverse sources can be combined and analysed effectively. 

By standardizing data formats, metadata, and classification schemes, records management enables organizations to harmonize disparate data sets and create a unified data repository. This interoperability is essential for the development of comprehensive AI models that leverage diverse data inputs to generate more accurate and holistic insights. Moreover, well-managed records provide a clear audit trail, allowing organisations to trace the provenance and lineage of data used in AI projects. 

Enhancing Accountability and Transparency 

Transparency and accountability are critical factors in the ethical deployment of AI systems. Stakeholders, including customers, regulators, and the public, demand visibility into how AI models are developed, trained, and used. Good records management practices provide the documentation and audit trails necessary to demonstrate accountability and transparency. 

For example, the development of an AI model for credit scoring requires documentation of the data sources, algorithms, and decision-making processes used. Effective records management ensures that this information is systematically recorded and readily accessible for review. In cases where AI decisions are challenged or questioned, well-maintained records provide the evidence needed to explain and justify the outcomes, thereby enhancing accountability and trust. 

Good records management is the linchpin of successful AI implementation. It ensures data integrity and quality, facilitates compliance with legal and regulatory requirements, enhances data security, enables data integration and interoperability, and promotes accountability and transparency. As AI continues to evolve and reshape industries, organisations must prioritise robust records management practices to unlock the full potential of their AI initiatives. By doing so, they can build a solid foundation for sustainable and ethical AI deployment, ultimately driving innovation and creating value for all stakeholders. 

Get ahead of the game with our Information and Records Management Practitioner Certificate.  Whether you are a records manager, Freedom of Information Officer or Data Protection Officer this practitioner level certificate will teach you the theory of records management alongside practical hands-on application. The next course starts in two weeks with a special introductory price. Places are limited, so please book now to avoid disappointment.  

New International Treaty on AI Signed 

In September the UK, EU, and US signed the Council of Europe Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law (AI Convention). It is the world’s first AI treaty including provisions to protect the public and their data, human rights, democracy and the rule of law. 

The Convention requires signatory countries to monitor the development of AI and ensure any technology using AI is managed within strict parameters. It also commits countries to act against activities which fall outside of these parameters and to tackle the misuse of AI models which pose a risk to public services and the wider public. 

The Convention sets out 3 over-arching safeguards: 

  • protecting human rights, including ensuring people’s data is used appropriately, their privacy is respected and AI does not discriminate against them 
  • protecting democracy by ensuring countries take steps to prevent public institutions and processes being undermined 
  • protecting the rule of law, by putting the onus on signatory countries to regulate AI-specific risks, protect its citizens from potential harms and ensure it is used safely 

The Convention does not apply directly; legislators in each jurisdiction have to implement it into their domestic law and there is a wide degree of freedom over how it is interpreted and applied. The European Commission has said the Convention will be implemented in the EU via the recently enacted EU AI Act which will become enforceable in stages over the next few years.  

The UK Position 

The UK has no AI regulation (yet). Despite media reports, the recent King’s Speech did not include a bill to regulate AI. The King said that the government would “seek to establish the appropriate legislation to place requirements on those working to develop the most powerful artificial intelligence models”. We expect a government consultation to be announced soon. However, it is likely that new AI requirements will be introduced in other forthcoming legislation e.g. the Product Safety and Metrology Bill. The published summary of this bill states that it aims to “support growth, provide regulatory stability, and deliver greater protection for consumers by addressing new product risks and opportunities, allowing the UK to keep pace with technological advances such as AI.” Managing AI in the context of product safety aligns with certain aspects of the EU AI Act.  

When an AI Bill does finally appear, it is likely to focus on the production of large language models (LLMs), the general-purpose technology that underpins AI products such as OpenAI’s ChatGPT and Microsoft’s Copilot. As the Labour election manifesto stated: 

“Labour will ensure the safe development and use of AI models by introducing binding regulation on the handful of companies developing the most powerful AI models and by banning the creation of sexually explicit deepfakes.” 

Whatever shape the UK’s AI regulation takes, the government will have to ensure that the AI Convention is implemented. Shabana Mahmood, Lord Chancellor and Justice Secretary, said:  

“Artificial intelligence has the capacity to radically improve the responsiveness and effectiveness of public services, and turbocharge economic growth. However, we must not let AI shape us – we must shape AI. This convention is a major step to ensuring that these new technologies can be harnessed without eroding our oldest values, like human rights and the rule of law.” 

If you are a DPO needing to stay abreast of the latest developments and best practices in AI implementation,  join our Artificial Intelligence and Machine Learning, How to Implement Good Information Governance workshop.

ICO 5th Call for Evidence on Generative AI 

Recently we wrote about how “How Generative AI’s Data Appetite is Fuelling Privacy Battles.” Last week the Information Commissioner’s Office (ICO) published its fifth call for evidence on Generative AI.  This call focuses on the allocation of accountability for data protection compliance across the generative AI supply chain. It is part of the ICO’s consultation series on generative AI ICO consultation series on generative AI and data protection. 

The fifth call for evidence addresses the recommendation for ICO guidance on the allocation of accountability in AI as a Service (AIaaS) contexts made in Sir Patrick Vallance’s Pro-innovation Regulation of Technologies Review.  
 
The allocation of accountability is complicated because of the different ways in which generative AI models, applications and services are developed, used and disseminated, but also the different levels of control and accountability that participating organisations may have.  
 
The ICO is interested in additional evidence on how this works in practice. In the meantime, it provides a summary of our current analysis, the policy positions we want to consult on and some examples which show how this analysis could be applied in practice.  
 
The deadline for submissions is 18th  September 2024.  

Enjoy reading our blog? Help us reach 10,000 subscribers by subscribing today! 
 
Join our Artificial Intelligence and Machine Learning, How to Implement Good Information Governance workshop for hands-on insights, key resource awareness, and best practices, ensuring you’re ready to navigate AI complexities fairly and lawfully. 

How Generative AI’s Data Appetite is Fuelling Privacy Battles

Like the monster plant in Little Shop of Horrors, Generative AI has an insatiable appetite; for data though rather than food. Generative AI applications, like ChatGPT and Midjourney, need a constant supply of data to train (and improve) their output algorithms. In the early days of AI development, this data came from public sources especially the internet. However, this “data scraping” was not without legal obstacles.

Where personal data is used to train AI models, of course GDPR applies. The transparency provisions and the requirement for a legal basis are of particular importance. In 2022, the Information Commissioner’s Office (ICO) issued a fine of more than £7.5 million to Clearview AI for GDPR breaches in the way it compiled its online database containing 20 billion images of people’s faces and data scraped from the internet.  The company did manage to successfully appealthe fine but the ICO, and other GDPR regulators in the EU, have issued clear warnings to AI companies to ensure they comply with GDPR.

To satisfy Generative AI’s demand for more data, AI developers have been striking deals with tech companies for access to the latter’s user data. This includes data generated by users whilst using popular websites and apps. In February it was reported that Tumblr and WordPress.com are preparing to sell user data to Midjourney and OpenAI. And (surprise surprise) Meta and Alexa have exploited user data, in the past, to train their AI models.

Elon Musk’s X (formerly Twitter) came under fire recently after it started collecting and using its users’ data, including their posts, to train X’s Grok AI model. This was allegedly done without notifying X users or asking for their consent. In June, the Irish Data Protection Commission (DPC), X’s Lead Supervisory Authority, made an urgent application under Section 134 of the Irish Data Protection Act 2018. This allows the DPC, where it considers there is an urgent need to act to protect the rights and freedoms of data subjects, to request the High Court for an order requiring the data controller to suspend, restrict or prohibit the processing of personal data.

This was the first time that any Lead Supervisory Authority has taken such action, and the first time that the DPC has sought to utilise its powers under Section 134. The DPC said the application was made to protect the rights and freedoms of X’s EU/EEA users, and came after extensive engagement between the DPC and X regarding its AI model training.  Last week, the DPC announced that X had agreed to suspend its processing of the personal data contained in the public posts of X’s EU/EEA users which it processed between 7 May 2024 and 1 August 2024, for the purpose of training its AI model.   

But this agreement is not the end of X’s privacy woes. Noyb, a privacy advocacy group headed by Max Schrems, has filed nine more GDPR complaints with regulators across Europe alleging that X appears to have breached a number of other GDPR provisions including the GDPR principles and the transparency rules. Several other major tech firms have also faced regulatory setbacks in Europe over privacy issues raised by their AI plans. In June Meta announced that it was pausing its plan to process user posts and images on Facebook and Instagram to train its AI tools after a number of GDPR complaints. LinkedIn was also the subject of a similar complaint by consumer organisations.

AI is a priority for the ICO. It’s existing guidance on AI explains how to apply the concepts of data protection law when developing or deploying AI and the AI toolkit helps organisations identify and mitigate risks during the AI lifecycle. The ICO consultation series on generative AI and data protection closed in June.

The training of Generative AI does not just pose GDPR compliance issues. In December last year, the New York Times announced it was suing OpenAI and Microsoft for copyright infringement. The lawsuit claimed the “unlawful use” of the paper’s “copyrighted news articles, in-depth investigations, opinion pieces, reviews, how-to guides, and more” to create AI products “threatens The Times’s ability to provide that service”.

Please subscribe to this blog and help us to get to 10,000 subscribers.

Join our Artificial Intelligence and Machine Learning, How to Implement Good Information Governance workshop for hands-on insights, key resource awareness, and best practices, ensuring you’re ready to navigate AI complexities fairly and lawfully.

The EU AI Act Comes into Force Today

The EU AI Act comes into force today although not all the provisions will become enforceable straight away. 

The Act sets out comprehensive rules for AI applications, including a risk-based system to address potential threats to health and safety, and human rights. It will ban certain AI applications that pose an “unacceptable risk,” including real-time and remote biometric identification systems such as facial recognition. Additionally, it will impose strict obligations on those considered “high risk,” encompassing AI used in
EU-regulated product safety categories, for example, cars and medical devices.
These obligations include adherence to data governance standards, transparency rules, and the incorporation of human oversight mechanisms. 

Detailed guides have been produced by lawyers Stephenson Harwood and Bird and Bird.

Here are the key dates for your diary:

  • August 1st, 2024: The AI Act enters into force
  • February 2025: Chapters I (general provisions) & II (prohibited AI systems) will apply
  • August 2025: Chapter III Section 4 (notifying authorities), Chapter V (general purpose AI models), Chapter VII (governance), Chapter XII (confidentiality and penalties), and Article 78 (confidentiality) will apply, except for Article 101 (fines for General Purpose AI providers)
  • August 2026: the whole AI Act applies, except for Article 6(1) & corresponding obligations (one of the categories of high-risk AI systems)
  • August 2027 – Article 6(1) & corresponding obligations apply.

In the UK, despite media reports, the King’s Speech did not include a bill to regulate AI. The King said that the government would “seek to establish the appropriate legislation to place requirements on those working to develop the most powerful artificial intelligence models”. Expect a government consultation to be announced soon.

Our AI Act workshop will help you understand the new law in detail and its interaction with the UK’s objectives and strategy for AI regulation.

The King’s Speech: What now for AI regulation and Data Protection reform?

The new Labour Government’s legislative programme was outlined in the King’s Speech at the State Opening of Parliament yesterday. Here are the key Bills information governance professionals need to look out for.

An AI Bill?

Despite media reports, the King’s Speech did not include a bill to regulate artificial intelligence(AI). The King said that the government would “seek to establish the appropriate legislation to place requirements on those working to develop the most powerful artificial intelligence models”. Expect a government consultation to be announced soon.

However, it is likely that new AI requirements will be introduced in other forthcoming legislation e.g the Product Safety and Metrology Bill. The published summary of this bill states that it aims to “support growth, provide regulatory stability, and deliver greater protection for consumers by addressing new product risks and opportunities, allowing the UK to keep pace with technological advances such as AI.” Managing AI in the context of product safety aligns with certain aspects of the EU AI Act. (see below)

When an AI Bill does finally appear, it is likely to focus on the production of large language models (LLMs), the general-purpose technology that underpins AI products such as OpenAI’s ChatGPT and Microsoft’s Copilot. As the Labour election manifesto says:

“Labour will ensure the safe development and use of AI models by introducing binding regulation on the handful of companies developing the most powerful AI models and by banning the creation of sexually explicit deepfakes.”

Meanwhile Europe is going full speed ahead on AI regulation. The EU AI Act will be on the EU statute books on 1st August 2024 and then become enforceable in stages. (A useful summary has been produced by lawyers at Stephenson Harwood.)

Cyber Security and Resilience Bill

A new Cyber Security and Resilience Bill will be introduced. It will expand regulation to cover more digital services and supply chains, empower regulators to ensure cyber security measures and mandate increased incident reporting to improve the government’s response to cyber-attacks including where a company has been held to ransom.

The Bill seems to be a response to recent high profile cyber-attacks. In June on Synnovis, the NHS service provider responsible for blood tests, swabs, bowel tests, and other critical services was the target of an attack affecting NHS patients across six London boroughs. Two major London hospital trusts had to cancel all non-emergency operations and blood tests.  It later transpired that, Qilin, a Russian cyber-criminal group, shared almost 400GB of private information on their darknet site.   

Digital Information and Smart Data Bill

No reference was made to data protection reform in the King’s Speech, but a Digital Information and Smart Data Bill was announced. The main provisions of the new Bill are:

  • Scientists will be able to ask for broad consent to use personal data for areas of scientific research, and allow legitimate researchers doing scientific research in commercial settings to make more use of personal data.
  • The Information Commissioner’s Office (ICO) will be transformed into a “more modern regulatory structure”, with a CEO, board and chair. It will also have new stronger powers.
  • The establishing of digital verification services including digital identity products to help people quickly and securely identify themselves when they use online services e.g. to help with things like moving house, pre-employment checks and buying age restricted goods and services. This is not the same as compulsory digital ID cards as some media outlets have reported.
  • The creation of a legal framework for Smart Data. This is the secure sharing of customer data, upon the customer’s (business or consumer) request, with authorised third-party providers (ATPs) who can enhance the customer data with broader, contextual ‘business’ data. These ATPs provide the customer with innovative services to improve decision making and engagement in a market. Open Banking is the only active example of a regime that is comparable to a ‘Smart Data scheme’ – but needs a legislative framework to put it on a permanent footing, from which it can grow and expand.

Most of these proposals are not particularly controversial and were in the Data Protection and Digital Information Bill  which failed to make it through Parliamentary “wash up” stage when the election was announced.

There may be more changes to come. We are told there will be “targeted reforms to some data laws that will maintain high standards of protection but where there is currently a lack of clarity impeding the safe development and deployment of some new technologies”.

There is much to chew over for IG professionals in the King’s Speech. As ever the devil will be in the detail (the Bills when published). Interesting times ahead.

This and other data protection developments will be discussed in detail on our forthcoming  GDPR Update  workshop.

AI Bill to be included in King’s Speech

A bill to regulate Artificial Intelligence(AI) will be one of 35 bills to be included in the King’s Speech tomorrow according to the Financial Times. The Bill will seek to enhance the legal safeguards surrounding the most cutting-edge AI technologies, according to people briefed on the plans.

The 2024 Labour election manifesto contained pledges to support the development of AI. It stated Labour would ensure their “industrial strategy supports the development of the AI sector and removes planning barriers to new datacentres.”  The Bill seeks to follow through on the manifesto pledge to regulate AI but only in some cases:

“Labour will ensure the safe development and use of AI models by introducing binding regulation on the handful of companies developing the most powerful AI models and by banning the creation of sexually explicit deepfakes.”

The Bill is likely to focus on the production of large language models (LLMs), the general-purpose technology that underlies AI products such as OpenAI’s ChatGPT.
It is a departure from the previous government’s approach which was not to place AI regulation on a statutory footing but to make use of “regulators’ domain-specific expertise to tailor the implementation of the principles to the specific context in which AI is used.” 

The new Bill follows the EU’s tougher approach.  The EU AI Act was published in the Official Journal of the EU last Friday (July 12th 2024) firing the gun for the enforcement countdown. It will be on the EU statute books on 1st August 2024 and then become enforceable in stages.

The main provisions of Act can be read here. In summary, the Act sets out comprehensive rules for AI applications, including a risk-based system to address potential threats to health and safety, and human rights. The Act will ban certain AI applications that pose an “unacceptable risk,” including real-time and remote biometric identification systems such as facial recognition. Additionally, it will impose strict obligations on those considered “high risk,” encompassing AI used in EU-regulated product safety categories, for example, cars and medical devices. These obligations include adherence to data governance standards, transparency rules, and the incorporation of human oversight mechanisms.

It will be interesting to read the text of the new Bill when it is published especially how it overlaps with the provisions on the UK GDPR.

Our AI Act workshop will help you understand the new law in detail and its interaction with the UK’s objectives and strategy for AI regulation.

EU AI Act Published in the EU Official Journal

The EU AI Act was published in the Official Journal last Friday (July 12th 2024) firing the gun for the enforcement countdown.

In summary, the Act sets out comprehensive rules for AI applications, including a risk-based system to address potential threats to health and safety, and human rights. The Act will ban certain AI applications that pose an “unacceptable risk,” including real-time and remote biometric identification systems such as facial recognition. Additionally, it will impose strict obligations on those considered “high risk,” encompassing AI used in EU-regulated product safety categories, for example, cars and medical devices. These obligations include adherence to data governance standards, transparency rules, and the incorporation of human oversight mechanisms. 

For a more detailed guide read the document produced by lawyers Stephenson Harwood.

Here are the key dates for your calendar:

  • August 1st, 2024: The AI Act will enter into force

  • February 2025: Chapters I (general provisions) & II (prohibited AI systems) will apply

  • August 2025: Chapter III Section 4 (notifying authorities), Chapter V (general purpose AI models), Chapter VII (governance), Chapter XII (confidentiality and penalties), and Article 78 (confidentiality) will apply, except for Article 101 (fines for General Purpose AI providers)

  • August 2026: the whole AI Act applies, except for Article 6(1) & corresponding obligations (one of the categories of high-risk AI systems)

  • August 2027 – Article 6(1) & corresponding obligations apply.


Our AI Act workshop will help you understand the new law in detail and its interaction with the UK’s objectives and strategy for AI regulation. 

Microsoft Recall Has a Privacy Problem

Microsoft’s upcoming feature, Recall has raised concerns about users’ privacy and its compliance with GDPR. The Information Commissioner’s Office (ICO) says it is contacting Microsoft for more information about the product

Recall captures encrypted snapshots of users’ screen and stores them locally on their computer. It is part of the new Copilot+ PCs. Microsoft insists that Recall is an “optional experience” designed with privacy in mind. Users can control which snapshots are collected, and Microsoft claims that no external parties, including themselves, can access these images without physical access to the device.

Despite Microsoft’s reassurances, the ICO is investigating the safeguards in place to protect user privacy. An ICO spokesperson said firms must “rigorously assess and mitigate risks to peoples’ rights and freedoms” before bringing any new products to market. “We are making enquiries with Microsoft to understand the safeguards in place to protect user privacy,” they said.

With the potential exposure of sensitive information including passwords, financial details, and personal queries captured in screenshots, Microsoft, as well as corporate users of the new feature, are going to have to evidence how they intend to comply with GDPR’s security obligations as set out in Article 32.

AI remains a key priority for the ICO. It has launched a series of consultations on how aspects of data protection law should apply to the development and use of generative AI models, building on its extensive guidance on data protection and AI. The ICO’s proactive stance underscores the importance of stringent robust user control and data protection measures when it comes to implementing AI powered tools. 

Join our Artificial Intelligence and Machine Learning, How to Implement Good Information Governance workshop for hands-on insights, key resource awareness, and best practices, ensuring you’re ready to navigate AI complexities fairly and lawfully.

EU AI Act Approved by European Parliament  

On Wednesday 13th March 2024, the European Parliament approved the text of the harmonised rules on artificial intelligence, the so-called  “Artificial Intelligence Act” (AI Act). Agreed upon in negotiations with member states in December 2023, the Act was endorsed by MEPs with 523 votes in favour, 46 against and 49 abstentions. It aims to “protect fundamental rights, democracy, the rule of law and environmental sustainability from high-risk AI, while boosting innovation and establishing Europe as a leader in the field.” Despite Brexit, UK businesses and entities engaged in AI-related activities will still be affected by the Act if they intend to operate within the EU market. The Act will have an extra territorial reach just like the EU GDPR 

The main provisions of Act can be read here. In summary, the Act sets out comprehensive rules for AI applications, including a risk-based system to address potential threats to health and safety, and human rights. The Act will ban some AI applications which pose an “unacceptable risk”, such as real-time and remote biometric identification systems like facial recognition, and impose strict obligations on others considered as “high risk”, such as AI in EU-regulated product safety categories such as cars and medical devices. These obligations include adherence to data governance standards, transparency rules, and the incorporation of human oversight mechanisms.  

Next steps 

The Act is still subject to a final lawyer-linguist check and is expected to be finally adopted before the end of the legislature (through the so-called corrigendum procedure). It also needs to be formally endorsed by the Council of Europe. 

The Act will enter into force twenty days after its publication in the official Journal, and be fully applicable 24 months after its entry into force, except for: bans on prohibited practises, which will apply six months after the entry into force date; codes of practise (nine months after entry into force); general-purpose AI rules including governance (12 months after entry into force); and obligations for high-risk systems (36 months after entry into force). 

Influence on UK AI Regulation 

The EU’s regulatory approach will impact the UK Government’s decisions on AI governance. An AI White Paper was published in March last year entitled  
“A pro-innovation approach to AI regulation”. The paper sets out the UK’s preference not to place AI regulation on a statutory footing but to make use of “regulators’ domain-specific expertise to tailor the implementation of the principles to the specific context in which AI is used.” In January 2024, the ICO launched  a consultation series on Generative AI, examining how aspects of data protection law should apply to the development and use of the technology. It is expected to issue more AI guidance later in 2024. 

Our AI Act workshop will help you understand the new law in detail and its interaction with the UK’s objectives and strategy for AI regulation.