How should the *Government* utlise AI?
The 'Generative AI framework for HM Government' gives us some insight as to how Whitehall plans to use powerful AI to govern - but is it safe?
Is having a slightly more productive civil service worth giving sensitive data access to black box AGIs?
A ‘Generative framework for HM Government’
The UK government’s Central Digital and Data Office has just released their 'Generative AI framework for HM Government'.
Its aim is to ‘to help readers understand generative AI, to guide anyone building generative AI solutions, and, most importantly, to lay out what must be taken into account to use generative AI safely and responsibly.’
Government use of ‘generative AI’ (which they use to mean Large Language Models) will unlock ‘significant productivity benefits’ for workers - (I agree! If used well. Some research shows that productivity only increases if the LLMs are correctly prompted, and can actually decrease productivity if prompts are suboptimal.)
They define Generative AI as a ‘broad field which aims to use computers to emulate the products of human intelligence or to build capabilities which go beyond human intelligence’, so I’m happy to call this AGI hereafter.
The paper gives us 10 (concise) common principles to guide the safe use of AI:
You know what generative AI is and what its limitations are.
You use generative AI lawfully, ethically and responsibly.
You know how to keep generative AI tools secure.
You have meaningful human control at the right stage.
You understand how to manage the full generative AI lifecycle.
You use the right tool for the job.
You are open and collaborative.
You work with commercial colleagues from the start.
You have the skills and expertise that you need to build and use generative AI.
You use these principles alongside your organisation’s policies and have the right assurance in place.
(This all sounds like a job advert for an AI start up..! I was waiting to see the salary at the end.)
The paper imagines LLMs could be used to: speed up the delivery of public services (no more waiting for HMRC to pay back the amount I’ve been emergency taxed would be nice); reduce staff workload; perform complicated tasks; improve accessibility of government information; and perform specialist tasks more cost-effectively.
Seems like the Labour Party who told us they want to reform public services with AI might be interested in adopting some of these. And if used right, I’d imagine the Tories will, too. (And note - they already are, as Deputy Prime Minister and Chancellor of the Duchy of Lancaster said here).
Failure Modes
Alongside figuring out what AGI can do for the Government, the paper rightly identifies some limitations.
Model hallucination (which is potentially a problem if it is given access to sensitive UK government data); they can be biased; lack reasoning; not true domain experts; lacking emotion; lack of real-time and up-to-date data; short-term memory; lack of explainability - ‘black box.’
It is the first and last of these limitations I find the most worrying. Embedding current, state-of-the-art LLMs into government systems with unfettered (or even just significant) access to sensitive data (and the paper even gives this example! I am impressed!). The paper alludes to this, writing ‘that there are types of use cases where you should currently avoid using generative AI.’
This example is telling, too, that issues need not arise through malevolent use. Just a simple user query that a civil servant or UK citizen may use, such as: “How do I apply for compassionate leave?”
To which “the LLM [tailored] its responses to the user’s question using information in the retrieved text.” - returning confidential data to the user. Which the paper is coining a ‘Significant privacy violation and leakage of confidential data’ and categorising it as a MEDIUM on its Likelihood risk rating.
One could imagine how other benign questions could retrieve even more significant and confidential data, especially, say, if it is utilised in the Ministry of Defence. So the paper concludes this failure mode with the following promising sentence, that: ‘Private data which contains confidential information or employs different levels of security controls must not be used to fine-tune or train an LLM.’
A Warning
If the AGI the UK Government wants to use is smarter-than-human and ‘goes beyond human capabilities’ then we best hope it is fully aligned (which the AGI field is yet to accomplish), otherwise we will see the kinds of failure modes seen above frequently and probably to the further detriment of trust in our public services.
Will giving current state-of-the-art LLMs access to that data really improve our public services by a substantial amount? I’m not convinced.
My model of the civil service is not that they have an unproductive work force, it’s that the system of management is bad and maybe has a culture of slowing and obstruction (which might be a natural defence against opportunistic Ministers) - or what they would call, I’m sure, ‘following protocol.’
I am not against the UK Government using AI to better our public services and make Whitehall more productive, I just have the following issue: is having a slightly more productive civil service worth giving sensitive data access to black box AGIs?
Maybe a better plan would be to use specific, siloed, less powerful chatbot models with limited access to extremely sensitive data and use it as basically a more advanced Google search for civil servants and politicians alike to use.
(I appreciate the publication of this paper, though, and I hope more like this outlining exactly how they plan to utilise AI in government and the risks associated.)