A change in leadership within Government provides an opportunity to update UK policy to keep pace with ongoing rapid and transformative technological change.
The UK faces extreme threats from rapid advances in frontier AI models and biotechnology. Biological risks top the National Risk Register and are worsening: AI tools are lowering the barrier to designing dangerous pathogens, while advances in biology are opening the door to novel dangers such as “mirror” organisms with potentially catastrophic results. Minister for AI, The Rt Hon Kanishka Narayan MP, has stated that ‘controlling our own AI is crucial for Britain’s national security’; AI systems can now autonomously compromise all major operating systems and web browsers. Loss of control of AI systems has moved from a hypothetical risk to a regular occurrence, with a 5x increase in incidents in the past year, and could pose an existential threat within a decade.
CLTR has long argued for bold action to protect Britain from these threats. The UK is taking action on these threats, with structural changes that put AI policy at the heart of government. Elevating the AI Minister to the Cabinet for the first time, establishing the new AI Taskforce, and moving the AI Security Institute (AISI) into the centre of government are important signals that AI is now a first-order national priority. But the UK Government, with support from likeminded partners must go further and faster to keep pace with the evolving threat landscape.
The UK is a leader on the international stage in these domains, with a renowned AI Security Institute, strong biosurveillance and life sciences sectors, and long-standing research excellence. Continued investment will enable these capabilities to become durable, leverageable national advantages, driving resilient growth by reindustrialising through AI, speeding up drug and vaccine discovery, and facilitating rapid adoption of sovereign capabilities.
But we cannot be complacent to the rapidly evolving scale, scope and impact of these frontier technological risks.
We are publishing several key recommendations that we have made to the UK Government to build the UK’s resilience to extreme AI and biological risks.
Priority Recommendations: Frontier AI
- Treat AI sovereignty ambitions as a national security imperative. Dependence on foreign AI is a strategic vulnerability. The Government should encourage sovereign and foreign compute buildout in Britain, procure our own secure AI models as fallback options, and form middle-power coalitions on joint AI infrastructure procurement, shared model evaluation and continuity of model and infrastructure access. This must include scaling secure, classified compute and sovereign AI agent infrastructure.
- Bolster the AI Security Institute. AISI’s early access and testing of Claude Mythos demonstrates the importance of independent testing capabilities for surfacing model behaviours that could create risks to our national security. This requires increasing and guaranteeing sustained long-term funding to recruit and retain scarce technical talent and keep sight of the most capable systems.
- Ensure the AI Taskforce treats AI security as a first-order priority. The Taskforce should cover AI security and resilience issues alongside AI opportunities, and be given the convening authority to match, to ensure a consistent approach. The AI Taskforce is well-positioned to drive a whole-of-government approach to AI security, ensuring that departments are joined up in their approaches and risks are not missed.
- Give AI security a dedicated route into the National Security Council (NSC), served by a new AI Security Coordination Unit. AI incidents will generate urgent, cross-departmental security decisions that must be resolved at the level of the National Security Council. Today these decisions compete for attention on the NSC’s crowded agenda, with no clear owner, dedicated forum, or standing capacity to prepare and escalate them at pace. AI security needs a dedicated AI sub-committee or a guaranteed standing remit, with a new AI Security Coordination Unit, modelled on the Biosecurity Coordination Unit, acting as its secretariat. The NSC should ensure loss of control risks are properly captured and assessed in the National Security Risk Assessment with clear ownership.
- Introduce emergency powers to ensure the UK Government can effectively respond to a major AI security incident. This could include the power to compel information from AI companies, direct their mitigation and response, or temporarily contain or restrict access, distribution or operation of services. This would ensure the UK Government has (rarely used) levers in a crisis, and could potentially be delivered through the Cyber Security and Resilience Bill (CSRB).
- Revitalise UK leadership on international AI safety. Secure new frontier company commitments at the 2027 AI summit in Geneva, prioritising third-party auditing of and transparency into companies’ internal deployment of AI, and assess options for institutionalising the summit series. This should include commissioning a proposal to host a second ‘Bletchley’-style summit before 2028, five years on from the inaugural AI safety summit, to progress international coordination on key AI risks such as loss of control.
- Give the Government real-time visibility of threats from uncontrolled AI and other risks. Establish an AI Threat Analysis Centre modelled on the Joint Terrorism Analysis Centre, fusing classified and open-source intelligence into a single picture by bringing together staff from AISI and the Intelligence Community. This should cover loss of control, cyber, biological and influence threats, amongst others. The government should incentivise and facilitate reporting of loss of control incidents, including by creating confidential channels with AI companies, and considering legal requirements to report sufficiently severe incidents to AISI and NCSC (e.g. via amendments to the CSRB).
- Make the UK the global leader in third-party AI auditing and assurance to hold companies to account on safety. The UK needs the capability to audit the safety claims and practices of AI developers. AI labs are losing control of their AI systems, escalating cyber and biological risks, and operating without the independent scrutiny standard in safety-critical industries. The UK can build on its trusted third-party assurance roadmap, including through trusted third-party tester schemes. Assurance is both a fast-growing industry the UK can lead and a way to build the trust needed to facilitate widespread AI adoption.
- Pursue international rules for third-party audit and oversight of automated AI R&D. Work to mandate independent third-party audits of automated AI R&D under international law, with governance frameworks guaranteeing human oversight and the ability to halt development if needed. This can be advanced in the next 12 months at the AI Summit in Geneva and through the UK’s forthcoming leadership of the G20 and G7.
- Require transparency into the values and loyalties embedded in frontier AI systems. The UK increasingly relies on a handful of mostly foreign AI systems, yet has little visibility of the values, instructions and loyalties built into them. This presents a growing risk to democratic accountability as these systems begin to operate in sensitive settings, and shape the information environment for millions of citizens. The Government should require developers to disclose the values and behavioural instructions embedded in frontier models, such as model specifications and system prompts, and mandate independent auditing by AISI for hidden model loyalties before such systems are used in the public sector or critical national infrastructure.
Priority Recommendations: Biosecurity
CLTR has worked closely with the Government during the planning, publication and implementation of the Biological Security Strategy (2023). We have seen myriad benefits of a well-coordinated, mission-driven strategy. But the risks are numerous; accelerating in likelihood and impact; and require a whole-of-government approach. Here, we state three of our most critical immediate recommendations for the Government:
- Make the UK’s voluntary synthetic nucleic acid synthesis screening guidance mandatory, aligning with growing international norms. Synthetic nucleic acids can be designed by AI tools, ordered from UK biotech companies, and used to build modified or entirely new pathogens. Screening customers and orders of synthetic nucleic acids is a critical chokepoint at this digital-physical intersection. CLTR has worked closely with HMG to advocate for publishing the current voluntary guidance; this should now shift to a mandatory screening regime, aligning with international norms.
- Enable relevant HMG staff to have earliest possible access to frontier AI, so they can stay ahead of our potential adversaries. Early access to frontier AI capabilities will meaningfully enhance defensive biosecurity activities, from accelerating vaccine design to improving threat detection. HMG should aggregate national security demands and negotiate privileged access for UK defenders.
- The UK should prohibit, and commit not to fund, research that leads directly to the creation of self-replicating “mirror” organisms. The rapidly advancing frontiers of AI and biology are leading to new and novel biological risks with the potential to pose existential-level threats to humanity, and potentially all life on Earth. CLTR has worked with the GCSA and GO-Science to provide practical policy recommendations, informed by our cutting-edge computational experiments.
