Executive summary
This report explores how aspects of best practice risk governance – particularly the Three Lines Model (3LoD), which separates risk ownership, oversight and audit – could be effectively implemented at frontier AI companies to ensure safer model development and deployment. This aims to support the UK government’s approach to potential future legislation mandating better practice risk management, as well as making the case to the companies themselves on the value of these approaches.
Overall, current risk management practices at AI companies appear to be relatively ad hoc and lack overarching strategy and structure. While there has been some recent progress – such as responsible scaling policies and more comprehensive safety evaluations – this isn’t supported by a holistic risk governance structure for how these initiatives interact and complement each other. Risk governance provides the overarching framework, context and rules of engagement for the various risk management practices and functions in an organisation. In short, it is the ‘glue’ that holds various risk management practices together. We argue that implementing best practice, like the Three Lines Model, would help AI companies better identify, assess and act on risks, thereby reducing the chance of harmful models being released.
The report covers:
- What risk governance is, why it is necessary to ensure cohesiveness, structure and challenge to risk management activities, and examples where it’s been effectively implemented in safety critical industries like nuclear, healthcare and aviation.
- Why frontier AI companies need better risk governance, and how they could practically do this by implementing the best practice Three Lines Model separating risk ownership, oversight and audit. We do this by conducting a gap analysis and mapping currently existing teams to this model.
- The importance of risk culture, and how better risk governance can operationalise the nascent safety cultures that already exist.
- Legislative options to mandate forms of best practice risk management in law, such as distinct risk and assurance functions and external audit.
- Concrete recommendations for governments and companies – copied below.
Recommendations
For Government
Require AI companies to establish and maintain an office of risk management and an internal audit function, submit an annual resilience statement demonstrating the efficacy of their risk management process, undergo an annual external audit, and establish a protected ‘speak up’ channel (whistleblowing equivalent) with appeal to external bodies where necessary.
Build consensus within business and civil society about the importance of more holistic risk management, including a specific focus on risk governance. This could include publishing papers or facilitating workshops.
For companies
Build consensus
- Encourage internal discussions about how best practice risk management (overarching risk management framework, 3LoD, appetite statements) might be useful, in particular, how clarifying risk ownership and reporting lines could reduce/help manage risks from AI.
- Champion and sponsor best practice and dynamic risk management in the organisation by board members and senior executives acknowledging that this will require buy-in across the organisation to be effective.
Implement better practice – an eight-point checklist:
- Encourage a stronger sense of risk ownership in research, product and engineering teams through workshops, training and engagement with specialist risk and internal audit functions.
- Experiment with an MVP version of 3LoD structure and related methodology, and explore ways of sharing learnings with other companies (e.g. via the Frontier Model Forum).
- Introduce an office of risk management with a central risk management team reporting to a Chief Risk Officer or equivalent to provide challenge, a degree of independent oversight, and risk reporting to the board.
- Introduce an independent internal audit team to provide assurance on the process.
- Formulate/agree on risk appetite statements based on thorough risk identification and assessment.
- Introduce measures to encourage and enhance healthy risk culture: leadership and ‘tone from the top’, regular pulse checks/surveys with results reviewed by the board, emphasis on ‘just culture’, a protected ‘speak up’ channel with independent review and board visibility, and regular cross-functional risk identification workshops at different levels.
- Seek external assurance on the risk management process and overall compliance via external audit.
- Produce an annual resilience statement demonstrating the efficacy of the risk management process.
If you’re interested in discussing this further, please reach out to Ben Robinson using ben@longtermresilience.org
