AI model governance is the set of approvals, evidence and technical limits that decide whether an AI model may be used, changed or retired. For UK boards, that means named ownership, tested controls and evidence a regulator or auditor can inspect — not a policy statement.
AI model governance controls are how that discipline becomes real. They turn ISO/IEC 42001, NIST AI RMF, UK GDPR and sector expectations into board-visible proof: an inventory, an approval record, test evidence, access boundaries, human review, drift monitoring and a withdrawal route — rather than policy intent.
The board question is not "which model are we using?" It is "what would stop this model doing the wrong thing, who owns that stop, and what evidence would we inspect if the answer harmed someone?" ISO describes ISO/IEC 42001 as a management-system standard for establishing, implementing, maintaining and continually improving an AI management system, and NIST's AI RMF core is organised around govern, map, measure and manage functions, not a one-off checklist.
Key takeaways
- Model approval should depend on named ownership, intended use, risk limits, test evidence, monitoring and a withdrawal path.
- The board should ask for evidence artefacts — an inventory entry, an approval record, a model card, test results — not a policy assertion that governance exists.
- A model inventory is the starting point, but change control, versioning and drift monitoring decide whether the inventory stays true.
- Most real deployments run on a third-party model, so vendor oversight — version-change notices, SLAs and sub-processor disclosure — is part of the control set, not outside it.
- UK GDPR, ICO guidance and the Data (Use and Access) Act 2025 matter whenever the model processes personal data or supports automated decision-making.
- Firms with an existing model risk management framework — banks under PRA SS1/23, and any FCA-regulated firm relying on accountability and Consumer Duty — should extend it to AI models rather than start a separate AI rulebook.
Where AI model governance controls belong
Model governance sits between corporate policy and engineering reality. A policy says which uses are allowed. The controls prove that a specific model, prompt, retrieval source, tool or vendor integration stays within those uses.
The control set should start before deployment. A board paper that approves an AI use case should name the model, its purpose, the data it uses, the decision it influences, the human or technical control that can overrule it, and the evidence required before release. The NIST AI RMF Core supports that shape because its governance function is cross-cutting, while map, measure and manage continue through the system lifecycle.
The same discipline is visible in engineered controls already in use. A model that reads operational data should be read-only by construction. A model that proposes a financial posting should sit behind confidence floors and reason codes. A model that drafts accountable work should create an append-only decision ledger. Those are not separate blog examples; they are the same governance principle expressed as different controls.
AI model governance and model risk management
Many boards already run a model risk management framework for pricing, credit, actuarial or capital models. AI model governance is best treated as an extension of that discipline, not a parallel one. The core habits are the same: a model inventory, an owner, independent validation, monitoring and a change process. AI adds new failure modes — prompt and retrieval changes, opaque vendor updates, drift in live behaviour — but it does not need a separate governance vocabulary.
For banks, this is now explicit. The PRA's model risk management principles for banks (SS1/23) set out five principles for identifying, managing and controlling model risk, and took effect on 17 May 2024. The principles apply to all models used to inform business decisions, whether developed in-house or bought from a vendor, and they treat model risk as a risk in its own right. A firm in scope should be able to show where its AI models sit in that framework, not maintain a second, weaker regime beside it.
For other FCA-regulated firms, the equivalent is the existing accountability regime: senior-manager responsibility, Consumer Duty and operational resilience. The board test is whether an AI model has been brought inside those duties, with a named senior manager who owns the outcome and can show the evidence.
The artefact that carries this at model level is the model card, or model documentation. A good model card records intended use, training-data provenance, known limitations, evaluation results and the version it describes. It is the single document a validator, auditor or regulator can read to understand what a model is for and where its edges are.
What the board needs to decide
Boards do not need to approve every prompt edit. They do need to set the decision frame within which management can approve, monitor and change models.
The board decision should cover five points:
- Risk class: which model uses need board approval, executive approval or local product approval.
- Permitted use: whether the model advises, drafts, scores, routes, recommends or acts.
- Decision boundary: whether a person, deterministic code or another control can stop the model output before it affects a record, customer, tenant, employee or regulated report.
- Evidence threshold: what test results, DPIA, security review, user testing, bias review or monitoring data must exist before go-live.
- Withdrawal route: who can pause the model, roll back a prompt or vendor version, notify affected users and report the incident.
The UK's regulatory position makes this board frame necessary. The government response to the AI regulation white paper confirmed five cross-sector principles for existing regulators: safety, security and robustness; appropriate transparency and explainability; fairness; accountability and governance; and contestability and redress. The DSIT government response also describes a context-based approach, so boards should expect their existing regulator to apply these principles through the duties already attached to their sector.
Controls and evidence the board should ask for
The board should see a short evidence pack for each material model, refreshed when the model, use case, data source or vendor changes. The pack does not need to be long. It needs to be inspectable.
| Control | Evidence the board or risk committee can inspect | Owner |
|---|---|---|
| Model inventory | Register entry showing model, provider, version, use case, data categories, system owner and business owner | Chief technology officer or head of data |
| Approval record | Decision paper recording permitted use, risk class, human or code gate, and approval date | Accountable executive |
| Model documentation | Model card recording intended use, training-data provenance, known limitations, evaluation results and the model version it describes | Product owner or model owner |
| Data protection assessment | DPIA or screening note where personal data is processed, including fairness, transparency and individual-rights impacts | Data protection officer |
| Test evidence | Evaluation set, failure cases, red-team notes, bias checks, acceptance criteria and sign-off | Product owner with risk review |
| Access boundary | Least-privilege role, read-only transaction, tool allow-list or segregation of write credentials | Engineering lead |
| Human review | Named reviewer role, override authority, audit trail and sample of accept, modify or reject decisions | Operations lead |
| Model versioning | Pinned model and prompt versions, a version log and a tested rollback route for the last known-good version | Engineering lead |
| Monitoring | Drift monitoring plus quality, complaint, incident and override metrics, with thresholds for escalation | Risk or model owner |
| Change control | Record of prompt, model, retrieval, rules and threshold changes, with rollback route | Change advisory owner |
| Retirement | Decommission record, data-retention decision and user communication where required | System owner |
The ICO's AI and data protection guidance says that AI guidance is under review following the Data (Use and Access) Act 2025, and the ICO's DUAA page says the Act amends UK GDPR, the Data Protection Act 2018 and PECR while phasing changes between June 2025 and June 2026. For automated decision-making, the ICO's DUAA summary for organisations says organisations can rely on a wider range of lawful bases for significant automated decisions if appropriate safeguards continue to apply and special-category data is treated differently.
That is why the evidence pack should distinguish between a model that assists a person and a system that makes or materially shapes a significant decision about an individual. The ICO's draft ADM consultation closed on 29 May 2026, and the ICO's technology guidance plan listed final ADM and profiling guidance for winter 2026 as of this update. Do not build a control regime around final wording that the regulator has not yet published.
Third-party and vendor models
Most organisations do not train their own model. They build on a vendor's — OpenAI, Anthropic, Google, Microsoft or a model accessed through a platform. That changes where the risk sits, but not who is accountable for it. The board still owns the outcome; the controls simply have to reach across the contract.
Three vendor-specific controls belong in the evidence pack, and none of the generic "what is AI model governance" content aimed at enterprises tends to cover them:
- Model-version change notices. A vendor can retire, upgrade or silently re-tune a model behind the same API name. Pin to a specific model version where the vendor allows it, subscribe to deprecation and version notices, and treat any version change as a change-control event that triggers re-testing, not a routine update. Behaviour can shift without a single line of your own code changing.
- SLAs and operational terms. The contract should say what availability, latency, support and incident-notification you are entitled to, and what happens on suspension or exit. A model inside a regulated process needs the same resilience thinking as any other critical supplier.
- Sub-processor and data disclosure. You should know where prompts and outputs are processed, which sub-processors the vendor uses, whether your data trains their models, and what the retention terms are. For personal data this feeds directly into the DPIA and the lawful-basis record.
This is the same discipline the PRA already expects of banks under SS1/23, whose principles apply to vendor models as well as in-house ones. For any board, the practical rule is simple: a vendor model is still your model risk. If the supplier cannot answer a version, security or data question, that gap is a risk decision the board should take knowingly, not a detail to leave with procurement.
Framework mapping for UK boards
Framework mapping is useful only if each line points to evidence. A crosswalk that says "aligned" without naming the artefact is decoration.
| Framework or regulator | What it expects in practice | Model-governance evidence |
|---|---|---|
| ISO/IEC 42001 | ISO says the standard specifies requirements for an AI management system and supports responsible development and use of AI systems | AI policy, roles, risk assessment, lifecycle controls, monitoring and improvement records |
| NIST AI RMF | NIST sets out govern, map, measure and manage functions, with actions and outcomes rather than an ordered checklist | Inventory, use-case context, evaluation plan, monitoring dashboard and risk-treatment record |
| UK GDPR and ICO | The ICO applies data-protection principles to AI and is updating guidance after the DUAA 2025 | DPIA, lawful-basis record, transparency wording, human-review route and complaint handling |
| UK pro-innovation principles | DSIT's framework asks regulators to apply safety, transparency, fairness, accountability and contestability within their remits | Board risk appetite, explainability evidence, fairness testing, owner register and redress process |
| FCA-regulated firms | The FCA says its approach is principles-based and focused on outcomes, with no plan for extra AI regulations, relying instead on existing frameworks | Senior manager accountability, Consumer Duty impact, operational resilience, model-risk oversight and customer-support evidence |
| PRA SS1/23 (banks) | The PRA treats model risk as a risk in its own right and sets five principles for banks' model risk management, covering in-house and vendor models | Model inventory, tiering, independent validation, monitoring and a self-assessment against the principles |
| NCSC secure AI guidance | NCSC guidance covers secure design, development, deployment, and operation and maintenance, including logging and monitoring after deployment | Threat model, supply-chain check, deployment approval, monitoring logs, incident playbook and update process |
For financial services, this is not theoretical. The FCA's AI approach, first published on 8 September 2025 and last updated on 13 February 2026, says the regulator wants safe and responsible adoption in UK financial markets, relies on existing frameworks rather than extra AI regulations, and points to Consumer Duty plus accountability and governance as relevant. The financial-services question is therefore not whether the model has a novel AI compliance label. It is whether existing senior-manager, customer-outcome and operational-resilience evidence still works once a model is inside the process. Our financial-services playbook takes that mapping further.
Common mistakes that weaken the control set
The first mistake is treating the model inventory as the control. An inventory tells you what exists. It does not prove the model was tested, that a person can overrule it, or that the organisation knows when performance has shifted.
The second mistake is approving a model once and ignoring the changes around it. A retrieval source changes. A vendor releases a new model version. A prompt gains a tool call. A confidence threshold moves. Any of those can change the risk profile without a new product launch. This is why model governance belongs in change control and versioning, not only in procurement.
The third mistake is confusing human presence with human review. A reviewer with no authority, no time, no training or no recorded decision is not a safeguard. The evidence should show when people accepted, modified or rejected model output, and what happened when they disagreed. That record belongs in the same family as a board AI risk register built from living evidence, not a static spreadsheet updated before a meeting.
The fourth mistake is leaving security outside the model-governance discussion. The NCSC secure AI development guidance is explicit that AI systems need secure design, secure development, secure deployment and secure operation and maintenance, including logging and monitoring once deployed. A model that is well governed on paper but exposed through weak credentials, poor logging or unreviewed tool access is still a board risk.
Frequently asked questions
What is AI model governance?
AI model governance is the set of approvals, evidence and technical limits that decide whether an AI model may be used, changed or retired. In practice it means a named owner, a defined permitted use, tested controls, live monitoring and a withdrawal route — with evidence a regulator or auditor could inspect, rather than a policy statement that governance exists.
What's the difference between AI governance and AI model governance?
AI governance is the whole-organisation frame: the board's risk appetite, policies, roles and oversight across every use of AI. AI model governance is the model-level layer inside that frame — the specific controls and evidence for a given model or system, from approval through monitoring to retirement. You need both: the appetite tells teams what is allowed; the model controls prove a particular model stays inside it.
Who should own AI model governance in a UK organisation?
The board sets the frame and holds ultimate accountability, but day-to-day ownership is shared: a named accountable executive for each material model, a model or product owner for its documentation and testing, the data protection officer where personal data is involved, and an engineering lead for the access and version controls. In FCA-regulated firms this should map to a senior manager under the accountability regime, not sit unassigned across a project team.
Does ISO/IEC 42001 cover model governance, or do we need something else?
ISO/IEC 42001 sets the management-system frame — policy, roles, risk assessment, lifecycle controls and continual improvement — but it does not, on its own, specify the model-level controls and evidence a board should inspect. Pair it with the NIST AI RMF for the govern-map-measure-manage detail, UK GDPR and ICO guidance for personal data, and, for banks, the PRA's SS1/23 model risk management principles.
How is AI model governance different from MLOps?
MLOps is the engineering practice of deploying, versioning and monitoring models. AI model governance is the accountability layer above it: who approved the model, for what use, with what evidence, and who can stop it. They overlap on versioning and monitoring — good MLOps produces much of the evidence governance needs — but MLOps alone does not decide permitted use, set risk appetite or make the outcome inspectable to a board or regulator.
What happens to model governance when we swap or upgrade a vendor's model?
Treat it as a change-control event, not a routine update. A new vendor model version can change behaviour without any change to your own code, so re-run the relevant tests, refresh the model card, re-confirm the approval, and check the vendor's version-change notice, SLA and any sub-processor or data-handling changes. Pin to a specific version where the vendor allows it, so upgrades happen when you choose, not silently.
Next step: test the evidence before deployment
Before approving a material model, ask management for one page: the model, the decision boundary, the evidence table, the framework mapping and the person who can stop it. If the answer needs a long reassurance paragraph, the control probably is not yet visible enough.
A practical route is to start with the Board AI Scorecard, then use the AI governance diagnostic to test whether policy, controls and evidence line up. Where the controls above have to be designed and evidenced rather than merely reviewed, that is the work our AI governance consultancy does. If you want to see how these controls appear in delivered systems, read the Trust page, the case studies, or the services page for how we build governance into production workflows.
Last reviewed: 6 July 2026.
Sources: ISO/IEC 42001:2023 · NIST AI RMF Core · ICO guidance on AI and data protection · ICO DUAA summary for organisations · ICO draft ADM consultation · ICO technology guidance plan · DSIT AI regulation government response · FCA AI approach · PRA SS1/23 model risk management principles for banks · NCSC secure AI development guidance
