The Hard Part of Medical AI Starts After the Accuracy Test

A medical AI system can pass an evaluation and still become harder to trust once it meets real clinical workflows. Patient populations differ, software changes, users adapt and the surrounding service evolves. The UK’s new AI Airlock phase puts that post-launch problem in the foreground.

On 6 October 2026, the UK government said it had accepted all 44 recommendations from the National Commission into the Regulation of AI in Healthcare. It also announced that Phase 3 of the Medicines and Healthcare products Regulatory Agency’s AI Airlock would focus on post-market surveillance and lifecycle management. The announcement includes future work: draft guidance was expected by December and a broader roadmap in spring 2027. Those are commitments, not completed regulatory changes. ^[1]

That distinction matters to hospitals and vendors. A policy direction can help teams anticipate what evidence will matter, but it does not tell them that a particular device has been approved or that a new legal duty is already in force. The Commission describes recommendations for a future framework; teams still need to follow the rules that currently apply to their product and market. ^[2]

A launch score cannot describe the whole lifecycle

Pre-deployment validation asks whether a system performs acceptably on a defined dataset, population and workflow. After launch, the questions multiply: does performance hold across sites, has the intended use drifted, did an update change outputs, are clinicians relying on it differently, and can a serious problem be traced to a particular version?

The Commission’s recommendation 31 calls for lifecycle management, including monitoring and reporting after deployment. That is a proposal about the future regulatory framework, not a statement that all medical AI must already follow one newly issued monitoring standard. Its practical message is nevertheless familiar to software safety: a product needs a controlled history, a route for incidents and a way to assess change over time. ^[2]

The Airlock exists to examine where AI medical-device rules are difficult to apply. Phase 2’s official programme report describes seven participating technologies and 51 applications, and it records challenges around translating controlled evidence to real-world performance. The report also discusses how human oversight may change as users become more familiar with a system. These observations come from a sandbox programme; they are useful questions for teams, not population-wide findings about every product. ^[3]

The change-control problem is clinical as well as technical

For conventional software, a release may be judged mainly by whether the new version passes its tests. A clinical model update can also alter which cases receive attention, how much explanation a clinician sees, or when a person chooses to override a recommendation. Even if average accuracy improves, a shift in false negatives for one subgroup or a change in workflow can alter risk.

This is why version identifiers need to connect more than model weights. A useful record ties together the model and preprocessing pipeline, intended use, evaluation evidence, deployment site, user-facing interface, relevant configuration and the date a change reached users. When a concern appears, the team should be able to identify which version was active and what monitoring could have detected the change.

Monitoring also needs a defined response. A dashboard without thresholds or an owner can produce more data without producing safer decisions. Teams can specify which signal triggers review, who assesses the evidence, how clinicians are notified, when use is limited, and what conditions must be met before an update or rollback. Some signals will be technical, such as missing inputs or a failure rate; others may require structured reports from clinical users.

Build an evidence loop before deployment

A practical lifecycle plan can start with five questions:

1. What is the intended use? State the population, clinical decision, user and workflow. Record important exclusions and what the system must not decide.
2. What can change? Track model, data-processing, interface and workflow changes, including vendor updates that may arrive outside a hospital’s usual release cycle.
3. What will be observed? Choose performance, reliability and usage signals that can be collected lawfully and interpreted in context. Plan for missing or delayed outcome labels.
4. Who acts on a signal? Assign an owner and escalation path, including how to pause use and communicate with clinical teams.
5. What proves a change is safe enough? Define evaluation, review and rollback criteria before an update is scheduled.

These are engineering prompts, not a substitute for regulatory or clinical advice. They help make later evaluation possible by preserving the link between a product version, the people using it and the environment where it is used.

For teams building agentic or adaptive features, incident handling is part of the same discipline. Our guide to NIST’s new incident practices describes how to preserve evidence and verify recovery after an agent crosses a control boundary. Medical-device teams need their own sector-specific obligations, but the underlying lesson about traceable changes is shared.

What the Airlock does—and does not—tell buyers

A sandbox can expose regulatory friction and help a regulator develop guidance. It does not certify every participant, guarantee that a product is safe in every setting, or turn a pilot into general approval. The government’s announcement also mentions upcoming events and future milestones; availability and dates should be checked directly as the programme progresses. ^[1]

For a hospital evaluating a tool today, the immediate work remains concrete: establish intended use, ask which model version is deployed, understand how updates are controlled, and agree who receives incident reports. For a supplier, the same questions become part of the product’s lifecycle evidence and customer support. A buyer should ask for the actual documentation and market authorization relevant to the product rather than infer approval from participation in a sandbox.

The policy signal is timely because medical AI is not static software. Models and surrounding workflows can change, while consequences remain attached to patients and clinical decisions. The UK has accepted recommendations and started another phase of regulatory learning; the future framework is still being developed. Teams that build a credible evidence trail now will be better positioned to adapt when the details arrive.