Updating Medical AI: A Change Plan Must Define the Evidence Before Release

A hospital has evaluated an AI tool and trained staff to use it. Six months later, the supplier offers a new model version. Its average benchmark score is better. Does that mean the hospital can assume that every local workflow, patient group and scanner configuration is equally well served?

An update changes the evidence question. The task is to establish what changed, whether the change is within the authorized scope, and whether the relevant verification and validation requirements have been met. A general statement that a model has improved leaves those questions open.

The FDA's final guidance on predetermined change control plans, dated August 2025, addresses planned modifications to AI-enabled device software. This article explains the distinction and proposes an original handoff worksheet. It does not evaluate an individual device or provide instructions for clinical care.

What a PCCP covers

The FDA recommends that a predetermined change control plan, or PCCP, describe planned modifications, the methods used to develop, validate and implement them, and an assessment of their impact. The agency reviews the plan within a marketing submission. Modifications described in an authorized PCCP can be implemented without a new submission for each such modification, subject to the applicable conditions.

This is bounded change management. It is not blanket permission for every future model or intended-use change. A supplier still needs to establish how a particular update relates to its authorized plan and other applicable requirements.

Separately, the FDA's broader AI-enabled device lifecycle guidance page remains labeled draft guidance in the source reviewed. Do not merge a final PCCP document and a separate draft into one binding rule.

Identify the actual change

“New model” is not a sufficient change description. Record whether the update changes the training data, model parameters, input preprocessing, supported hardware, output thresholds or user interface. Several changes may occur together, making the old evidence harder to interpret.

Ask what remains fixed. If the intended population, task and operating environment are unchanged, evidence may focus on the proposed modification. If those boundaries move, an earlier evaluation may no longer answer the new question. The appropriate regulatory assessment belongs with qualified device and regulatory teams.

Keep the device version, evaluation dataset version and local integration version separate. Otherwise a failed test may be impossible to trace to the component that changed.

Define acceptance before seeing the new result

Choosing the threshold after inspecting a favorable benchmark makes a release decision difficult to defend. Write the primary performance measures, subgroup analyses and failure conditions before comparing versions.

For a classification task, overall accuracy can conceal clinically relevant tradeoffs. A model might reduce one kind of error while increasing another. The evaluation should reflect intended use and the consequences of those errors, rather than an attractive aggregate chosen for marketing.

Also specify workflow requirements: output availability, clear failure messages, compatibility and human review behavior. A statistically stronger model that sends its result to the wrong screen is not a successful system update.

Work a deliberately nonclinical example

Imagine a mock image-tagging benchmark used only to teach release evaluation. There are two equally sized groups of 100 labeled test images. The old model correctly tags 90 images in group A and 80 in group B. The new model tags 98 in A and 76 in B.

The aggregate improves while group B declines. This does not by itself prove unfairness, establish statistical significance or identify a clinical consequence. It shows why a release discussion needs the distribution of outcomes, the paired cases and uncertainty around estimates.

If the predeclared rule required no unacceptable decline in either group, the new aggregate would not settle the decision. More evidence or a different modification may be needed. Do not convert these toy percentages into acceptable clinical thresholds.

Keep local validation separate from the supplier's evidence

A hospital needs to know whether its inputs and workflow fit the device's supported conditions. Differences in acquisition hardware, protocols, patient mix or data routing can matter. Local checks complement the supplier's evidence; they do not create a new authorized indication.

Use representative local test inputs under the institution's privacy and governance process. Include missing fields, unsupported input types and integration failures. Confirm how users recognize a failed or stale result, and whether the interface makes model version changes visible where needed.

Training should address the changed behavior. If a threshold or output format changes, a short release note may be insufficient for people whose decisions depend on interpreting it correctly.

Plan monitoring and rollback together

Before release, name who receives problems, which signals trigger investigation and who can stop use. Monitor input failures and operational incidents as well as outcome measures that may arrive later.

Rollback requires a usable prior configuration, compatible data and a documented restoration procedure. If an update changes stored formats, the previous application may no longer read new records. Testing restoration is more informative than writing “rollback available” in a checklist.

A staged rollout can limit exposure while evidence accumulates, but it needs appropriate oversight and must fit the authorized use and institution's procedures. The phrase “pilot” does not exempt a medical deployment from its obligations.

Ask for an evidence handoff

A useful release packet contains the change description, plan mapping, test protocol, results with denominators, unresolved limitations, installation checks and recovery instructions. The receiving team should be able to tell which claims are observed, which are assumptions and which remain untested.

As a self-check, suppose a supplier says an update is covered by a PCCP but cannot identify the relevant modification or validation criteria. The missing item is traceable evidence linking this release to the plan. A better overall score cannot substitute for that connection.

For background, read the site's medical device cybersecurity handoff. Version control matters both for functional changes and for security maintenance.

Sources and scope

• FDA final PCCP guidance, August 2025.
• FDA draft AI lifecycle guidance, January 2025.

Status checked October 22, 2026. The worksheet and mock benchmark are original teaching aids. They are not device-specific regulatory or clinical recommendations.