Keeping Clinical AI Accurate Over Time: New Research Tackles Model Drift and Fairness

Clinical AI models often work well on the day they launch. Months later, they can quietly degrade as medical practice evolves, record systems change, and patient populations shift. This problem, known as model drift, threatens patient outcomes when it goes undetected. A new four-year, $1.6 million grant from the National Library of Medicine aims to give health systems practical ways to keep these tools accurate for everyone they serve [1].

Why clinical AI models lose accuracy

Predictive models in medicine face two related weaknesses. First, performance drifts over time as clinical workflows, information systems, and patient demographics change [1]. Second, models often generalize poorly across different clinical, geographic, or sociodemographic groups [1]. Left unchecked, both problems erode trust and can widen disparities in care.

Dataset shift — temporal changes in practice patterns, patient mix, and data infrastructure — is now well documented as a primary driver of deteriorating model accuracy [5]. While best practices exist for training and validating new models, there has been limited work on prospective validation and ongoing maintenance [5].

From VA kidney injury model to broader subgroup monitoring

Sharon Davis, PhD, Assistant Professor of Biomedical Informatics at Vanderbilt University Medical Center, previously developed data-driven methods that monitor deployed models, detect drift, and recommend updates [1]. That approach helped maintain a national Department of Veterans Affairs risk model for acute kidney injury [1].

Most maintenance methods, however, track performance across the patient population as a whole [1]. The new project asks a different question: how do gaps in accuracy between patient groups shift over time, and how can we catch and correct them? [1]

Four research aims backed by NIH funding

Using VA and Vanderbilt Health data, Davis and colleagues will study prediction models for surgical complications, hospital-acquired acute kidney injury, and postpartum hemorrhage [1]. The work has four specific aims:

• Track how performance gaps between patient groups change over time, in both traditional machine learning models and large language models used for prediction [1].
• Develop monitoring and updating methods that restore accuracy both overall and within subgroups [1].
• Use synthetic patient data to support monitoring for small groups, such as patients at rural hospitals [1].
• Account for the way an effective AI tool changes the very outcomes it predicts — a shift that can be mistaken for drift [1].

That last aim will be tested on a postpartum hemorrhage prediction tool already in use at Vanderbilt Health [1].

Fairness, governance, and the maintenance gap

Davis serves on VUMC’s Artificial Intelligence Governance Committee, providing expertise in algorithmic fairness and bias [2][3]. Her career has centered on methods that support sustainable and fair clinical AI deployments [2][3]. The new grant extends that work by addressing a critical gap: best practices for model maintenance policies, performance monitoring perspectives, and updating strategies remain underdeveloped [5].

Health systems deploying AI tools need more than initial validation. They need automated surveillance systems that promote long-term performance and utility across diverse populations [2][3]. This research aims to deliver those practical methods.

What this means for hospitals and clinicians

A model that performs well on deployment day won’t necessarily keep performing well, or perform equally well for every group of patients [1]. The goal is to give health systems practical ways to keep these tools accurate and dependable for everyone they serve [1]. As clinical AI adoption accelerates, the ability to monitor and update models — especially for underrepresented subgroups — becomes a patient safety imperative.

If your organization is evaluating clinical AI tools, ask vendors about their post-deployment monitoring strategy. Look for evidence of subgroup performance tracking and a clear update pathway. The research underway at Vanderbilt may soon provide open methods you can adopt directly.

Sources

1. Sharon Davis receives research grant to keep clinical AI models …
2. Open questions and research gaps for monitoring and … - Frontiers
3. Sharon E. Davis, PhD, MS, FAMIA - Vanderbilt University Medical Center
4. Sharon Davis, PhD, MS, FAMIA - Vanderbilt University Medical Center
5. Sharon DAVIS | Predoctoral Trainee | M.Stat. | Vanderbilt University …