How data drift impacts the safety and interpretability of machine learning models predicting risk from blood glucose control.

Objective: Predictive algorithms trained from historical data and deployed in dynamic environments are at risk from data drift. Machine learning models using data collected by sensors making continuous measurements could be impacted by both changes in the device itself and their users, driving drift and impacting safety. To maintain predictive performance, algorithms must be continuously monitored and tuned to overcome fundamental changes to both input data (covariate shift) and the relationship with the output (concept drift). Here, we aim to understand how changes to user behaviour, physiology and sensors could impact the safety of models using automated sensor readings from continuous glucose monitors (CGM). Methods and analysis: In this paper, we investigate how data drift in a machine learning model trained to predict short-term risk from blood glucose control for individuals with type-1 diabetes. We simulate how changes in both user behaviour and accuracy of the sensor could lead to covariate shift and concept drift. For each scenario, we quantify the changes to input data (Jensen-Shannon divergence), the impact to model performance metrics and the explainability of the model (ie, shift in feature importance). Results: We demonstrate that using a combination of covariate shift detection, multiple performance metrics and feature importance offers a powerful methodology of identifying different types of drifts in sensor data. For blood glucose management, our scenarios focused on user behaviour (ie, changes to blood glucose dynamics and CGM use) and device/sensor noise and variability, finding more simplistic approaches to drift detection could incorrectly identify risk to model safety. Conclusion: Machine learning and AI can enhance clinical decision-making, but often lack the transparency required to ensure ongoing safety. Combining complementary monitoring techniques enables clearer identification of changes in data or model behaviour, helping determine when retraining or intervention is needed.