(C) PLOS One
This story was originally published by PLOS One and is unaltered.
 . . . . . . . . . .



Deep learning for blood glucose prediction: Reproducibility challenges and factors affecting differential performance [1]

['Baiying Lu', 'Department Of Computer Science', 'Dartmouth College', 'Hanover', 'New Hampshire', 'United States Of America', 'Biratal Wagle', 'Department Of Quantitative Biomedical Sciences', 'Zhaohui Liang', 'Emory University']

Date: 2026-09 

Blood glucose prediction is a critical component of next-generation diabetes technologies, such as artificial pancreas systems, where reliable performance is essential for safety and effectiveness. Although deep learning methods have achieved promising advances in this area, a critical gap remains in understanding the reproducibility and generalizability of these methods. To contextualize the gap, this study reviewed 67 recent papers that proposed a deep learning method for glucose prediction to identify key reproducibility challenges. Next, we adopted a standardized framework, encompassing technical, statistical, and conceptual reproducibility evaluations, to experimentally assess the reproducibility of eight representative deep learning methods. To achieve this, we reimplemented and evaluated these eight deep learning methods using over 1.36 million continuous glucose monitoring samples (5,061 days) from 128 individuals with type 1 diabetes across three public datasets: OhioT1DM, DiaTrend, and T1DEXI. We found that even though these models demonstrated good technical and statistical reproducibility, their conceptual reproducibility—the ability to generalize to datasets with different diabetes management patterns—was limited. Further analyses revealed that each model’s overall prediction performance was strongly influenced by individual glycemic control, with higher prediction errors observed among participants with lower time with blood glucose in the target range (70–180 mg/dL). This study identified key reproducibility challenges associated with current blood glucose prediction methods within type 1 diabetes populations, highlighting the need for increased transparency, dataset diversity, standardized evaluation practices, and code accessibility to ensure reproducible and reliable models for blood glucose prediction.

Managing type 1 diabetes requires individuals to frequently monitor their blood glucose levels and constantly consider how decisions related to food, activity, insulin use, and more affect their blood glucose levels. Accurate blood glucose prediction is critical to inform diabetes management strategies and mitigate adverse events. Consequently, many researchers have explored deep learning models for predicting future blood glucose levels, with the goal of providing earlier warnings of adverse events and personalized guidance for diabetes management. However, despite the progress, little effort exists on evaluating the reproducibility of published models for blood glucose prediction. To address this gap, we evaluated the reproducibility of eight state-of-the-art deep learning models using three publicly available datasets from distinct populations with type 1 diabetes. This evaluation allowed us to examine not only whether the reported results could be accurately replicated but also whether these models could generalize to new groups of individuals with different blood glucose management strategies and outcomes. Findings from this study revealed several barriers that limit model reproducibility. Furthermore, we found that the performance of many models was significantly affected by the heterogeneity of glucose management across individuals. Our work highlights the importance of transparent research practices and evaluation across diverse datasets to ensure that future glucose prediction models are reliable and equitable for all people living with diabetes.

Data Availability: The datasets used in this study are all publicly available datasets: OhioT1DM by Marling et al., 2020, DiaTrend by Prioleau et al., 2023, and T1DEXI by Riddell et al. 2023. The OhioT1DM dataset can be accessed at https://webpages.charlotte.edu/rbunescu/data/ohiot1dm/OhioT1DM-dataset.html . The DiaTrend dataset can be accessed at https://doi.org/10.7303/syn38187184 . The T1DEXI dataset can be accessed at https://doi.org/10.25934/PR00008428 . For the privacy of study participants, the OhioT1DM, DiaTrend, and T1DEXI datasets are protected via controlled-access managed by the dataset creators/owners. Each dataset can be accessed in accordance with the respective data use agreement.

Copyright: © 2026 Lu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

In this study, we adopted the framework proposed by McDermott et al. [ 13 ] to inform our approach for reproducibility evaluation. McDermott et al. argue that “in order for a study to be fully reproducible, it must meet three criteria: 1) technical reproducibility (can results be reproduced under technically identical conditions?); 2) statistical reproducibility (can results be reproduced under statistically identical conditions?), and 3) conceptual reproducibility or replicability (can results be reproduced under conceptually identical conditions?)” [ 13 ]. Guided by this framework, we selected eight representative deep learning studies that evaluated their methods on the publicly available OhioT1DM [ 12 ] to assess technical and statistical reproducibility using the same dataset and code (when available). Furthermore, we evaluated conceptual reproducibility using two additional publicly available datasets (DiaTrend [ 14 ] and T1DEXI [ 15 ]) that feature greater diversity in participants and diabetes management. Our approach sought to conduct a comprehensive assessment of reproducibility under consistent conditions as well as an examination of how each model’s performance responds when applied to datasets with different behavioral or management profiles. By situating reproducibility within a structured evaluation paradigm, this study aims to clarify the extent to which current blood glucose prediction methods generalize beyond their original development settings and to provide insights for developing more robust and clinically meaningful machine-learning approaches for diabetes care.

To examine potential reproducibility gaps and challenges, we analyzed 67 peer-reviewed papers published between 2018 and 2025 presenting deep learning methods for blood glucose prediction (see supplementary S1 File ). This effort revealed substantial limitations in methodological transparency across many published work. In particular, we found that only 23.9% of studies made their code publicly available and 37.3% developed and evaluated their models using private datasets - see Fig 1 . These findings suggest that there is limited access to implementation details and training data which is a major barrier for reproducibility studies. Beyond the issue of data and code availability, we also observed an over-reliance on a small number of public datasets. Eighty-five percent (85%) of papers evaluated on at least one public dataset, and the OhioT1DM dataset [ 12 ] - with only 12 participants - was the most commonly used public dataset, appearing in 55.2% of studies. This narrow evaluation strategy also has the potential to limit model generalizability and challenge the robustness of reported results. Furthermore, we observed significant variability of reported results across papers. While over 80% of studies reported root mean square error (RMSE) when predicting glucose values 30 minutes in advance, RMSE values varied widely, ranging from below 5 mg/dL to above 25 mg/dL; thereby making it difficult to assess the reliability of reported results. These observations indicate that while deep learning-based methods are promising for blood glucose prediction, reproducibility studies are critically needed in this domain.

Diabetes is a leading chronic condition with significant implications and costs in the U.S. and globally [ 1 – 3 ]. At the individual level, the complexity of having diabetes, especially type 1 diabetes, requires daily (and life-long) management of blood glucose (BG) levels to maintain a healthy standard of living and reduce the risk of micro- and macro-vascular complications [ 4 ]. Yet, the majority of people with diabetes (e.g., 79% of adults with diabetes and 83% of youths with type 1 diabetes in the U.S.) do not achieve clinical targets or risk factor control goals [ 3 , 5 ]. To facilitate the complex task of managing diabetes, technological innovations such as continuous glucose monitors (CGMs) and automated insulin delivery (AID) systems are being used to transform the standard of care [ 6 , 7 ]. However, several advanced technologies rely heavily on the ability to accurately predict blood glucose in advance to support important management decisions that seek to prevent adverse glycemic events (i.e., hypoglycemia and hyperglycemia). Over the past several years, deep learning has been increasingly applied to blood glucose prediction tasks, leading to a growing body of research that explores various model architectures and prediction strategies [ 8 ]. However, as diabetes technology continues to advance, a comprehensive evaluation of machine learning methods for blood glucose prediction is critical. Yet, limited research exists on assessing the reproducibility and generalizability of state-of-the-art (SOTA) methods, particularly deep learning [ 9 – 11 ] that are widely used for blood glucose prediction.

Materials and methods

This study systematically evaluates the reproducibility of eight state-of-the-art deep learning methods for blood glucose prediction across three publicly available datasets (OhioT1DM [12], DiaTrend [14], and T1DEXI [15]). Following the reproducibility framework proposed by McDermott et al. (2021) [13], we assessed technical, statistical, and conceptual reproducibility through standardized experimental replication. The following sections describe the datasets, selected models, and implementation procedures adopted for our reproducibility evaluation study.

Ethics statement This study involves analysis of only publicly available de-identified data. As a result, the Committee of Protection of Human Subjects at Dartmouth College determined that the proposed activities are not research involving human subjects, thus institutional review board (IRB) review was not required. The IRB waiver is documented in STUDY00033374.

[END]
---
[1] Url: https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001633

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/