(1127) Improved AI Prediction of Preterm Birth by Expanded Training and Retraining

Objective: To assess improvements in Artificial Intelligence (AI) for predicting preterm birth (PTB) after the addition of 1,165,618 images to the training dataset and an 8-month retraining period.

Study Design:
Proprietary AI software (V1) was initially trained on 877,141 de-identified ultrasound images from 19,940 unique ultrasound exams from a cohort of women who delivered at the University of Kentucky from 2017 to 2021. The AI trained on 79% of exams, and predictive performance was validated on a separate 21% of unique exams (n=4306). Only de-identified ultrasound images were used to predict the days until delivery. Statistical analyses utilized R2 values and mean absolute error (MAE).

An additional 1,165,618 ultrasound images with corresponding patient data from 2021 to 2023 were added to the training set (V2). The AI underwent 8 months of retraining (V3). The original validation set remained blinded and was utilized to evaluate the AI’s performance using the original algorithm (V1) and dataset versus its performance in V2 (expanded images dataset) and V3 (expanded images dataset with retraining).

Results:
After introduction of additional data and retraining, the AI's predictive performance improved across all outcomes (Table 1). The correlation coefficient for spontaneous PTB increased from 0.48 (V1) to 0.64 (V2) and 0.72 (V3). Correlation coefficients for prediction of Term+ all PTB increased from 0.85 (V1) to 0.88 (V2) to 0.92 (V3).

MAE was reduced (Table 2) in all categories from V2 and V3. For Term + all PTB, the MAE reduced from 14.30 days (V2) to 12.90 days (V3). Notable reductions were observed in the MAE for spontaneous PTB (22.08 days (V2) to 19.99 days (V3)), and for iatrogenic PTB (2.44 days (V2) to 19.33 days (V3)).

Conclusion: AI’s ability to learn and improve its performance in PTB prediction is demonstrated. Expansion of the dataset and retraining sequentially improved its predictive ability and notably reduced the MAE.