Development and performance of a generative pretrained transformer for diabetes care

Abstract

Aims: To design and evaluate the performance of a diabetes-related Generative Pretrained Transformer (GPT).

Methods: A prompt-engineered layer over GPT was developed in four stages: (1) literature review on GPT tools development; (2) selection and preprocessing of 65 information sources about diabetes care strategies, patient education, diabetes technologies, and cultural care, among others; (3) prototype development; and (4) final tool evaluation using 420 diabetes-related questions adapted from three validated instruments. Outcomes were accuracy, rationale, citations, disclaimers, and emoji exclusion. Statistical analyses included descriptive statistics, chi-square tests and bias assessment. Compliance with data protection regulations and ethical standards was ensured.

Results: Diabetes Help GPT showed high overall accuracy (91.7 %), with 100 % rationale inclusion, 93.3 % citations, 84.8 % disclaimers, and minimal emoji use (13.3 %). Accuracy was highest in general diabetes knowledge and nutrition questions; slightly lower in insulin-related items (82.3 %). Disclaimer and emoji usage varied significantly by question format (p = 0.026 and p < 0.001). No accuracy bias was detected.

Conclusions: Diabetes Help GPT delivers accurate, well-sourced responses, supporting healthcare professionals in diabetes care. Unlike existing GPT models in medicine, it was developed through a transparent, expert-led process with curated content and iterative validation. It should complement, and not replace, professionals' criteria.

Keywords: Artificial intelligence; Diabetes; Generative pretrained transformer; Health workers.