Assessing the performance of chat generative pretrained transformer (ChatGPT) in answering chronic kidney disease-related questions

Ther Apher Dial. 2025 Oct;29(5):760-765. doi: 10.1111/1744-9987.14239. Epub 2024 Dec 16.

Abstract

Background: Chatbots produced by artificial intelligence are frequently used in health information today. We aimed to investigate the reliability and reproducibility of the answers given by Chat Generative Pretrained Transformer (ChatGPT), one of the most used chatbots, to frequently asked questions related to chronic kidney failure.

Methods: We reviewed frequently asked questions related to chronic kidney disease (CKD) from social media platforms and Internet. The questions were asked to ChatGPT, and the answers were scored from 1 to 4 by two experienced nephrologists.

Results: Eighty-five frequently asked questions about chronic renal failure were examined and 60 of them were included in the study after exclusion criteria. Fifty-one (85%) of the questions received 1 point, 7 (11.7%) received 2 points and 2 (3.3%) received 3 points. The similarity rates of the answers to the repeated questions were between 80% and 100%.

Conclusion: ChatGPT has provided reliable responses with high reproducibility to inquiries related to CKD.

Keywords: ChatGPT; artificial intelligence; chronic kidney disease; frequently asked questions.

MeSH terms

  • Artificial Intelligence*
  • Female
  • Generative Artificial Intelligence
  • Humans
  • Internet
  • Male
  • Middle Aged
  • Renal Insufficiency, Chronic*
  • Reproducibility of Results
  • Social Media*
  • Surveys and Questionnaires