Evaluating ChatGPT-4.0's accuracy and potential in idiopathic scoliosis conservative treatment: a preliminary study on clarity, validity, and expert perceptions

Francesco Negrini; Calogero Malfitano; Giorgio Ferriero; Giovanni Morone; Alberto Negrini; Fabio Zaina; Irene Ferrario; Carlotte Kiekens; Stefano Negrini; Jacopo Vitale

doi:10.1007/s00586-025-09166-4

Evaluating ChatGPT-4.0's accuracy and potential in idiopathic scoliosis conservative treatment: a preliminary study on clarity, validity, and expert perceptions

Eur Spine J. 2026 Feb;35(2):422-430. doi: 10.1007/s00586-025-09166-4. Epub 2025 Jul 21.

Authors

Francesco Negrini^{1

2}, Calogero Malfitano^{3

4}, Giorgio Ferriero^{1

2}, Giovanni Morone^{5

6}, Alberto Negrini⁷, Fabio Zaina⁷, Irene Ferrario⁷, Carlotte Kiekens⁸, Stefano Negrini^{8

9}, Jacopo Vitale^{10

11}

Affiliations

¹ Department of Biotechnology and Life Sciences, University of Insubria, Varese, Italy.
² Institute of Tradate, Istituti Clinici Scientifici Maugeri IRCCS, Tradate, Italy.
³ Department of Biomedical Sciences for Health, University of Milan, Milan, Italy. calogero.malfitano@unimi.it.
⁴ Azienda di Servizi alla Persona Istituti Milanesi Martinitt e Stelline e Pio Albergo Trivulzio, Milan, Italy. calogero.malfitano@unimi.it.
⁵ Department of Life, Health and Environmental Sciences, University of L Aquila, L'Aquila, Italy.
⁶ IRCCS Fondazione Santa Lucia, Rome, Italy.
⁷ ISICO (Italian Scientific Spine Institute), Milan, Italy.
⁸ IRCCS Galeazzi- Sant'Ambrogio Hospital, Milan, Italy.
⁹ Department of Biomedical, Surgical and Dental Sciences, University of Milan, Milan, Italy.
¹⁰ Spine Center, Schulthess-Klinik, Zurich, Switzerland.
¹¹ Institute of Sports Sciences, University of Physical Culture in Cracow, Cracow, Poland.

PMID: 40689984
DOI: 10.1007/s00586-025-09166-4

Abstract

Purpose: This study aimed to evaluate the scientific accuracy, content validity, and clarity of ChatGPT-4.0's responses on conservative management of idiopathic scoliosis. The research explored whether the model could effectively support patient education in an area where non-surgical treatment information is crucial.

Methods: Fourteen frequently asked questions (FAQs) regarding conservative scoliosis treatment were identified using a systematic, multi-step approach that combined web-based inquiry and expert input. Each question was submitted individually to ChatGPT-4.0 on December 6, 2024, using a standardized patient prompt ("I'm a scoliosis patient. Limit your answer to 150 words"). The generated responses were evaluated by a panel of 37 experts from a specialized spinal deformity center via an online survey using a 6-point Likert scale. Content validity was assessed using the Content Validity Ratio (CVR) and Content Validity Index (CVI), and inter-rater reliability was calculated with Fleiss' kappa. Experts also provided categorical feedback on reasons for any rating discrepancies.

Results: Eleven out of 14 responses met the CVR threshold (≥ 0.38), yielding an overall CVI of 0.68. Three responses - addressing "What is scoliosis?", "Can exercises or physical therapy cure scoliosis?", "What is the best sport for scoliosis?"- showed lower validity (CVR scores: 0.37, 0.37, and - 0.58, respectively), primarily due to factual inaccuracies and insufficient detail. Clarity received the highest ratings (median = 6), while comprehensiveness, professionalism, and response length each had a median score of 5. Inter-rater reliability was slight (Fleiss' kappa = 0.10).

Conclusion: ChatGPT-4.0 generally provides clear and accessible information on conservative idiopathic scoliosis management, supporting its potential as a patient education tool. Nonetheless, variability in response accuracy and expert evaluation underscores the need for further refinement and expert supervision before wider clinical application.

Keywords: Artificial intelligence; Natural Language processing; Patient education as topic; Rehabilitation; Scoliosis.