ChatGPT as a Source of Patient Information for Lumbar Spinal Fusion and Laminectomy: A Comparative Analysis Against Google Web Search

Patrick P Nian; Jayson Saleet; Matthew Magruder; Ian J Wellington; Jack Choueka; John K Houten; Ahmed Saleh; Afshin E Razi; Mitchell K Ng

doi:10.1097/BSD.0000000000001582

ChatGPT as a Source of Patient Information for Lumbar Spinal Fusion and Laminectomy: A Comparative Analysis Against Google Web Search

Clin Spine Surg. 2024 Feb 20. doi: 10.1097/BSD.0000000000001582. Online ahead of print.

Authors

Patrick P Nian¹, Jayson Saleet², Matthew Magruder³, Ian J Wellington⁴, Jack Choueka³, John K Houten⁵, Ahmed Saleh³, Afshin E Razi³, Mitchell K Ng³

Affiliations

¹ Departments of Orthopaedic Surgery, SUNY Downstate Health Sciences University, College of Medicine, Brooklyn, NY.
² Boston University School of Medicine, Boston, MA.
³ Maimonides Medical Center, Brooklyn, NY.
⁴ University of Connecticut, Farmington, CT.
⁵ Department of Neurosurgery, Icahn School of Medicine at Mount Sinai, New York, NY.

PMID: 38409676
DOI: 10.1097/BSD.0000000000001582

Abstract

Study design: Retrospective Observational Study.

Objective: The objective of this study was to assess the utility of ChatGPT, an artificial intelligence chatbot, in providing patient information for lumbar spinal fusion and lumbar laminectomy in comparison with the Google search engine.

Summary of background data: ChatGPT, an artificial intelligence chatbot with seemingly unlimited functionality, may present an alternative to a Google web search for patients seeking information about medical questions. With widespread misinformation and suboptimal quality of online health information, it is imperative to assess ChatGPT as a resource for this purpose.

Methods: The first 10 frequently asked questions (FAQs) related to the search terms "lumbar spinal fusion" and "lumbar laminectomy" were extracted from Google and ChatGPT. Responses to shared questions were compared regarding length and readability, using the Flesch Reading Ease score and Flesch-Kincaid Grade Level. Numerical FAQs from Google were replicated in ChatGPT.

Results: Two of 10 (20%) questions for both lumbar spinal fusion and lumbar laminectomy were asked similarly between ChatGPT and Google. Compared with Google, ChatGPT's responses were lengthier (340.0 vs. 159.3 words) and of lower readability (Flesch Reading Ease score: 34.0 vs. 58.2; Flesch-Kincaid grade level: 11.6 vs. 8.8). Subjectively, we evaluated these responses to be accurate and adequately nonspecific. Each response concluded with a recommendation to discuss further with a health care provider. Over half of the numerical questions from Google produced a varying or nonnumerical response in ChatGPT.

Conclusions: FAQs and responses regarding lumbar spinal fusion and lumbar laminectomy were highly variable between Google and ChatGPT. While ChatGPT may be able to produce relatively accurate responses in select questions, its role remains as a supplement or starting point to a consultation with a physician, not as a replacement, and should be taken with caution until its functionality can be validated.