Objective: In clinical spine surgery research, manually reviewing surgical forms to categorize patients by their surgical characteristics is a crucial yet time-consuming task. Natural language processing (NLP) is a machine learning tool used to adaptively parse and categorize important features from text. These systems function by training on a large, labeled data set in which feature importance is learned prior to encountering a previously unseen data set. The authors aimed to design an NLP classifier for surgical information that can review consent forms and automatically classify patients by the surgical procedure performed.
Methods: Thirteen thousand two hundred sixty-eight patients who underwent 15,227 surgeries from January 1, 2012, to December 31, 2022, at a single institution were initially considered for inclusion. From these surgeries, 12,239 consent forms were classified based on the Current Procedural Terminology (CPT) code, categorizing them into 7 of the most frequently performed spine surgeries at this institution. This labeled data set was split 80%/20% into train and test subsets, respectively. The NLP classifier was then trained and the results demonstrated its performance on the test data set using CPT codes to determine accuracy.
Results: This NLP surgical classifier had an overall weighted accuracy rate of 91% for sorting consents into correct surgical categories. Anterior cervical discectomy and fusion had the highest positive predictive value (PPV; 96.8%), whereas lumbar microdiscectomy had the lowest PPV in the testing data (85.0%). Sensitivity was highest for lumbar laminectomy and fusion (96.7%) and lowest for the least common operation, cervical posterior foraminotomy (58.3%). Negative predictive value and specificity were > 95% for all surgical categories.
Conclusions: Utilizing NLP for text classification drastically improves the efficiency of classifying surgical procedures for research purposes. The ability to quickly classify surgical data can be significantly beneficial to institutions without a large database or substantial data review capabilities, as well as for trainees to track surgical experience, or practicing surgeons to evaluate and analyze their surgical volume. Additionally, the capability to quickly and accurately recognize the type of surgery will facilitate the extraction of new insights from the correlations between surgical interventions and patient outcomes. As the database of surgical information grows from this institution and others in spine surgery, the accuracy, usability, and applications of this model will continue to increase.
Keywords: classification; curriculum development; machine learning; natural language processing.