Performance evaluation of the open-source Yet Another Spindle Algorithm sleep staging algorithm against gold standard manual evaluation of polysomnographic records in adolescence

Davide Benedetti; Emma Frati; Orsolya Kiss; Dilara Yuksel; Ugo Faraguna; Brant P Hasler; Peter L Franzen; Duncan B Clark; Fiona C Baker; Massimiliano de Zambotti

doi:10.1016/j.sleh.2023.07.019

Performance evaluation of the open-source Yet Another Spindle Algorithm sleep staging algorithm against gold standard manual evaluation of polysomnographic records in adolescence

Sleep Health. 2023 Dec;9(6):910-924. doi: 10.1016/j.sleh.2023.07.019. Epub 2023 Sep 13.

Authors

Affiliations

¹ Center for Health Sciences, SRI International, Menlo Park, California, USA; Department of Translational Research and of New Surgical and Medical Technologies, University of Pisa, Pisa, Italy. Electronic address: d.benedetti5@studenti.unipi.it.
² Center for Health Sciences, SRI International, Menlo Park, California, USA; Columbia College, Columbia University, NYC, New York, USA.
³ Center for Health Sciences, SRI International, Menlo Park, California, USA.
⁴ Department of Translational Research and of New Surgical and Medical Technologies, University of Pisa, Pisa, Italy; Department of Developmental Neuroscience, IRCCS Fondazione Stella Maris, Pisa, Italy.
⁵ Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA.

PMID: 37709595
DOI: 10.1016/j.sleh.2023.07.019

Abstract

Goal and aims: To evaluate an automatic sleep scoring algorithm against manual polysomnography sleep scoring.

Focus method/technology: Yet Another Spindle Algorithm automatic sleep staging algorithm.

Reference method/technology: Manual sleep scoring.

Sample: 327 nights (151 healthy adolescents), from the NCANDA study.

Design: Participants underwent one-to-three overnight polysomnography recordings, one consisting of an event-related-potential paradigm.

Core analytics: Epoch by Epoch and discrepancy analyses (Bland Altman plots) were conducted on the overall sample.

Additional analytics and exploratory analyses: Epoch by Epoch and discrepancy analysis were repeated separately on standard polysomnography nights and event-related potential nights. Regression models were estimated on age, sex, scorer, and site of recording, separately on standard polysomnography nights and event-related potential nights.

Core outcomes: The Yet Another Spindle Algorithm sleep scoring algorithm's average sensitivity of 93.04% for Wake, 87.67% for N2, 84.46% for N3, 86.02% for rapid-eye-movement, and 40.39% for N1. Specificity was 96.75% for Wake, 97.31% for N1, 88.87% for N2, 97.99% for N3, and 97.70% for rapid-eye-movement. The Matthews Correlation Coefficient was highest in rapid-eye-movement sleep (0.85) while lowest in N1 (0.39). Cohen's Kappa mirrored Matthews Correlation Coefficient results. In Bland-Altman plots, the bias between Yet Another Spindle Algorithm and human scoring showed proportionality to the manual scoring measurement size.

Important additional outcomes: Yet Another Spindle Algorithm performance was reduced in event-related-potential/polysomnography nights for N3 and rapid-eye-movement. According to the Matthews Correlation Coefficient, the Yet Another Spindle Algorithm performance was affected by younger age, male sex, recording sites, and scorers.

Core conclusion: Results support the use of Yet Another Spindle Algorithm to score adolescents' polysomnography sleep records, possibly with classification outcomes supervised by an expert scorer.

Keywords: Automatic scoring; Performance evaluation; Polysomnography; Sleep; YASA.

MeSH terms

Adolescent
Algorithms
Humans
Male
Polysomnography / methods
Reproducibility of Results
Sleep Stages*
Sleep*