Classifying performance impairment in response to sleep loss using pattern recognition algorithms on single session testing

Abstract

There is currently no "gold standard" marker of cognitive performance impairment resulting from sleep loss. We utilized pattern recognition algorithms to determine which features of data collected under controlled laboratory conditions could most reliably identify cognitive performance impairment in response to sleep loss using data from only one testing session, such as would occur in the "real world" or field conditions. A training set for testing the pattern recognition algorithms was developed using objective Psychomotor Vigilance Task (PVT) and subjective Karolinska Sleepiness Scale (KSS) data collected from laboratory studies during which subjects were sleep deprived for 26-52h. The algorithm was then tested in data from both laboratory and field experiments. The pattern recognition algorithm was able to identify performance impairment with a single testing session in individuals studied under laboratory conditions using PVT, KSS, length of time awake and time of day information with sensitivity and specificity as high as 82%. When this algorithm was tested on data collected under real-world conditions from individuals whose data were not in the training set, accuracy of predictions for individuals categorized with low performance impairment were as high as 98%. Predictions for medium and severe performance impairment were less accurate. We conclude that pattern recognition algorithms may be a promising method for identifying performance impairment in individuals using only current information about the individual's behavior. Single testing features (e.g., number of PVT lapses) with high correlation with performance impairment in the laboratory setting may not be the best indicators of performance impairment under real-world conditions. Pattern recognition algorithms should be further tested for their ability to be used in conjunction with other assessments of sleepiness in real-world conditions to quantify performance impairment in response to sleep loss.

Figure 1. Number of PVT lapses as a function of length of time awake

Twelve subjects underwent a 50-hr sleep deprivation where the PVT was administered every 2 hours. The average number of PVT lapses (RT > 500 msec) were computed over the first, second and third 16 hours of wakefulness for each subject and then across subjects. The number of PVT lapses for each time awake bin is plotted as mean ± standard deviation.

Figure 2. Classification of response by time within study for the two different methods

The number of test sessions classified as 1, 2 or 3 were compared between EDWS 1, the first extended duration work shift of interns on a Q3 schedule, and EDWS 6, the sixth extended duration work shift ~18 days later. Test sessions classified as 1 decreased from EDWS 1 to EDWS 6 and those classified as 2 or 3 increased across EDWS for both the kNN (lower left panel) and Naïve Bayes (upper right panel) methods as well as the classification based on PVT lapses (lower right panel). Actual classification, determined post-hoc based on relative mean slowest 10% RTs, is presented for comparison (upper left panel).