Prediction of mental effort derived from an automated vocal biomarker using machine learning in a large-scale remote sample

Nick Taptiklis; Merina Su; Jennifer H Barnett; Caroline Skirrow; Jasmin Kroll; Francesca Cormack

doi:10.3389/frai.2023.1171652

Prediction of mental effort derived from an automated vocal biomarker using machine learning in a large-scale remote sample

Front Artif Intell. 2023 Aug 3:6:1171652. doi: 10.3389/frai.2023.1171652. eCollection 2023.

Authors

Nick Taptiklis¹, Merina Su¹, Jennifer H Barnett^{1

2}, Caroline Skirrow^{1

3}, Jasmin Kroll¹, Francesca Cormack^{1

2}

Affiliations

¹ Cambridge Cognition, Tunbridge Court, Cambridge, United Kingdom.
² Department of Psychiatry, Herschel Smith Building for Brain & Mind Sciences, University of Cambridge, Cambridge, United Kingdom.
³ Department of Psychological Science, University of Bristol, Bristol, United Kingdom.

Abstract

Introduction: Biomarkers of mental effort may help to identify subtle cognitive impairments in the absence of task performance deficits. Here, we aim to detect mental effort on a verbal task, using automated voice analysis and machine learning.

Methods: Audio data from the digit span backwards task were recorded and scored with automated speech recognition using the online platform NeuroVocalix^TM, yielding usable data from 2,764 healthy adults (1,022 male, 1,742 female; mean age 31.4 years). Acoustic features were aggregated across each trial and normalized within each subject. Cognitive load was dichotomized for each trial by categorizing trials at >0.6 of each participants' maximum span as "high load." Data were divided into training (60%), test (20%), and validate (20%) datasets, each containing different participants. Training and test data were used in model building and hyper-parameter tuning. Five classification models (Logistic Regression, Naive Bayes, Support Vector Machine, Random Forest, and Gradient Boosting) were trained to predict cognitive load ("high" vs. "low") based on acoustic features. Analyses were limited to correct responses. The model was evaluated using the validation dataset, across all span lengths and within the subset of trials with a four-digit span. Classifier discriminant power was examined with Receiver Operating Curve (ROC) analysis.

Results: Participants reached a mean span of 6.34 out of 8 items (SD = 1.38). The Gradient Boosting classifier provided the best performing model on test data (AUC = 0.98) and showed excellent discriminant power for cognitive load on the validation dataset, across all span lengths (AUC = 0.99), and for four-digit only utterances (AUC = 0.95).

Discussion: A sensitive biomarker of mental effort can be derived from vocal acoustic features in remotely administered verbal cognitive tests. The use-case of this biomarker for improving sensitivity of cognitive tests to subtle pathology now needs to be examined.

Keywords: automated speech recognition; cognitive load; computerized cognitive assessment; mental effort; remote testing; voice markers; voice-based assessment.

Grants and funding

This project was supported by an Innovate UK grant (103864). Innovate UK did not have a role in the writing of the manuscript of the decision to submit it for publication. Authors had full access to the full data in the study and accept responsibility to submit for publication.