Predicting Student Performance Using Machine Learning in fNIRS Data

Abstract

Increasing student involvement in classes has always been a challenge for teachers and school managers. In online learning, some interactivity mechanisms like quizzes are increasingly used to engage students during classes and tasks. However, there is a high demand for tools that evaluate the efficiency of these mechanisms. In order to distinguish between high and low levels of engagement in tasks, it is possible to monitor brain activity through functional near-infrared spectroscopy (fNIRS). The main advantages of this technique are portability, low cost, and a comfortable way for students to concentrate and perform their tasks. This setup provides more natural conditions for the experiments if compared to the other acquisition tools. In this study, we investigated levels of task involvement through the identification of correct and wrong answers of typical quizzes used in virtual environments. We collected data from the prefrontal cortex region (PFC) of 18 students while watching a video lecture. This data was modeled with supervised learning algorithms. We used random forests and penalized logistic regression to classify correct answers as a function of oxyhemoglobin and deoxyhemoglobin concentration. These models identify which regions best predict student performance. The random forest and penalized logistic regression (GLMNET with LASSO) obtained, respectively, 0.67 and 0.65 area of the ROC curve. Both models indicate that channels F4-F6 and AF3-AFz are the most relevant for the prediction. The statistical significance of these models was confirmed through cross-validation (leave-one-subject-out) and a permutation test. This methodology can be useful to better understand the teaching and learning processes in a video lecture and also provide improvements in the methodologies used in order to better adapt the presentation content.

Keywords: education; fNIRS; logistic regression; machine learning; neuroscience; prefrontal cortex; random forest.

Figure 1

The questions are based on content exposed at earlier times throughout the video (indicated in blue). The red dots show the exact timing of the questions.

Figure 2

Montage layout: The position of the optodes follows the universal configuration of the 10-10.

Figure 3

Double cross-validation implemented: In the outer (external) loop of double cross-validation, each interaction excludes one subject and all remaining data subjects are divided into two subsets referred to as training and test sets. The training set used in the inner (internal) loop, while the test set was exclusively used for model assessment.

Figure 4

The ROC curve is created by plotting the true positive rate (sensitivity) against the false positive rate (specificity) at various threshold settings.

Figure 5

In this map, the red dots represent the sources and the yellow dots the detectors. We identified the most important channels from the total iterations in training the model. The frequency of the main covariables identified were: deoxyhemoglobin (blue circles) in channel 18 (highly relevant in all subjects) and oxyhemoglobin (green circles) in channel 3 (present in 60% of the subjects).

Figure 6

Random forest outputs: level of importance of each covariate with a detailed zoom at the top-5 ones.

Figure 7

Boxplots show differences between the groups: 1, certainly right exercise; 0.5, not sure/next idea; 0, probably wrong/random guess.

Figure 8

In these maps, the red dots represent the sources and the yellow dots the detectors. Panel (A) refers to the GLMNET Model output and strongly indicates channel 18 HHb (F4-F6) and channel 4 O2Hb (AF7-FP1). Panel (B) refers to the Random Forest Model output and indicates greater relevance for channel 18 HHb (F4-F6) and channel 7 HHb (AF3- AFz). The channel 18 region is the dorsolateral prefrontal region, associated with attention and working memory.