Eye-Tracking Analysis for Emotion Recognition

Abstract

This article reports the results of the study related to emotion recognition by using eye-tracking. Emotions were evoked by presenting a dynamic movie material in the form of 21 video fragments. Eye-tracking signals recorded from 30 participants were used to calculate 18 features associated with eye movements (fixations and saccades) and pupil diameter. To ensure that the features were related to emotions, we investigated the influence of luminance and the dynamics of the presented movies. Three classes of emotions were considered: high arousal and low valence, low arousal and moderate valence, and high arousal and high valence. A maximum of 80% classification accuracy was obtained using the support vector machine (SVM) classifier and leave-one-subject-out validation method.

Figure 1

Distribution of emotions caused by movies (numbered from 1 to 21) on the valence-arousal plane along with created classes (marked red).

Figure 2

Typical sequence of the presented movies.

Figure 3

Preprocessing of eye-tracking signal: (a) pure signal, (b) signal after interpolation, and (c) signal after low-pass filtering.

Figure 4

A fragment of an eye-tracking (fixations, saccades, and coordinates of the “sight”).

Figure 5

The distribution of the average pupil diameter and the average saccade amplitude for the three classes of emotions.

Figure 6

Removing the effect of movie luminance: (a) calculated luminance of a movie; (b) blue line, registered pupil diameter for a participant; red line, estimated pupil diameter (as a response for movie luminance); (c) pupil diameter after removing the luminance effect.

Figure 7

Two sample frames from movies with one participant's gaze positions.

Figure 8

Changes in pupil eye diameter of one of the participants after removing the influence of luminance calculated using two methods.

Figure 9

Movie dynamics index.

Figure 10

Dependence of arousal parameter on the movie dynamics.