A Comparison of the Affectiva iMotions Facial Expression Analysis Software With EMG for Identifying Facial Expressions of Emotion

Abstract

Human faces express emotions, informing others about their affective states. In order to measure expressions of emotion, facial Electromyography (EMG) has widely been used, requiring electrodes and technical equipment. More recently, emotion recognition software has been developed that detects emotions from video recordings of human faces. However, its validity and comparability to EMG measures is unclear. The aim of the current study was to compare the Affectiva Affdex emotion recognition software by iMotions with EMG measurements of the zygomaticus mayor and corrugator supercilii muscle, concerning its ability to identify happy, angry and neutral faces. Twenty participants imitated these facial expressions while videos and EMG were recorded. Happy and angry expressions were detected by both the software and by EMG above chance, while neutral expressions were more often falsely identified as negative by EMG compared to the software. Overall, EMG and software values correlated highly. In conclusion, Affectiva Affdex software can identify facial expressions and its results are comparable to EMG findings.

Keywords: EMG; affectiva; automatic recognition; emotion recognition software; facial expressions of emotion.

FIGURE 1

Three-dimensional scatter plot of mean difference scores between the Affectiva scores for smile and brow furrow and the difference scores between “joy” and “anger,” as well as between the EMG amplitudes for zygomaticus mayor and corrugator supercilii activity. Note that the difference score is computed to be more negative (closer to –1) if the respective measure indicates a more negative (i.e. angry) expression and more positive (closer to 1) if the measure indicated a positive (i.e. happy) expression. Red dots indicate the difference scores in the angry condition, green dots in the happy condition and blue dots in the neutral condition. Difference scores of all three types were significantly positive (close to 1) in the happy condition and significantly negative (close to –1) in the angry condition.

FIGURE 2

Three-dimensional scatter plot of mean difference scores between the Affectiva scores for smile and brow furrow and the scores for “joy” and “anger” during the EMG condition, as well as the EMG amplitudes for zygomaticus mayor and corrugator supercilii activity. Note that the difference score is computed to be more negative (closer to –1) if the respective measure indicates a more negative (i.e., angry) expression and more positive (closer to 1) if the measure indicated a positive (i.e., happy) expression. Red dots indicate the difference scores in the angry condition, green dots in the happy condition and blue dots in the neutral condition.