Perception of face and body expressions using electromyography, pupillometry and gaze measures

Abstract

Traditional emotion theories stress the importance of the face in the expression of emotions but bodily expressions are becoming increasingly important as well. In these experiments we tested the hypothesis that similar physiological responses can be evoked by observing emotional face and body signals and that the reaction to angry signals is amplified in anxious individuals. We designed three experiments in which participants categorized emotional expressions from isolated facial and bodily expressions and emotionally congruent and incongruent face-body compounds. Participants' fixations were measured and their pupil size recorded with eye-tracking equipment and their facial reactions measured with electromyography. The results support our prediction that the recognition of a facial expression is improved in the context of a matching posture and importantly, vice versa as well. From their facial expressions, it appeared that observers acted with signs of negative emotionality (increased corrugator activity) to angry and fearful facial expressions and with positive emotionality (increased zygomaticus) to happy facial expressions. What we predicted and found, was that angry and fearful cues from the face or the body, attracted more attention than happy cues. We further observed that responses evoked by angry cues were amplified in individuals with high anxiety scores. In sum, we show that people process bodily expressions of emotion in a similar fashion as facial expressions and that the congruency between the emotional signals from the face and body facilitates the recognition of the emotion.

Keywords: electromyography; emotional body language; facial expressions; fixations; pupil dilation; scenes.

Figure 1

Stimulus examples. Bodily (blurred facial features) and facial expressions of emotion.

Figure 2

Gaze fixations on emotional face and body expressions. Fixation and saccade maps of one participant on three different trials. The fixation duration based heat maps show data from all participants on all trials per emotional face condition. For visualization purposes, these heat maps are presented against a background of one exemplar stimulus from that condition. The heat maps show that participants had more fixations on fearful faces, covering a greater area, yet with a clear center on the eye region. The picture with the body postures shows the distribution of fixations and saccades of one participant on three different trials. The yellow lines around the head and body are the interest areas and the yellow numbers, the percentage of the total fixation duration that fell in either the face or the body ROI. The blue arrows indicate the saccades, starting from the chest (fixation cross) and the blue numbers the fixation durations.

Figure 3

Categorizing facial and bodily expressions of emotion. Means of all measurements. The error bars represent the standard error.

Figure 4

Categorizing facial and bodily expressions of emotion. The figure shows one stimulus exemplar per condition with a superimposed fixation map (duration based and averaged per condition). In experiment 2, participants categorized facial expressions, whereas in experiment 3, they categorized bodily expressions. For visualization purposes, these heat maps are presented against a background of one exemplar stimulus from that condition.