Computer Vision Analysis of Reduced Interpersonal Affect Coordination in Youth With Autism Spectrum Disorder

Abstract

Atypical social-emotional reciprocity is a core feature of autism spectrum disorder (ASD) but can be difficult to operationalize. One measurable manifestation of reciprocity may be interpersonal coordination, the tendency to align the form and timing of one's behaviors (including facial affect) with others. Interpersonal affect coordination facilitates sharing and understanding of emotional cues, and there is evidence that it is reduced in ASD. However, most research has not measured this process in true social contexts, due in part to a lack of tools for measuring dynamic facial expressions over the course of an interaction. Automated facial analysis via computer vision provides an efficient, granular, objective method for measuring naturally occurring facial affect and coordination. Youth with ASD and matched typically developing youth participated in cooperative conversations with their mothers and unfamiliar adults. Time-synchronized videos were analyzed with an open-source computer vision toolkit for automated facial analysis, for the presence and intensity of facial movements associated with positive affect. Both youth and adult conversation partners exhibited less positive affect during conversations when the youth partner had ASD. Youth with ASD also engaged in less affect coordination over the course of conversations. When considered dimensionally across youth with and without ASD, affect coordination significantly predicted scores on rating scales of autism-related social atypicality, adaptive social skills, and empathy. Findings suggest that affect coordination is an important interpersonal process with implications for broader social-emotional functioning. This preliminary study introduces a promising novel method for quantifying moment-to-moment facial expression and emotional reciprocity during natural interactions. LAY SUMMARY: This study introduces a novel, automated method for measuring social-emotional reciprocity during natural conversations, which may improve assessment of this core autism diagnostic behavior. We used computerized methods to measure facial affect and the degree of affect coordination between conversation partners. Youth with autism displayed reduced affect coordination, and reduced affect coordination predicted lower scores on measures of broader social-emotional skills.

Keywords: affect/emotion; computer vision; facial expression; interpersonal coordination; social-emotional reciprocity; synchrony.