This study piloted a role play assessment of conversational skills for adolescents and young adults with high-functioning autism/Asperger syndrome (HFA/AS). Participants completed two semi-structured role plays, in which social context was manipulated by changing the confederate's level of interest in the conversation. Participants' social behavior was rated via a behavioral coding system, and performance was compared across contexts and groups. An interaction effect was found for several items, whereby control participants showed significant change across context, while participants with HFA/AS showed little or no change. Total change across contexts was significantly correlated with related social constructs and significantly predicted ASD. The findings are discussed in terms of the potential utility of the CASS in the evaluation of social skill.