The ability to detect agency is fundamental for understanding the social world. Underlying this capacity are neural circuits that respond to patterns of intentional biological motion in the superior temporal sulcus and temporoparietal junction. Here we show that the brain's blood oxygenation level dependent (BOLD) response to such motion is modulated by the representation of the actor. Dynamic social interactions were portrayed by either live-action agents or computer-animated agents, enacting the exact same patterns of biological motion. Using an event-related design, we found that the BOLD response associated with the perception and interpretation of agency was greater when identical physical movements were performed by real rather than animated agents. This finding has important implications for previous work on biological motion that has relied upon computer-animated stimuli and demonstrates that the neural substrates of social perception are finely tuned toward real-world agents. In addition, the response in lateral temporal areas was observed in the absence of instructions to make mental inferences, thus demonstrating the spontaneous implementation of the intentional stance.