This study used functional magnetic resonance imaging to examine the neural substrates of empathy, Theory of Mind (ToM), and self-other differentiation involved in the adaptive understanding of people's internal states. Three conditions were distinguished in both sad and neutral (no obvious emotion) contexts. The empathy condition involved imagining what another person is feeling while the more cognitively loaded ToM condition involved imagining what would make another person feel better. The self-reference condition required participants to imagine how they would feel in someone else's situation. Areas previously implicated in empathy, ToM, and self-other differentiation were identified within the different conditions, regardless of emotional context. Specifically, the frontal and temporal poles responded more strongly for ToM than for empathy. The self-reference condition was associated with stronger dorsolateral prefrontal response than the empathy condition, while the reverse comparison revealed a stronger role for right frontal pole. Activations in frontal pole and orbitofrontal cortex were shared between the three conditions. Contrasts of parameter estimates demonstrated modulation by emotional context. The findings of common and differential patterns of responding observed in prefrontal and temporal regions suggest that within the social cognition network empathy, ToM and self-other differentiation have distinct roles that are responsive to context.