Complex human behavior is organized around temporally distal outcomes. Behavioral studies based on tasks such as normal prehension, multi-step object use and imitation establish the existence of relative hierarchies of motor control. The retrieval errors in apraxia also support the notion of a hierarchical model for representing action in the brain. In this review, three functional brain imaging studies of action observation using the method of repetition suppression are used to identify a putative neural architecture that supports action understanding at the level of kinematics, object centered goals and ultimately, motor outcomes. These results, based on observation, may match a similar functional-anatomic hierarchy for action planning and execution. If this is true, then the findings support a functional-anatomic model that is distributed across a set of interconnected brain areas that are differentially recruited for different aspects of goal-oriented behavior, rather than a homogeneous mirror neuron system for organizing and understanding all behavior.