Observation of limb movements in human subjects resulted in increased motor-evoked potential (MEP) amplitude elicited by magnetic stimulation of motor cortex in the muscles involved in that movement, suggesting that an observation-execution matching (OEM) system exists in humans. We investigated whether the OEM system is activated by speech gestures presented in the visual and auditory modalities. We found that visual observation of speech movement enhanced MEP amplitude specifically in muscles involved in production of the observed speech. In contrast, listening to the sound did not produce MEP enhancement. The findings suggest that the OEM system may be modality specific. It may be involved in action recognition in the visual modality, but is not responsible for perception of simple items of sound.