Scalp event-related potential (ERP) studies in humans indicate that face processes taking place between 130 and 170 ms after stimulus onset at posterior sites (N170) are strongly reduced when another face stimulus is processed concurrently or has been presented shortly before for a prolonged period. These observations suggest that neural representations of individual faces compete in the occipitotemporal cortex as early as 130 ms. Here, we tested the respective role of spatial attention and sensory competition in accounting for the amplitude reduction of the N170 during concurrent face stimulation. ERPs time locked to a lateralized face stimulus were recorded while subjects were fixating either a face or a controlled scrambled-face stimulus (context factor) and were engaged in either a high- or a low-attentional load task at fixation (task factor). The N170 amplitude to the lateralized face stimulus was reduced both when the central stimulus was a face compared with a scrambled face and when the attentional load at fixation was high. However, these effects of context and task factors were largely additive. Most importantly, spatial attention modulated visual processes as early as 80 ms after stimulus onset, whereas sensory competition effects started at about 130 ms. These results provide strong evidence that the N170 in response to faces is modulated by spatial attention, and also that spatial attention and sensory competition do not reflect the same mechanisms of early selection of visual information in the extrastriate cortex.