Autism is associated with widespread atypicalities in perception, cognition and social behavior. A crucial question concerns how these atypicalities are reflected in the underlying brain activation. One way to examine possible perturbations of cortical organization in autism is to analyze the activation of category-selective ventral visual cortex, already clearly delineated in typical populations. We mapped out the neural correlates of face, place and common object processing, using functional magnetic resonance imaging (fMRI), in a group of high-functioning adults with autism and a typical comparison group, under both controlled and more naturalistic, viewing conditions. There were no consistent group differences in place-related regions. Although there were no significant differences in the extent of the object-related regions, there was more variability for these regions in the autism group. The most marked group differences were in face-selective cortex, with individuals with autism evincing reduced activation, not only in fusiform face area but also in superior temporal sulcus and occipital face area. Ventral visual cortex appears to be organized differently in high-functioning adults with autism, at least for face-selective regions, although subtle differences may also exist for other categories. We propose that cascading developmental effects of low-level differences in neuronal connectivity result in a much more pronounced effect on later developing cortical systems, such as that for face-processing, than earlier maturing systems (those for objects and places).