Studies in semantics traditionally focus on knowledge of objects. By contrast, less is known about how objects relate to each other. In an fMRI study, we tested the hypothesis that the neural processing of categorical spatial relations between objects is distinct from the processing of the identity of objects. Attending to the categorical spatial relations compared with attending to the identity of objects resulted in greater activity in superior and inferior parietal cortices (especially on the left) and posterior middle frontal cortices bilaterally. In an accompanying lesion study, we tested the hypothesis that comparable areas would be necessary to represent categorical spatial relations and that the hemispheres differ in their biases to process categorical or coordinate spatial relations. Voxel-based lesion symptom mapping results were consistent with the fMRI observations. Damage to a network comprising left inferior frontal, supramarginal, and angular gyri resulted in behavioral impairment on categorical spatial judgments. Homologous right brain damage also produced such deficits, albeit less severely. The reverse pattern was observed for coordinate spatial processing. Right brain damage to the middle temporal gyrus produced more severe deficits than left hemisphere damage. Additional analyses suggested that some areas process both kinds of spatial relations conjointly and others distinctly. The left angular and inferior frontal gyrus processes coordinate spatial information over and above the categorical processing. The anterior superior temporal gyrus appears to process categorical spatial information uniquely. No areas within the right hemisphere processed categorical spatial information uniquely. Taken together, these findings suggest that the functional neuroanatomy of categorical and coordinate processing is more nuanced than implied by a simple hemispheric dichotomy.