The present survey develops a previous position paper, in which I suggested that the multimodal semantic impairment observed in advanced stages of semantic dementia is due to the joint disruption of pictorial and verbal representations, subtended by the right and left anterior temporal lobes, rather than to the loss of a unitary, amodal semantic system. The main goals of the present review are (a) to survey a larger set of data, in order to confirm the differences in conceptual representations at the level of the right and left hemispheres, (b) to examine if language-mediated information plays a greater role in left hemisphere semantic knowledge than sensory-motor information in right hemisphere conceptual knowledge, and (c) to discuss the models that could explain both the differences in conceptual representations at the hemispheric level and the prevalence of the left hemisphere language-mediated semantic knowledge over the right hemisphere perceptually based conceptual representations.