The relative contributions of the hippocampus and the perirhinal cortex to recognition memory are currently the subject of intense debate. Whereas some authors propose that both structures play a similar role in recognition memory, others suggest that the hippocampus might mediate recollective and/or associative aspects of recognition memory, whereas the perirhinal cortex may mediate item memory. Here we investigate an alternative functional demarcation between these structures, following reports of stimulus-specific perceptual deficits in amnesics with medial temporal lobe (MTL) lesions. Using a novel recognition memory test for faces and scenes, participants with broad damage to MTL structures, which included the hippocampus and the perirhinal cortex, were impaired on both face and scene memory. By contrast, participants with damage limited to the hippocampus showed deficits only in memory for scenes. These findings imply that although both the hippocampus and surrounding cortex contribute to recognition memory, their respective roles can be distinguished according to the type of material to be remembered. This interaction between lesion site and stimulus category may explain some of the inconsistencies present in the literature.