The hippocampus and its associated structures play a key role in human memory, yet the underlying neuronal mechanisms remain unknown. Here, we report that during encoding and recognition, single neurons in the medial temporal lobe discriminated faces from inanimate objects. Some units responded selectively to specific emotional expressions or to conjunctions of facial expression and gender. Such units were especially prevalent during recognition, and the responses depended on stimulus novelty or familiarity. Traces of exposure to faces or objects were found a few seconds after stimulus removal as well as 10 hr later. Some neurons maintained a record of previous stimulus presentation that was more accurate than the person's conscious recollection. We propose that the human medial temporal lobe constructs a "cognitive map" of stimulus attributes comparable to the map of the spatial environment described in the rodent hippocampus.