We evaluated the effects of estrogen and tamoxifen, a selective estrogen receptor modulator, on positron emission tomography (PET) measures of brain glucose metabolism and magnetic resonance imaging (MRI) measures of hippocampal atrophy. Three groups of postmenopausal women were studied, women taking estrogen (ERT+), women with breast cancer taking tamoxifen (TAM), and women not taking estrogen or tamoxifen (ERT-). All subjects received a PET scan, an MRI scan, and cognitive testing. The TAM group showed widespread areas of hypometabolism in the inferior and dorsal lateral frontal lobes relative to the other two groups. The ERT- group showed lower metabolism in the inferior frontal cortex and temporal cortex with respect to the ERT+ group. The TAM group also showed significantly lower semantic memory scores than the other two groups. Finally, the TAM group had smaller right hippocampal volumes than the ERT+ group, an effect that was of borderline significance. Both right and left hippocampal volumes were significantly smaller than the ERT+ group when a single outlier was removed. The ERT- group had hippocampal volumes that were intermediate to the other two groups. These findings provide physiological and anatomical evidence for neuroprotective effects of estrogen.