Many women complain of memory and other cognitive/emotional difficulties at times that are associated with changes in estrogen levels. However, the biological mechanisms through which estrogen may exert these effects remain poorly understood. The effect of estrogen treatment on cognition and brain function in healthy women, and those with Alzheimer's disease, is controversial. Here we review the evidence that, in healthy women, estrogen affects the dopaminergic, serotonergic, and cholinergic systems, and brain regions crucial to higher cognitive function and mood. We will also present results from recent in vivo randomized-controlled neuroimaging experiments in our laboratory demonstrating that, in young females, and those in mid-life: (1) brain function is modulated by normal variation in ovarian function; (2) acute loss of ovarian hormones increases neuronal membrane breakdown; and (3) acute suppression of ovarian function is associated with reduced activation of brain regions critical to memory.