Cognitive complaints by breast cancer survivors receiving chemotherapy have led to an increasing interest in elucidating the possible causes of such impairment. Although a number of neuroimaging studies have been conducted, only a handful of them have taken into account cognitive status pre-chemotherapy. The current study included pre-chemotherapy and post-chemotherapy assessment. In addition, various factors such as depression, anxiety, fatigue and days since surgery were considered during analyses. Breast cancer patients performed an fMRI verbal recall task before and an average of 1 month after chemotherapy. Well matched controls also performed the task with a similar timeline. Pre-chemotherapy analyses revealed that patients activated the anterior cingulate less than controls during memory retrieval when anxiety and fatigue scores were added as covariates during group comparisons. In addition, there were also changes in brain activation from pre- to post-chemotherapy in patients but not in controls. Post-chemotherapy, patients had less activation in the bilateral insula, the left inferior orbitofrontal cortex and the left middle temporal gyrus. Finally, patients also showed significantly less activation when compared to controls. Brain regions included: the right middle and superior temporal gyrus, the right medial frontal gyrus, the right inferior orbitofrontal cortex, the left insula and left superior temporal pole. Importantly, depression, anxiety, and particularly fatigue accounted for some of brain activation differences. Our results suggest that chemotherapy in part plays a role in brain activation differences and it also highlights the importance of rigorously controlling for confounding variables. Only by controlling such factors can we understand the role that chemotherapy may play on cognition.