We sought to develop and evaluate a composite memory score from the neuropsychological battery used in the Alzheimer's Disease (AD) Neuroimaging Initiative (ADNI). We used modern psychometric approaches to analyze longitudinal Rey Auditory Verbal Learning Test (RAVLT, 2 versions), AD Assessment Schedule - Cognition (ADAS-Cog, 3 versions), Mini-Mental State Examination (MMSE), and Logical Memory data to develop ADNI-Mem, a composite memory score. We compared RAVLT and ADAS-Cog versions, and compared ADNI-Mem to RAVLT recall sum scores, four ADAS-Cog-derived scores, the MMSE, and the Clinical Dementia Rating Sum of Boxes. We evaluated rates of decline in normal cognition, mild cognitive impairment (MCI), and AD, ability to predict conversion from MCI to AD, strength of association with selected imaging parameters, and ability to differentiate rates of decline between participants with and without AD cerebrospinal fluid (CSF) signatures. The second version of the RAVLT was harder than the first. The ADAS-Cog versions were of similar difficulty. ADNI-Mem was slightly better at detecting change than total RAVLT recall scores. It was as good as or better than all of the other scores at predicting conversion from MCI to AD. It was associated with all our selected imaging parameters for people with MCI and AD. Participants with MCI with an AD CSF signature had somewhat more rapid decline than did those without. This paper illustrates appropriate methods for addressing the different versions of word lists, and demonstrates the additional power to be gleaned with a psychometrically sound composite memory score.