The apolipoprotein E (APOE) e4 allele is the most common genetic variant associated with Alzheimer's disease (AD). We sought to investigate the distribution of APOE genotypes across the full clinical AD spectrum including AD, late-stage amnestic mild cognitive impairment (L-aMCI), early-stage aMCI (E-aMCI), subjective memory impairment (SMI), and controls. We prospectively recruited 713 AD patients, 735 aMCI patients, 575 SMI patients, and 8,260 individuals as controls. The frequency of the APOE e4 allele revealed an ordered fashion in the AD (30.8%), L-aMCI (24.0%), E-aMCI (15.1%), SMI (11.7%), and control (9.1%) groups. APOE e3/e4 and e4/e4 genotype frequencies also appeared in an ordered fashion in the AD group (39.1% of e3/e4 and 10.9% of e4/e4), as well as the L-aMCI (28.3% and 9.4%), E-aMCI (22.3% and 3.7%), SMI (18.3% and 1.9%), and control (15.1% and 0.8%) groups. In the comparisons of APOE e3/e3 vs. e3/e4 genotypes, all patient groups had a higher frequency of APOE e3/e4 relative to the control group. Relative to the SMI and E-aMCI groups, the AD and L-aMCI groups had higher frequency of the APOE e3/e4 genotype, and the AD group had a higher frequency relative to the L-aMCI group. However, there was no significant difference between the E-aMCI and SMI groups. In our longitudinal data, APOE e4 carrier showed a steeper incline slope in a clinical dementia rating sum of boxes (CDR-SB) score than APOE e4 non-carrier in SMI (B = 0.0066, p = 0.0104), E-aMCI (B = 0.0313, p < 0.0001), and L-aMCI (B = 0.0178, p = 0.0007). APOE e4 carrier showed a steeper decline slope in the CDR-SB than APOE e4 non-carrier in AD (B = - 0.0309, p = 0.0003). These findings suggest that E-aMCI and SMI are associated with a similarly increased frequency of the APOE e4 allele compared to controls, suggesting a greater genetic risk for AD and the importance of monitoring the allele more closely.