Research indicates that apoliprotein E (ApoE) plays a role in the development of Alzheimer's disease (AD) and possibly in the cognitive decline associated with normative aging. More recently, researchers have shown that ApoE is expressed in olfactory brain structures, and a relationship among ApoE, AD, and olfactory function has been proposed. In the current analyses, we investigated the contribution of ApoE and odor identification in decline trajectories associated with normative cognitive aging in various domains, using longitudinal data on cognitive performance available from the Swedish Adoption/Twin Study of Aging. Data on both ApoE status and olfactory functioning were available from 455 individuals ranging in age from 50 to 88 years at the first measurement occasion. Odor identification was measured via a mailed survey. Cognitive performance was assessed in up to 5 waves of in-person testing covering a period of 16 years. Latent growth curve analyses incorporating odor identification and ApoE status indicated a main effect of odor identification on the performance level in three cognitive domains: verbal, memory, and speed. A main effect of ApoE on rates of decline after age 65 was found for verbal, spatial, and speed factors. The consistency of results across cognitive domains provides support for theories that posit central nervous system-wide origins of the olfaction-cognition-ApoE relationship; however, olfactory errors and APOE ε4 show unique and differential effects on cognitive trajectory features.