Enduring interest in the Apolipoprotein E (ApoE) polymorphism is ensured by its evolutionary-driven uniqueness in humans and its prominent role in geriatrics and gerontology. We use large samples of longitudinally followed populations from the Framingham Heart Study (FHS) original and offspring cohorts and the Long Life Family Study (LLFS) to investigate gender-specific effects of the ApoE4 allele on human survival in a wide range of ages from midlife to extreme old ages, and the sensitivity of these effects to cardiovascular disease (CVD), cancer, and neurodegenerative disorders (ND). The analyses show that women's lifespan is more sensitive to the e4 allele than men's in all these populations. A highly significant adverse effect of the e4 allele is limited to women with moderate lifespan of about 70 to 95 years in two FHS cohorts and the LLFS with relative risk of death RR = 1.48 (p = 3.6 × 10(-6)) in the FHS cohorts. Major human diseases including CVD, ND, and cancer, whose risks can be sensitive to the e4 allele, do not mediate the association of this allele with lifespan in large FHS samples. Non-skin cancer non-additively increases mortality of the FHS women with moderate lifespans increasing the risks of death of the e4 carriers with cancer two-fold compared to the non-e4 carriers, i.e., RR = 2.07 (p = 5.0 × 10(-7)). The results suggest a pivotal role of non-sex-specific cancer as a nonlinear modulator of survival in this sample that increases the risk of death of the ApoE4 carriers by 150% (p = 5.3 × 10(-8)) compared to the non-carriers. This risk explains the 4.2 year shorter life expectancy of the e4 carriers compared to the non-carriers in this sample. The analyses suggest the existence of age- and gender-sensitive systemic mechanisms linking the e4 allele to lifespan which can non-additively interfere with cancer-related mechanisms.