Several studies have associated physical activity with the risk of dementia, but mostly did so during short follow-up. It remains unclear whether physical activity also affects dementia during longer follow-up. We examined the association between physical activity and risk of dementia during a follow-up period up to 14 years. From 1997 to 1999, physical activity was assessed using a validated questionnaire in 4,406 elderly persons (age range 61-97) from the prospective, population-based Rotterdam Study. Follow-up for dementia was complete until January 1, 2011. We used Cox proportional hazards models to assess the association between physical activity and incident dementia. Next, we stratified follow-up time using a cut-off of 4 years. We separately investigated dementia due to Alzheimer disease. During 38,631 person-years, 583 participants developed dementia. When adjusting for age and sex, we found a borderline significant association between higher physical activity and lower risk of dementia (HR 0.95; 95% CI 0.87-1.04). This association was confined to follow-up up to 4 years (HR 0.82; 95% CI 0.71-0.95), and not to follow-up of at least 4 years (HR 1.04; 95% CI 0.93-1.16). Additional adjustments only slightly attenuated the associations. A similar pattern was present for Alzheimer disease. We found a higher level of physical activity to be associated with a lower risk of dementia. This association was confined to follow-up for up to 4 years and not to longer follow-up, suggesting either a role for reverse causality or only a short term effect of late-life physical activity in an elderly population.