Although poorer self-rated health (SRH) is associated with increased mortality, less is known about its impact on functioning. This study evaluated whether poorer SRH was associated with decline in walking speed and whether caregiving, often considered an indicator of chronic stress, modified this relation. The sample included 891 older US women from the Caregiver-Study of Osteoporotic Fractures. SRH was assessed at the baseline Caregiver-Study of Osteoporotic Fractures interview, conducted in 1999-2001, and was categorized as fair/poor or excellent/good. Rapid walking speed over 2, 3, or 6 m was measured at baseline and 2 annual follow-up interviews. Respondents with fair/poor SRH walked significantly slower at baseline than those with excellent/good health (mean = 0.8 (standard deviation, 0.3) vs. 1.0 (standard deviation, 0.3) m/second, P < 0.001). In adjusted linear mixed models of percentage change in walking speed, respondents with fair/poor SRH experienced a greater decline in walking speed than those with excellent/good SRH (-5.66% vs. -0.60%, P = 0.01). Caregivers with fair/poor SRH declined more than noncaregivers (-9.26% vs. -4.09%). High-intensity caregivers had the largest decline (-12.88%), whereas low-intensity caregivers in excellent/good SRH had no decline (2.61%). In summary, poorer SRH was associated with decline in walking speed in older women, and the stress of caregiving may have exacerbated its impact.