Performance measures of physical function (gait speed, chair stands, standing balance) and cognitive function [Teng-modified Mini-Mental Status Exam (3MS) and digit symbol substitution test (DSST)] were assessed at baseline in 3,075 participants in the Health, Aging and Body Composition Study. Each physical function measure was examined for the strength and magnitude of association with cognitive function. All physical function measures were associated with both the 3MS and DSST scores (p < 0.001), and in multivariate analysis each relationship was independent of demographic characteristics, weight, physical activity and comorbid health conditions of participants. The association of motor performance was consistently greater for the DSST than the 3MS and, among the motor tests, gait speed retained a significant association with both cognitive measures independent of demographic, weight, physical activity and comorbid health conditions. In this large cohort of high-functioning older adults, the correlation between physical and cognitive function was not entirely explained by demographics. Longitudinal studies are needed to determine the direction of causality in this relationship.