When the motivation for exercise is high and people are retired, the cost of time used for physical exercise may be lower and individuals may exercise more compared to individuals with a low motivational level and in working life. The aim was to study the effect of time cost of physical exercise on the amount of physical exercise and on health-related quality of life. We used 2-year data (n = 1,292) from a 4-year randomised controlled trial in a population-based sample of Eastern Finnish men and women, 57-78 years of age at baseline, in 2005-2006. In the statistical analysis, physical exercise and health outcomes were assumed to be endogenous variables explained with a set of exogenous variables. The statistical modelling was done by panel data instrumental variable regressions. Health-related quality of life was evaluated by the RAND 36-item survey and motives for exercise with a questionnaire. Joy as the motivation for physical exercise and retirement increased the amount of physical exercise per week (p < 0.001). A higher amount of exercise was associated with physical (p < 0.001) and mental (p < 0.001) components of quality of life. Moreover, a higher amount of physical exercise decreased the metabolic risk factor score (p < 0.001). The motivation and extra time, i.e. retirement, have a significant impact on the time spent on physical exercise (p < 0.001). Our data agree with the theory that high motivation and retirement lower the time cost of physical exercise. The results emphasise that motivation and the labour market position are important in determining the cost of physical exercise.