Regular exercise is important for the maintenance of good health, but many people do very little exercise. There are public health concerns about activating the sedentary and, from an epidemiological measurement perspective, there are advantages in assessing physical inactivity. Using pooled data from 17,053 participants in Australian population surveys, we identified 5078 people (29.7%) who could be classified as sedentary in their recreational exercise habits. Logistic regression analyses found the inactive to be more likely to be older, less well educated, and to have lower incomes. The main reasons given for not exercising were no time (33%), being physically unable (23%), and not wanting to exercise (13%): these were more likely to be reported by those who were older, and who had lower incomes; the only exception was that being 55 years or older was not associated with having less time. Women were more likely to report being physically unable to exercise. Lower income was associated with being physically unable, and with not wanting to exercise. Those with children, while wanting to exercise, were more likely to report having no time. The greatest health benefits will result from the sedentary becoming more active, and inexpensive and convenient activities such as walking need to be emphasized.