Vehicle traffic is increasing worldwide, and this is a major concern because traffic-related air pollution and noise may influence health. The aim of this study was to evaluate whether health-related quality of life (HRQoL) is associated with vehicle traffic density in area of residence. A total of 16,410 individuals, 40 to 45 years old, were asked to participate in this study (response rate: 55% for men, 66% for women). Using the 12-Item Short Form Health Survey (SF-12) questionnaire, both physical and mental HRQoL were investigated. Multiple linear regression analyses showed that women living in areas with high traffic density had significantly poorer physical HRQoL than women living in areas with moderate or low vehicle traffic density. There were no similar findings among men. Mental HRQoL was not associated with vehicle traffic density in the area of residence, neither for women nor for men. There is an association between high vehicle traffic density in residential area and reduced HRQoL in women.