To date, the effect of tooth loss on all-cause mortality among elderly patients with a different weight group has not been assessed. This retrospective cohort study evaluated the data obtained from a government-sponsored, annual physical examination program for elderly citizens residing in Taipei City during 2005 to 2007, and follow-up to December 31, 2010. We recruited 55,651 eligible citizens of Taipei City aged ≥ 65 years, including 29,572 men and 26,079 women, in our study. Their mortality data were ascertained based on the national death files. The number of missing teeth was used as a representative of oral health status. We used multivariate Cox proportional hazards regression analysis to determine the association between tooth loss and all-cause mortality. After adjustment for all confounders, the hazard ratios (HRs) of all-cause mortality in participants with no teeth, 1 to 9 teeth, and 10 to 19 teeth were 1.36 [95% confidence interval (CI): 1.15-1.61], 1.24 (95% CI: 1.08-1.42), and 1.19 (95% CI: 1.09-1.31), respectively, compared with participants with 20 or more teeth. A significant positive correlation of body mass index (BMI) with all-cause mortality was found in underweight and overweight elderly patients and was represented as a U-shaped curve. Subgroup analysis revealed a significant positive correlation in underweight (no teeth: HR = 1.49, 95% CI: 1.21-1.83; 1-9 teeth: HR = 1.23, 95% CI: 1.03-1.47; 10-19 teeth: HR = 1.20, 95% CI: 1.06-1.36) and overweight participants (no teeth: HR = 1.37, 95% CI: 1.05-1.79; 1-9 teeth: HR = 1.27, 95% CI: 1.07-1.52). The number of teeth lost is associated with an increased risk of all-cause mortality, particularly for participants with underweight and overweight.