Parathyroidectomy (PTx) drastically improves biochemical parameters and clinical symptoms related to severe secondary hyperparathyroidism (SHPT) but the effect of PTx on survival has not been adequately investigated. Here we analyzed data on 114,064 maintenance hemodialysis patients from a nationwide registry of the Japanese Society for Dialysis Therapy to evaluate the associations of severity of SHPT and history of PTx with 1-year all-cause and cardiovascular mortality. We then compared the mortality rate between 4428 patients who had undergone PTx and 4428 propensity score-matched patients who had not despite severe SHPT. During a 1-year follow-up, 7926 patients of the entire study population died, of whom 3607 died from cardiovascular disease. Among patients without a history of PTx, severe SHPT was associated with an increased risk for all-cause and cardiovascular mortality. However, such an increased risk of mortality was not observed among patients with a history of PTx. In the propensity score-matched analysis, patients who had undergone PTx had a 34% and 41% lower risk for all-cause and cardiovascular mortality, respectively, compared to the matched controls. The survival benefit associated with PTx was robust in several sensitivity analyses and consistent across subgroups, except for those who had persistent postoperative SHPT. Thus, successful PTx may reduce the risk for all-cause and cardiovascular mortality in hemodialysis patients with severe, uncontrolled SHPT.