Several trials comparing the efficacy of standard melphalan and prednisone (MP) therapy with MP plus thalidomide (MPT) in elderly patients with multiple myeloma (MM) have been reported, with inconsistent results. The primary goal of our study was to evaluate the efficacy and toxicity of MP versus MPT in newly diagnosed patients with MM who were transplant-ineligible or over age 65. A total of 135 patients were enrolled. Either minimal response or better or partial response or better were more frequent with MPT treatment (p = 0.001). After a median follow-up of 30 months, median progression-free survival (PFS) and overall survival (OS) were 33 and 52 months for MPT versus 22 and 32 months for MP, respectively. The comparison showed a significant advantage for MPT versus MP in PFS (p = 0.02) and only a trend for OS (p = 0.07). Severe adverse events were observed more frequently with MPT. In conclusion, our results show an improved activity of MPT at a cost of increased toxicity. We believe that MPT can be considered one of the new standard of care for elderly or transplant-ineligible patients with MM.