Most couples with severe male factor infertility are treated with assisted reproduction technology and little has been known about the prognosis of severe male factor infertility itself. We investigated the prognosis of infertile male patients with severe oligozoospermia. Thirty-nine patients with severe nonobstructive oligozoospermia were followed more than 6 months without any medical or surgical intervention. Retrospective analyses of the natural sequence of the condition and influences on the future fertility potential of the study participants were conducted. Sperm concentration, motility, and morphology between first semen analysis and last semen analysis were not significantly different. However, during the follow-up period, 5 (12.8%) patients became azoospermic. In 7 (17.9%) patients, the sperm count declined to a severe level that could be detected only after centrifugation. Three patients underwent microdissection testicular sperm extraction (TESE) for sperm retrieval after confirmation of azoospermia. The sperm retrieval was successful only in 1 of the 3 patients. Therefore, male patients diagnosed with severe oligozoospermia should be informed about possible aggravation of their residual spermatogenesis function and the necessity of intermittent follow-up semen analyses. If follow-up semen tests show a declining tendency, sperm cryopreservation may be recommended for these patients. If azoospermia develops during the follow-up period, early TESE procedure should be considered to improve the chance of sperm retrieval.