Obstructive azoospermia (OA) is one of the most common causes of male infertility. Transrectal ultrasound (TRUS) has been used to diagnose OA for many years. From 2009 to 2013, we evaluated a prospective cohort of 1249 patients with suspected OA using TRUS. It was found that dilation of the ejaculatory duct (ED) (29.9%, 374/1249) was the most common cause of OA, followed by seminal vesicle (SV) abnormalities (28.5%, 356/1249). A total of 237 patients were diagnosed with congenital defects (agenesis and/or hypoplasia) of the SV, constituting more than half of the cases of SV disease in OA (19.0%, 237/1249). In contrast to ED, congenital defects of the SV could not be corrected with surgical treatment. Therefore, it is meaningful to compare TRUS and magnetic resonance imaging (MRI) for accurate diagnosis of SV defects. Among our patients, 30 with agenesis or/and hypoplasia of the SV on TRUS were further evaluated using pelvic MRI within 2 years, with the objective of verifying the TRUS results. The concordance rate for diagnosing congenital defects of the SV was 73.3% (22/30). We concluded that TRUS is a reliable and convenient method for diagnosing agenesis or hypoplasia of the SV in OA patients with a high concordance with MRI while MRI is useful in patients with inconclusive TRUS findings.