De novo DNA methylation (DNAme) in mammalian germ cells is dependent on DNMT3A and DNMT3L. However, oocytes and spermatozoa show distinct patterns of DNAme. In mouse oocytes, de novo DNAme requires the lysine methyltransferase (KMTase) SETD2, which deposits H3K36me3. We show here that SETD2 is dispensable for de novo DNAme in the male germline. Instead, the lysine methyltransferase NSD1, which broadly deposits H3K36me2 in euchromatic regions, plays a critical role in de novo DNAme in prospermatogonia, including at imprinted genes. However, males deficient in germline NSD1 show a more severe defect in spermatogenesis than Dnmt3l-/- males. Notably, unlike DNMT3L, NSD1 safeguards a subset of genes against H3K27me3-associated transcriptional silencing. In contrast, H3K36me2 in oocytes is predominantly dependent on SETD2 and coincides with H3K36me3. Furthermore, females with NSD1-deficient oocytes are fertile. Thus, the sexually dimorphic pattern of DNAme in mature mouse gametes is orchestrated by distinct profiles of H3K36 methylation.