Chromodomain Y (CDY) is one of the candidate genes for male dyszoospermia related to Y chromosome microdeletion (YCM). However, the function of CDY in regulating spermatogenesis has not been completely determined. The mouse Cdyl (CDY-like) gene is the homolog of human CDY. In the present study, we generated a germline conditional knockout (cKO) model of mouse Cdyl. Significantly, the CdylcKO male mice suffered from the defects in spermatogonia maintenance and spermatozoon morphogenesis, demonstrating teratozoospermia and a progressive infertility phenotype in early adulthood. Importantly, patterns of specific histone methylation and acetylation were extensively changed, which disturbed the transcriptome in CdylcKO testis. Our findings indicated that Cdyl is crucial for spermatogenesis and male fertility, which provides novel insights into the function of CDY gene, as well as the pathogenesis of YCM-related reproductive failure.