The chromatoid body (CB) is a germ granule in the cytoplasm of postmeiotic haploid round spermatids that is loaded with RNA and RNA-binding proteins. Following the discovery of small non-coding RNA-mediated gene regulation and the identification of PIWI-interacting RNAs (piRNAs) that have crucial roles in germ line development, the function of the CB has slowly begun to be revealed. Male germ cells utilise small RNAs to control the complex and specialised process of sperm production. Several microRNAs have been identified during spermatogenesis. In addition, a high number of piRNAs are present both in embryonic and postnatal male germ cells, with their expression being impressively induced in late meiotic cells and haploid round spermatids. At postmeiotic stage of germ cell differentiation, the CB accumulates piRNAs and proteins of piRNA machinery, as well as several other proteins involved in distinct RNA regulation pathways. All existing evidence suggests a role for the CB in mRNA regulation and small RNA-mediated gene control, but the mechanisms remain uncharacterised. In this review, we summarise the current knowledge of the CB and its association with small RNA pathways.