Following fertilization in mammals, the gametes are reprogrammed to create a totipotent zygote, a process that involves de novo establishment of chromatin domains. A major feature occurring during preimplantation development is the dramatic remodelling of constitutive heterochromatin, although the functional relevance of this is unknown. Here, we show that heterochromatin establishment relies on the stepwise expression and regulated activity of SUV39H enzymes. Enforcing precocious acquisition of constitutive heterochromatin results in compromised development and epigenetic reprogramming, which demonstrates that heterochromatin remodelling is essential for natural reprogramming at fertilization. We find that de novo H3K9 trimethylation (H3K9me3) in the paternal pronucleus after fertilization is catalysed by SUV39H2 and that pericentromeric RNAs inhibit SUV39H2 activity and reduce H3K9me3. De novo H3K9me3 is initially non-repressive for gene expression, but instead bookmarks promoters for compaction. Overall, we uncover the functional importance for the restricted transmission of constitutive heterochromatin during reprogramming and a non-repressive role for H3K9me3.