Stem cells are characterised by a capacity to self renew and generate progeny capable of differentiating along several defined lineage paths. Embryonic Stem (ES) cells are derived from the inner cell mass (ICM) of early-stage embryos and can contribute to all tissues of the developing embryo. Discovering how ES cell pluripotency and lineage induction is achieved is important for understanding normal development and for successfully applying stem cell-based therapies. A series of recent studies have shown that the chromatin profile of ES cells is unusual and have revealed a critical role for the Polycomb Repressive Complexes (PRCs) in maintaining pluripotency. In human and mouse ES cells many genes that encode transcription factors that are required for lineage specification bind PRC2 and carry bivalent (or opposing) histone signatures, being enriched for conventional indicators of active chromatin such as acetylated H3K9 and methylated H3K4, while lying within domains of repressive trimethylated H3K27. Mutant ES cells that lack H3K27 methylation inappropriately expressed these genes showing that PRC2 represses lineage-specific gene programs in ES cells. Here we discuss the implications of these new discoveries and explore the interdependence of PRC1 and PRC2 in regulating lineage-specific gene expression in ES cells.