Embryonic stem (ES) cells are invaluable for their therapeutic potential as well as for the study of early development. Their clinical use demands an understanding of ES cell differentiation, particularly with respect to cell proliferation and the maintenance of genomic integrity, processes for which the transcription factor p53 is essential. However, although the function of p53 as a tumor suppressor has been extensively studied, its role in ES cell biology has not been clearly elucidated. To study p53 activity and regulation in differentiating ES cells, we used knock-in constructs to create a novel reporter system that provides a direct readout of p53 transcriptional activity. We thereby determine that the p53 pathway is active in ES cells, but that p53 activity and the p53-dependent stress response decrease upon differentiation. Although p53 protein levels and activity are usually primarily controlled by the ubiquitin ligase MDM2, we identify the MDM2 homolog MDM4 as the key modulator of p53 activity in differentiating ES cells. Our results provide a better understanding of ES cell regulation and could help to optimize ES cell differentiation protocols for their use in regenerative medicine.