New finds in the palaeoanthropological and genomic records have changed our view of the origins of modern human ancestry. Here we review our current understanding of how the ancestry of modern humans around the globe can be traced into the deep past, and which ancestors it passes through during our journey back in time. We identify three key phases that are surrounded by major questions, and which will be at the frontiers of future research. The most recent phase comprises the worldwide expansion of modern humans between 40 and 60 thousand years ago (ka) and their last known contacts with archaic groups such as Neanderthals and Denisovans. The second phase is associated with a broadly construed African origin of modern human diversity between 60 and 300 ka. The oldest phase comprises the complex separation of modern human ancestors from archaic human groups from 0.3 to 1 million years ago. We argue that no specific point in time can currently be identified at which modern human ancestry was confined to a limited birthplace, and that patterns of the first appearance of anatomical or behavioural traits that are used to define Homo sapiens are consistent with a range of evolutionary histories.