Recent paleoanthropological discoveries reveal a diverse, potentially speciose human fossil record. Such extensive morphological diversity results from the action of divergent evolutionary forces on an evolving lineage. Here, we apply quantitative evolutionary theory to test whether random evolutionary processes alone can explain the morphological diversity seen among fossil australopith and early Homo crania from the Plio-Pleistocene. We show that although selection may have played an important role in diversifying hominin facial morphology in the late Pliocene, this is not the case during the early evolution of the genus Homo, where genetic drift was probably the primary force responsible for facial diversification.