The early history of island Southeast Asia is often characterized as the story of two major population dispersals: the initial Paleolithic colonization of Sahul approximately 45 ka ago and the much later Neolithic expansion of Austronesian-speaking farmers approximately 4 ka ago. Here, in the largest survey of Indonesian Y chromosomes to date, we present evidence for multiple genetic strata that likely arose through a series of distinct migratory processes. We genotype an extensive battery of Y chromosome markers, including 85 single-nucleotide polymorphisms/indels and 12 short tandem repeats, in a sample of 1,917 men from 32 communities located across Indonesia. We find that the paternal gene pool is sharply subdivided between western and eastern locations, with a boundary running between the islands of Bali and Flores. Analysis of molecular variance reveals one of the highest levels of between-group variance yet reported for human Y chromosome data (e.g., Phi(ST) = 0.47). Eastern Y chromosome haplogroups are closely related to Melanesian lineages (i.e., within the C, M, and S subclades) and likely reflect the initial wave of colonization of the region, whereas the majority of western Y chromosomes (i.e., O-M119*, O-P203, and O-M95*) are related to haplogroups that may have entered Indonesia during the Paleolithic from mainland Asia. In addition, two novel markers (P201 and P203) provide significantly enhanced phylogenetic resolution of two key haplogroups (O-M122 and O-M119) that are often associated with the Austronesian expansion. This more refined picture leads us to put forward a four-phase colonization model in which Paleolithic migrations of hunter-gatherers shape the primary structure of current Indonesian Y chromosome diversity, and Neolithic incursions make only a minor impact on the paternal gene pool, despite the large cultural impact of the Austronesian expansion.