Since the advent of modern plant breeding in the 1930s, North American maize has undergone a dramatic adaptation to high-input agriculture. Despite the importance of genetic contributions to historical yield increases, little is known about the underlying genomic changes. Here we use high-density SNP genotyping to characterize a set of North American maize lines spanning the history of modern breeding. We provide a unique analysis of genome-wide developments in genetic diversity, ancestry, and selection. The genomic history of maize is marked by a steady increase in genetic differentiation and linkage disequilibrium, whereas allele frequencies in the total population have remained relatively constant. These changes are associated with increasing genetic separation of breeding pools and decreased diversity in the ancestry of individual lines. We confirm that modern heterotic groups are the product of ongoing divergence from a relatively homogeneous landrace population, but show that differential landrace ancestry remains evident. Using a recent association approach, we characterize signals of directional selection throughout the genome, identifying a number of candidate genes of potential agronomic relevance. However, overall we find that selection has had limited impact on genome-wide patterns of diversity and ancestry, with little evidence for individual lines contributing disproportionately to the accumulation of favorable alleles in today's elite germplasm. Our data suggest breeding progress has mainly involved selection and recombination of relatively common alleles, contributed by a representative but limited set of ancestral lines.