The Jomon and the Yayoi are considered to be the two major ancestral populations of the modern mainland Japanese. The Jomon people, who inhabited mainland Japan, admixed with Yayoi immigrants from the Asian continent. To investigate the population history in the Jomon period (14,500-2,300 years before present [YBP]), we analyzed whole Y-chromosome sequences of 345 Japanese males living in mainland Japan. A phylogenetic analysis of East Asian Y chromosomes identified a major clade (35.4% of mainland Japanese) consisting of only Japanese Y chromosomes, which seem to have originated from indigenous Jomon people. A Monte Carlo simulation indicated that ~70% of Jomon males had Y chromosomes in this clade. The Bayesian skyline plots of 122 Japanese Y chromosomes in the clade detected a marked decrease followed by a subsequent increase in the male population size from around the end of the Jomon period to the beginning of the Yayoi period (2,300 YBP). The colder climate in the Late to Final Jomon period may have resulted in critical shortages of food for the Jomon people, who were hunter-gatherers, and the rice farming introduced by Yayoi immigrants may have helped the population size of the Jomon people to recover.