We analyzed genome-wide single-nucleotide polymorphism data of 11,069 Japanese individuals recruited from all 47 prefectures of Japan to clarify their genetic structure. The principal component analysis at the prefectural level enabled us to study the relationship between geographical location and genetic differentiation. The results revealed that the mainland Japanese were not genetically homogeneous, and the genetic structure could be explained mainly by the degree of Jomon ancestry and the geographical location. One of the interesting findings was that individuals in the Shikoku region (i.e., Tokushima Prefecture, Kagawa Prefecture, Ehime Prefecture, and Kochi Prefecture) were genetically close to Han Chinese. Therefore, the genetic components of immigrants from continental East Asia in the Yayoi period may have been well maintained in Shikoku. The present results will be useful for understanding the peopling of Japan, and also provide suggestions for recruiting subjects in genetic association studies.