AA genome species in the genus Oryza are valuable resources for improvement of cultivated rice. Oryza rufipogon and O. barthii were progenitors of two domesticated rice species, O. sativa and O. glaberrima, respectively. We used chloroplast single-nucleotide repeats (RCt1-10) to evaluate genetic diversity among AA genome species. Higher diversity was detected in the American species O. glumaepatula and the Asian species O. rufipogon. Other chloroplast sequences indicated that O. glumaepatula shares high similarity with O. longistaminata. Insertions of retrotransposable elements, however, showed a close relation between O. barthii and O. glumaepatula. To clarify phylogenetic relationships among AA genomes, whole-genome sequences obtained from different species were used to develop chloroplast INDEL markers. The INDEL patterns clearly showed multiple maternal origins of O. glumaepatula. The complicated origins have resulted in high genetic diversity in this species. In contrast, the Australian endemic species O. meridionalis tended to show narrower diversity than the other species. High variation in O. rufipogon, reconfirmed using the chloroplast INDELs, covered the variation in O. meridionalis and part of the variation in O. glumaepatula. Maternal lineages including O. barthii, O. longistaminata and the remainder of O. glumaepatula were phylogenetically close to each other and carried low genetic diversity. They were separated from independent lineages, suggesting that they had diverged from a single ancestral maternal lineage, but diverged later to keep gene flow within respective species, as SSR compositions suggested. Genetic relationships among AA genome species indicate how these species have evolved and become distributed across four continents.