The pinniped family Otariidae (sea lions and fur seals) is composed of 7 extant genera with 14 species. They are mainly distributed in the Southern Hemisphere, but the fossil record is only known from the Northern Hemisphere until Pliocene. To clarify the biological and zoogeographical events during their evolution, it is necessary to reconstruct a robust phylogenetic tree. However, phylogenetic relationships among otariids continue to be controversial, except for the basal position of the northern fur seal among the extant otariids. We reconstructed phylogenetic trees of otariids based on mitochondrial genomes and multiple nuclear genes (IRBP and type I STS markers). The monophyly of the otariids including both sea lions and fur seals in the Southern Hemisphere was strongly supported by both the mitochondrial and nuclear evidence. We propose a novel evolutionary and dispersal scenario of otariids based on this phylogenetic hypothesis, estimated divergence times, and fossil records. According to our results, the center of origin of the southern otariids is hypothesized to be the eastern South Pacific along the west coast of South America.