Rice (Oryza sativa L.) is an important staple crop. The exploitation of the great heterosis that exists in the inter-subspecific crosses between the indica and japonica rice has long been considered as a promising way to increase the yield potential. However, the male and female sterility frequently occurred in the inter-subspecific hybrids hampered the utilization of the heterosis. Here we report that the inter-subspecific hybrid sterility in rice is mainly affected by the genes at Sb, Sc, Sd and Se loci for F1 male sterility and the gene at S5 locus for F1 female sterility. The indica-compatible japonica lines (ICJLs) developed by pyramiding the indica allele (S-i) at Sb, Sc, Sd and Se loci and the neutral allele (S-n) at S5 locus in japonica genetic background through marker-assisted selection are compatible with indica rice in pollen fertility and in spikelet fertility. These results showed a great promise of overcoming the inter-subspecific hybrid sterility and exploiting the heterosis by developing ICJLs.