The trait of low cadmium (Cd) accumulation in brown rice (Oryza sativa) is important for food safety. An effective way to reduce Cd accumulation in the grain is to control Cd transfer from the roots to the shoots. Here, we investigated genotypic variation in the shoot Cd concentration among 146 accessions from a rice core collection and performed a quantitative trait locus (QTL) analysis to determine the loci controlling shoot Cd accumulation. Furthermore, we physiologically characterized the two accessions used for QTL analysis. Large genotypic variation (13-fold) in the shoot Cd concentration was found. A major QTL was detected on chromosome 11 using a F2 population derived from Badari Dhan (a high-Cd accession) and Shwe War (a low-Cd accession). This QTL explained 16.1% of the phenotypic variation in Cd accumulation. Furthermore, this QTL was confirmed by analysis of advanced progeny. Physiological studies showed that Badari Dhan and Shwe War did not differ in uptake of Cd by the roots, but differed greatly in the translocation of Cd from the roots to the shoots. Taken together, our findings suggest that the major QTL detected is responsible for the translocation of Cd from the roots to the shoots.