Reducing cadmium (Cd) accumulation in rice grain is an important issue for human health. The aim of this study was to manipulate both expression and tissue localization of OsHMA3, a tonoplast-localized Cd transporter, in the roots by expressing it under the control of the OsHMA2 promoter, which shows high expression in different organs including roots, nodes, and shoots. In two independent transgenic lines, the expression of OsHMA3 was significantly enhanced in all organs compared with non-transgenic rice. Furthermore, OsHMA3 protein was detected in the root pericycle cells and phloem region of both the diffuse vascular bundle and the enlarged vascular bundle of the nodes. At the vegetative stage, the Cd concentration in the shoots and xylem sap of the transgenic rice was significantly decreased, but that of the whole roots and root cell sap was increased. At the reproductive stage, the concentration of Cd, but not other essential metals, in the brown rice of transgenic lines was decreased to less than one-tenth that of the non-transgenic rice. These results indicate that expression of OsHMA3 under the control of the OsHMA2 promoter can effectively reduce Cd accumulation in rice grain through sequestering more Cd into the vacuoles of various tissues.