We report on the design and fabrication of a Fe3O4 core-mesoporous silica nanoparticle shell (Fe3O4@MSNs)-based mitochondria-targeting drug nanocarrier. A guanidinium derivative (GA) was conjugated onto the Fe3O4@MSNs as the mitochondria-targeting ligand. The fabrication of the Fe3O4@MSNs and their functionalization with GA were carried out by the sol-gel polymerization of alkoxysilane groups. Doxorubicin (DOX), an anti-cancer drug, was loaded into the pores of a GA-attached Fe3O4@MSNs due to both its anti-cancer properties and to allow for the fluorescent visualization of the nanocarriers. The selective and efficient mitochondria-targeting ability of a DOX-loaded GA-Fe3O4@MSNs (DOX/GA-Fe3O4@MSNs) was demonstrated by a co-localization study, transmission electron microscopy, and a fluorometric analysis on isolated mitochondria. It was found that the DOX/GA-Fe3O4@MSNs selectively accumulated into mitochondria within only five minutes; to the best of our knowledge, this is the shortest accumulation time reported for mitochondria targeting systems. Moreover, 2.6 times higher amount of DOX was accumulated in mitochondria by DOX/GA-Fe3O4@MSNs than by DOX/TPP-Fe3O4@MSNs. A cell viability assay indicated that the DOX/GA-Fe3O4@MSNs have high cytotoxicity to cancer cells, whereas the GA-Fe3O4@MSNs without DOX are non-cytotoxic; this indicates that the DOX/GA-Fe3O4@MSNs have great potential for use as biocompatible and effective mitochondria-targeting nanocarriers for cancer therapy.