The primary biochemical reaction of purple acid phosphatases (PAP) is to catalyze the hydrolysis of phosphate esters and anhydrides. However, the soybean GmPAP3 gene expression is induced by NaCl, osmotic, and oxidative treatments, indicating a possible role of PAP in abiotic stress responses. Confocal and electron microscopic studies demonstrated that GmPAP3 protein is mainly localized in mitochondria, a primary site for reactive oxygen species (ROS) production. When subjected to NaCl and polyethylene glycol (PEG) treatments, ectopic expression of GmPAP3 in transgenic tobacco BY-2 cells mimicked the protective effects exhibited by the antioxidant ascorbic acid: increase in the percentage of cells with active mitochondria; reduction in the percentage of dead cells; and reduced accumulation of ROS. In addition, when GmPAP3 transgenic Arabidopsis thaliana seedlings were subjected to NaCl, PEG, and paraquat (PQ) treatments, the percentage of root elongation was significantly higher than the wild type. Furthermore, PQ-induced lipid peroxidation in these transgenic seedlings was also reduced. In summary, the mitochondrial localized GmPAP3 may play a role in stress tolerance by enhancing ROS scavenging.