Root exudates play an important role in the phytoremediation of soils contaminated by organic pollutants, but how root exudate components affect the remediation process is not well understood. In this study, we explored the effects and mechanisms of the major root exudates, including glucose, organic acids, and serine, in the rhizoremediation of pyrene-contaminated soil. The results showed that glucose increased the degradation of pyrene (54.3 ± 1.7%) most significantly compared to the organic acids (45.5 ± 2.5%) and serine (43.2 ± 0.1%). Glucose could significantly facilitate the removal of pyrene in soil through promoting dehydrogenase activity indicated by a positive correlation between the removal efficiency of pyrene and the soil dehydrogenase activity (p < 0.01). Furthermore, root exudates were able to change soil microbial community, particularly the bacterial taxonomic composition, thereby affecting the biodegradation of pyrene. Glucose could alter soil microbial community and enhance the amount of Mycobacterium markedly, which is dominant in the degradation of pyrene. These findings provide insights into the mechanisms by which root exudates enhance the degradation of organic contaminants and advance our understanding of the micro-processes involved in rhizoremediation.