AFF1 and AFF4 belong to the AFF (AF4/FMR2) family of proteins, which function as scaffolding proteins linking two different transcription elongation factors, positive elongation factor b (P-TEFb) and ELL1/2, in super elongation complexes (SECs). Both AFF1 and AFF4 regulate gene transcription through elongation and chromatin remodeling. However, their function in the osteogenic differentiation of mesenchymal stem cells (MSCs) is unknown. In this study, we show that small interfering RNA (siRNA)-mediated depletion of AFF1 in human MSCs leads to increased alkaline phosphatase (ALP) activity, enhanced mineralization and upregulated expression of osteogenic-related genes. On the contrary, depletion of AFF4 significantly inhibits the osteogenic potential of MSCs. In addition, we confirm that overexpression of AFF1 and AFF4 differentially affects osteogenic differentiation in vitro and MSC-mediated bone formation in vivo. Mechanistically, we find that AFF1 regulates the expression of DKK1 via binding to its promoter region. Depletion of DKK1 in HA-AFF1-overexpressing MSCs abrogates the impairment of osteogenic differentiation. Moreover, we detect that AFF4 is enriched in the promoter region of ID1. AFF4 knockdown blunts the BRE luciferase activity, SP7 expression and ALP activity induced by BMP2 treatment. In conclusion, our data indicate that AFF1 and AFF4 differentially regulate the osteogenic differentiation of human MSCs.