With the worldwide rising need of severe bone defect treatment, the development of available bone substitutes with optimal mechanical strength, sustained drug release, cell affinity and osteoinductivity remains a great challenge. In this study, an rhBMP-2 loaded polymer-coated mesoporous bioactive glass (MBG) composite scaffold was developed. The uncrosslinked poly(glycerol sebacate) (PGS) or PEGylated PGS (PEGS) coating modification had enhanced the mechanical strength of the composite scaffold, solved the brittleness problem of the MBG matrix, increased the cell affinity of the material surface, and diminished the initial burst release of rhBMP-2 from mesopores of MBG. The results indicated that the PGS coating promoted the proliferation of rat bone marrow stem cells (rBMSCs), while the PEGS coating exhibited an enhancement in the osteogenic differentiation of rBMSCs. The in vivo ectopic bone formation results provide strong evidence that the rhBMP-2-loaded MBG/PEGS composite scaffolds exhibited a rapid bone forming capacity and might yield extraordinary achievements in the field of bone tissue engineering. The design considerations can be extended to other artificial scaffolds and are expected to provide new thoughts on the development of future tissue engineering materials.