A three-level hierarchical calcium phosphate/collagen/hydroxyapatite (CaP/Col/HAp) scaffold for bone tissue engineering was developed using biomimetic synthesis. Porous CaP ceramics were first prepared as substrate materials to mimic the porous bone structure. A second-level Col network was then composited into porous CaP ceramics by vacuum infusion. Finally, a third-level HAp layer was achieved by biomimetic mineralization. The three-level hierarchical biomimetic scaffold was characterized using scanning electron microscopy, energy-dispersive x-ray spectra, x-ray diffraction and Fourier transform infrared spectroscopy, and the mechanical properties of the scaffold were evaluated using dynamic mechanical analysis. The results show that this scaffold exhibits a similar structure and composition to natural bone tissues. Furthermore, this three-level hierarchical biomimetic scaffold showed enhanced mechanical strength compared with pure porous CaP scaffolds. The biocompatibility and osteoinductivity of the biomimetic scaffolds were evaluated using in vitro and in vivo tests. Cell culture results indicated the good biocompatibility of this biomimetic scaffold. Faster and increased bone formation was observed in these scaffolds following a six-month implantation in the dorsal muscles of rabbits, indicating that this biomimetic scaffold exhibits better osteoinductivity than common CaP scaffolds.