In this paper, we theoretically design a dual-band graphene-based terahertz (THz) absorber combining the magnetic resonance with a THz cold mirror without any metallic loss. The absorption spectrum of the all-dielectric THz absorber can be actively manipulated after fabrication due to the tunable conductivity of graphene. After delicate optimization, two ultra-narrow absorption peaks are achieved with respective full width at half maximum (FWHM) of 0.0272 THz and 0.0424 THz. Also, we investigate the effect of geometric parameters on the absorption performance. Coupled mode theory (CMT) is conducted on the dual-band spectrum as an analytic method to confirm the validity of numerical results. Furthermore, physical mechanism is deeply revealed with magnetic and electric field distributions, which demonstrate a totally different principle with traditional plasmonic absorber. Our research provides a significant design guide for developing tunable multi-resonant THz devices based on all-dielectric configuration.