Ultrafast dynamics of Dirac fermions near the Fermi surface in monolayer and multilayer graphene are revealed using optical pump mid-infrared probe spectroscopy. The energy loss rate of Dirac fermions is also determined via energy-resolved transient transmissivity spectra, which is significantly suppressed as the number of layers in graphene increases.