We investigate the transient photoconductivity of graphene at various gate-tuned carrier densities by optical-pump terahertz-probe spectroscopy. We demonstrate that graphene exhibits semiconducting positive photoconductivity near zero carrier density, which crosses over to metallic negative photoconductivity at high carrier density. These observations can be accounted for by the interplay between photoinduced changes of both the Drude weight and carrier scattering rate. Our findings provide a complete picture to explain the opposite photoconductivity behavior reported in (undoped) graphene grown epitaxially and (doped) graphene grown by chemical vapor deposition. Notably, we observe nonmonotonic fluence dependence of the photoconductivity at low carrier density. This behavior reveals the nonmonotonic temperature dependence of the Drude weight in graphene, a unique property of two-dimensional massless Dirac fermions.