Hematite has been receiving increasing attention for its application in photoelectrochemical (PEC) water oxidation but usually exhibits poor efficiency. We fabricated a stable gradient-structured FeOx(PO4)y layer on hematite by diffusively incorporating phosphate onto the surface layer of hematite films at a low temperature. X-ray photoelectron spectroscopy depth profile and Fe K-edge grazing-incidence X-ray absorption near-edge structure and extended X-ray absorption fine structure analysis demonstrated the formation of a ∼50 nm overlayer with a gradient phosphorus distribution and structural evolution from the outer surface to the depth. The as-prepared photoanodes showed highly improved PEC water oxidation performance. Up to 8.5-fold enhancement in the photocurrent density at 1.23 V versus reversible hydrogen electrode was achieved relative to the pristine anode. This strategy is applicable for hematite photoanodes prepared by different methods and with different morphologies and structures. The improvement in the water oxidation activity is mainly attributed to the enhanced separation of photogenerated electron-hole pairs, which is derived from the increased hole diffusion length in the gradient-structured overlayer. This work develops a simple and universal method to boost the PEC water oxidation efficiency with versatile hematite photoanodes.