Research on two-dimensional designer optical structures, or metasurfaces, has mainly focused on controlling the wavefronts of light propagating in free space. Here, we show that gradient metasurface structures consisting of phased arrays of plasmonic or dielectric nanoantennas can be used to control guided waves via strong optical scattering at subwavelength intervals. Based on this design principle, we experimentally demonstrate waveguide mode converters, polarization rotators and waveguide devices supporting asymmetric optical power transmission. We also demonstrate all-dielectric on-chip polarization rotators based on phased arrays of Mie resonators with negligible insertion losses. Our gradient metasurfaces can enable small-footprint, broadband and low-loss photonic integrated devices.