Melanopsin-expressing intrinsically photosensitive retinal ganglion cells (ipRGCs) combine inputs from outer-retinal rod/cone photoreceptors with their intrinsic phototransduction machinery to drive a wide range of so-called non-image-forming (NIF) responses to light. Defining the contribution of each photoreceptor class to evoked responses is vital for determining the degree to which our sensory capabilities depend on melanopsin and for optimizing NIF responses to benefit human health. We addressed this problem by recording electrophysiological responses in the mouse pretectal olivary nucleus (PON) (a target of ipRGCs and origin of the pupil light reflex) to a range of gradual and abrupt changes in light intensity. Dim stimuli drove minimal changes in PON activity, suggesting that rods contribute little under these conditions. To separate cone from melanopsin influences, we compared responses to short (460 nm) and longer (600/655 nm) wavelengths in mice carrying a red shifted cone population (Opn1mw®) or lacking melanopsin (Opn4⁻/⁻). Our data reveal a surprising difference in the quality of information available from medium- and short-wavelength-sensitive cones. The majority cone population (responsive to 600/655 nm) supported only transient changes in firing and responses to relatively sudden changes in light intensity. In contrast, cones uniquely sensitive to the shorter wavelength (S-cones) were better able to drive responses to gradual changes in illuminance, contributed a distinct off inhibition, and at least partially recapitulated the ability of melanopsin to sustain responses under continuous illumination. These data reveal a new role for S-cones unrelated to color vision and suggest renewed consideration of cone contributions to NIF vision at shorter wavelengths.