Visual systems break scenes down into individual features, processed in distinct channels, and then selectively recombine those features according to the demands of particular behavioral tasks. In primates, for example, there are distinct pathways for motion and form processing. While form vision utilizes color information, motion pathways receive input from only a subset of cone photoreceptors and are generally colorblind. To explore the link between early channeling of visual information and behavioral output across vertebrate species, we measured the chromatic inputs to the optomotor response of larval zebrafish. Using cone-isolating gratings, we found that there is a strong input from both red and green cones but not short-wavelength cones, which nevertheless do contribute to another behavior, phototaxis. Using a motion-nulling method, we measured precisely the input strength of gratings that stimulated cones in combination. The fish do not respond to gratings that stimulate different cone types out of phase, but have an enhanced response when the cones are stimulated together. This shows that red and green cone signals are pooled at a stage before motion detection. Since the two cone inputs are combined into a single 'luminance' channel, the response to sinusoidal gratings is colorblind. However, we also find that the relative contributions of the two cones at isoluminance varies with spatial frequency. Therefore, natural stimuli, which contain a mixture of spatial frequencies, are likely to be visible regardless of their chromatic composition.