Previously, we discovered that the broadband cells in the two magnocellular (large cell) layers of the monkey lateral geniculate nucleus (LGN) are much more sensitive to luminance contrast than are the color-sensitive cells in the four parvocellular (small cell) layers. We now report that this large difference in contrast sensitivity is due not to LGN circuitry but to differences in sensitivity of the retinal ganglion cells that provide excitatory synaptic input to the LGN neurons. This means that the parallel analysis of color and luminance in the visual scene begins in the retina, probably at a retinal site distal to the ganglion cells.