We explored the neural basis for spatial color contrast (red looks redder surrounded by green) and temporal color contrast (red looks redder if preceded by green) in primary visual cortex (V1) of the alert macaque. Using pairs of stimuli, we found a subset of neurons that gave stronger responses to sequences of red and green spots and stronger responses to adjacent red and green spots. These cells combined their cone inputs linearly: for a red-ON-center cell, the sum of the OFF response to green and the ON response to red predicted the peak response to red preceded by green. These 'color' cells, which could underlie hue discrimination because they show cone opponency, could mediate spatial and temporal color contrast. In contrast, the majority of cortical cells, which do not show overt cone opponency but which are often orientation tuned and/or direction selective, are by themselves incapable of mediating hue discrimination. The remarkable degree of specialization shown by cells in V1, especially that of the double-opponent color cells, is discussed.