The visual image formed on the retina represents an amalgam of visual scene properties, including the reflectances of surfaces, their relative positions, and the type of illumination. The challenge facing the visual system is to extract the "meaning" of the image by decomposing it into its environmental causes. For each local region of the image, that extraction of meaning is only possible if information from other regions is taken into account. Of particular importance is a set of image cues revealing surface occlusion and/or lighting conditions. These information-rich cues direct the perceptual interpretation of other more ambiguous image regions. This context-dependent transformation from image to perception has profound-but frequently under-appreciated-implications for neurophysiological studies of visual processing: To demonstrate that neuronal responses are correlated with perception of visual scene properties, rather than visual image features, neuronal sensitivity must be assessed in varied contexts that differentially influence perceptual interpretation. We review a number of recent studies that have used this context-based approach to explore the neuronal bases of visual scene perception.