Physiological, computational, and psychophysical studies of stereopsis have assumed that the perceived surface structure of binocularly viewed images is primarily specified by the pattern of binocular disparities in the two eyes' views. A novel set of stereoscopic phenomena are reported that demonstrate the insufficiency of this view. It is shown that the visual system computes the contrast relationships along depth discontinuities to infer the depth, lightness, and opacity of stereoscopically viewed surfaces. A novel theoretical framework is introduced to explain these results. It is argued that the visual system contains mechanisms that enforce two principles of scene interpretation: a generic view principle that determines qualitative scene geometry, and anchoring principles that determine how image data are quantitatively partitioned between different surface attributes.