Specular image structure modulates the perception of three-dimensional shape

Abstract

Retinal images are produced by interactions between a surface's 3D shape, material properties, and surrounding light field. In order to recover the 3D geometry of a surface, the visual system must somehow separate aspects of image structure generated by a surface's shape from structure generated by its material properties or the light field in which it is embedded. Attributing image structure to the wrong physical source would cause the visual system to interpret changes in one physical property (such as reflectance) as changes in another (such as shape). Many previous studies have shown that the visual system does not conflate image structure generated by specular reflectance with 3D shape, but they did not assess the physical conditions where it would be computationally most difficult to disentangle these different sources of image structure. Here, we show that varying the specular roughness and curvature of surfaces embedded in natural light fields can strongly modulate perceived shape. Despite the complexity of these interactions, we show how an image's gradient structure mediates its interpretation as a specular reflection or a change in 3D shape. Our findings provide a coherent explanation of when and why specular reflections impact perceived shape and reveal how the static surface properties, simplified light fields, and experimental methods used in previous studies may explain their inconsistent results.