The projected pattern of retinal-image motion supplies the human visual system with valuable information about properties of the three-dimensional environment. How well three-dimensional properties can be recovered depends both on the accuracy with which the early motion system estimates retinal motion, and on the way later processes interpret this retinal motion. Here we combine both early and late stages of the computational process to account for the hitherto puzzling phenomenon of systematic biases in three-dimensional shape perception. We present data showing how the perceived depth of a hinged plane ('an open book') can be systematically biased by the extent over which it rotates. We then present a Bayesian model that combines early measurement noise with geometric reconstruction of the three-dimensional scene. Although this model has no in-built bias towards particular three-dimensional shapes, it accounts for the data well. Our analysis suggests that the biases stem largely from the geometric constraints imposed on what three-dimensional scenes are compatible with the (noisy) early motion measurements. Given these findings, we suggest that the visual system may act as an optimal estimator of three-dimensional structure-from-motion.