Given an estimate of the binocular disparity between a pair of points and an estimate of the viewing distance, or knowledge of eye position, it should be possible to obtain an estimate of their depth separation. Here we show that, when points are arranged in different vertical geometric configurations across two intervals, many observers find this task difficult. Those who can do the task tend to perceive the depth interval in one configuration as very different from depth in the other configuration. We explore two plausible explanations for this effect. The first is the tilt of the empirical vertical horopter: Points perceived along an apparently vertical line correspond to a physical line of points tilted backwards in space. Second, the eyes can rotate in response to a particular stimulus. Without compensation for this rotation, biases in depth perception would result. We measured cyclovergence indirectly, using a standard psychophysical task, while observers viewed our depth configuration. Biases predicted from error due either to cyclovergence or to the tilted vertical horopter were not consistent with the depth configuration results. Our data suggest that, even for the simplest scenes, we do not have ready access to metric depth from binocular disparity.