When an object casts a shadow on a background surface, both the offset of the shadow and the blur of its penumbra are potential cues to the distance between the object and the background. However, the shadow offset and blur are also affected by the direction and angular extent of the light source and these are often unknown. This means that the observer must make some assumptions about the illumination, the expected distribution of depth, or the relation between offset and depth in order to use shadows to make distance judgments. Here, we measure human judgments of perceived depth over a range of shadow offsets, blurs, and lighting directions to gain insight into this internal model. We find that distance judgments are relatively unaffected by blur or light direction, whereas the shadow offset has a strong and linear effect. The data are consistent with two models, a generic shadow-to-depth model and a Bayesian model.