Despite frequent saccadic gaze shifts we perceive the surrounding visual world as stable. It has been proposed that the brain uses extraretinal eye position signals to cancel out saccade-induced retinal image motion. Nevertheless, stimuli flashed briefly around the onset of a saccade are grossly mislocalized, resulting in a shift and, under certain conditions, an additional compression of visual space. Perisaccadic mislocalization has been related to a spatio-temporal misalignment of an extraretinal eye position signal with the corresponding saccade. Here, we investigated perceptual mislocalization of human observers both in saccade and fixation conditions. In the latter conditions, the retinal stimulation during saccadic eye movements was simulated by a fast saccade-like shift of the stimulus display. We show that the spatio-temporal pattern of both the shift and compression components of perceptual mislocalization can be surprisingly similar before real and simulated saccades. Our findings suggest that the full pattern of perisaccadic mislocalization can also occur in conditions which are unlikely to involve changes of an extraretinal eye position signal. Instead, we suggest that, under the conditions of our experiments, the arising difficulty to establish a stable percept of a briefly flashed stimulus within a given visual reference frame yields mislocalizations before fast retinal image motion. The availability of visual references appears to exert a major influence on the relative contributions of shift and compression components to mislocalization across the visual field.