While reading this text, your eyes jump from word to word. Yet you are unaware of the motion this causes on your retina; the brain somehow compensates for these displacements and creates a stable percept of the world. This compensation is not perfect; perisaccadically, perceptual space is distorted. We show that this distortion can be traced to a representation of retinal position in the medial temporal and medial superior temporal areas. These cells accurately represent retinal position during fixation, but perisaccadically, the same cells distort the representation of space. The time course and magnitude of this distortion are similar to the mislocalization found psychophysically in humans. This challenges the assumption in many psychophysical studies that the perisaccadic retinal position signal is veridical.