We aimed to identify neuroanatomical regions associated with deficits to the graphemic buffer, a working memory component of the spelling system that holds the sequence of letter identities during production. We evaluated 331 patients with left hemisphere ischemic stroke with various spelling tests and magnetic resonance diffusion-weighted imaging and perfusion-weighted imaging, within 48 hours of stroke onset. A voxel-wise statistical map showed that ischemia in voxels in posterior and inferior frontal and parietal cortex, subcortical white matter underlying prefrontal cortex, lateral occipital gyrus, or caudate was associated with impairment in maintaining the sequence of letter identities while spelling.