Language comprehension relies on processing of context. Working memory (WM) evoked by linguistic cues for spatial and nonspatial aspects of a visual scene was investigated by correlating fMRI BOLD signal (or 'activation') with reaction times (RTs). Subjects were asked to indicate either the relative positions or ages of people or objects (referenced by the personal pronouns "he/she/it") in a previously shown image. Good performers of a particular task showed shorter RTs than poor performers. Task-specific activation that is greater in good performers than poor ones is taken to indicate involvement of a given region in performance of the task. Our results indicate that dorsoposterior precuneus supports spatial WM during linguistic processing while a network of areas including the caudate support nonspatial WM in categorization of age. We argue that within-subjects variation of RTs across trials reflects effort. Good performers have higher activity in precuneus as a function of effort compared to poor performers during the spatial task, whereas the opposite is found for the nonspatial task, providing further evidence for specifically spatial WM in dorsoposterior precuneus. Task-independent performance-related modulations of activity were found in Broca's area and amygdala. Broca's area activity increased with effort in both tasks, with a greater increase in good performers than in poor performers, consistent with the region's general role in verbal WM. By contrast, activation in amygdala decreased with effort, with a greater decrease in good performers. We take this deactivation to reflect performance-mediating emotional control. These findings indicate that multiple parallel memory systems are available during language processing, appropriate for different tasks, with performance reflecting which system is selected trial-by-trial and subject-by-subject.