Recent delayed matching studies have demonstrated that maintaining trial-unique stimuli in working memory modulates activity in temporal lobe structures. In contrast, most previous studies that focused on the role of the prefrontal cortex (PFC) used familiar stimuli. We combined fMRI with a delayed-match-to-sample (DMS) task in humans that allowed us to manipulate stimulus pre-exposure (trial-unique vs. familiar objects) and stimulus domain (object vs. location). A visually guided saccade task was used to localize the frontal eye fields (FEF). We addressed two questions: First, we examined whether delay-period activity within PFC regions was more strongly engaged when stimuli were familiar (pre-exposed) than when they were not seen previously (trial-unique). Second, we examined the role of regions within the PFC in object vs. location working memory. Subjects were instructed to remember one stimulus domain while ignoring the other over an 8-s delay period. Object-specific delay-period activity was greatest in the posterior orbitofrontal cortex (OFC) bilaterally, and was stronger for familiar than trial-unique objects. In addition, consistent with previous findings, right posterior superior frontal sulcus, and the FEF were specifically active during the delay period of the location DMS task. These activations outside FEF were not related to saccadic eye movements. In contrast to previous reports, object-specific delay activity was more prominent in the posterior OFC than in the ventrolateral PFC, and was found to be greater for familiar than for trial-unique objects. These results suggest a critical role for the orbitofrontal cortex for maintaining object information in working memory.