Functional neuroimaging is uniquely placed to examine the dynamic nature of normal human memory, the distributed brain networks that support it, and how they are modulated. Memory has traditionally been classified into context-specific memories personally experienced ("episodic memory") and impersonal non-context-specific memories ("semantic memory"). However, we suggest that another useful distinction is whether events are personally relevant or not. Typically the factors of personal relevance and temporal context are confounded, and it is as yet not clear the precise influence of either on how memories are stored or retrieved. Here we focus on the retrieval of real-world memories unconfounding personal relevance and temporal context during positron emission tomography (PET) scanning. Memories differed along two dimensions: They were personally relevant (or not) and had temporal specificity (or not). Recollection of each of the resultant four memory subtypes-autobiographical events, public events, autobiographical facts, and general knowledge-was associated with activation of a common network of brain regions. Within this system, however, enhanced activity was observed for retrieval of personally relevant, time-specific memories in left hippocampus, medial prefrontal cortex, and left temporal pole. Bilateral temporoparietal junctions were activated preferentially for personal memories, regardless of time specificity. Finally, left parahippocampal gyrus, left anterolateral temporal cortex, and posterior cingulate cortex were involved in memory retrieval irrespective of person or time. Our findings suggest that specializations in memory retrieval result from associations between subsets of regions within a common network. We believe that these findings throw new light on an old debate surrounding episodic and declarative theories of memory and the precise involvement of the hippocampus.