Functional neuroimaging studies have demonstrated preferential involvement of bilateral prefrontal cortex during episodic memory encoding and retrieval. The aim of the present study is to address the question whether left prefrontal model for encoding holds when highly non-verbal material is used, and which region of the brain is critically related to successful retrieval. To do this, seven normal subjects were investigated using functional magnetic resonance imaging (fMRI) during encoding and retrieval of word and checkerboard pattern. Our results revealed that word encoding activated the left prefrontal cortices and right cerebellum, whereas pattern encoding activated the bilateral middle frontal gyrus, superior parietal lobule, premotor area, and occipital visual cortex. Word-specific activation was found in the ventral prefrontal cortices, and pattern-specific activation located in the right dorsal prefrontal cortex. Conjunction analysis during encoding of word and pattern showed that activity in the left dorsal prefrontal cortex and the right cerebellum might relate to common neural network for encoding regardless of the type of material. Finally, the present study demonstrates strong association between the left ventral prefrontal cortex and retrieval success for word. The evidence, that both encoding and retrieval of words activated the left ventral prefrontal cortex, indicates that this area is involved in active and strategic operation of the mnemonic representation. A lack of the right prefrontal activation during retrieval was interpreted as that activity in this region might relate to retrieval effort rather than success.