By recalling gustatory memories, it is possible to generate vivid gustatory perceptions in the absence of gustatory inputs. This gustatory image influences our gustatory processing. However, the mechanism of the "top-down" modulation of gustatory perception in the human is still unclear. Our findings propose a new perspective on the neural basis of gustatory processing. Although gustatory imagery and gustatory perception shared common parts of neural substrates, there was an asymmetrical topography of activation in the insula: the left insula was predominantly activated by gustatory imagery tasks. In addition, the middle and superior frontal gyri were not activated by gustatory perception but they participated in the generation of gustatory hallucinations. These regions in the frontal cortex may mediate the "top-down" control of retrieving gustatory information from the storage of long-term memories.