Self-esteem has been associated with neural responses to self-reflection and attitude toward social feedback but in different brain regions. The distinct associations might arise from different tasks or task-related attitudes in the previous studies. The current study aimed to clarify these by investigating the association between self-esteem and neural responses to evaluation of one's own personality traits and of others' opinion about one's own personality traits. We scanned 25 college students using functional MRI during evaluation of oneself or evaluation of social feedback. Trait self-esteem was measured using the Rosenberg self-esteem scale after scanning. Whole-brain regression analyses revealed that trait self-esteem was associated with the bilateral orbitofrontal activity during evaluation of one's own positive traits but with activities in the medial prefrontal cortex, posterior cingulate, and occipital cortices during evaluation of positive social feedback. Our findings suggest that trait self-esteem modulates the degree of both affective processes in the orbitofrontal cortex during self-reflection and cognitive processes in the medial prefrontal cortex during evaluation of social feedback.