Is there a distinct area within the human visual system that has a preferential response to numerals, as there is for faces, words, or scenes? We addressed this question using intracranial electrophysiological recordings and observed a significantly higher response in the high-frequency broadband range (high γ, 65-150 Hz) to visually presented numerals, compared with morphologically similar (i.e., letters and false fonts) or semantically and phonologically similar stimuli (i.e., number words and non-number words). Anatomically, this preferential response was consistently localized in the inferior temporal gyrus and anterior to the temporo-occipital incisure. This region lies within or close to the fMRI signal-dropout zone produced by the nearby auditory canal and venous sinus artifacts, an observation that may account for negative findings in previous fMRI studies of preferential response to numerals. Because visual numerals are culturally dependent symbols that are only learned through education, our novel finding of anatomically localized preferential response to such symbols provides a new example of acquired category-specific responses in the human visual system.