Are speech-specific processes localized in dedicated cortical regions or do they emerge from developmental plasticity in the connections among non-dedicated regions? Here we claim that all the brain regions activated by the processing of auditory speech can be re-classified according to whether they respond to non-verbal environmental sounds, pitch changes, unfamiliar melodies, or conceptual processes. We therefore argue that speech-specific processing emerges from differential demands on auditory and conceptual processes that are shared by speech and non-speech stimuli. This has implications for domain- vs. process-specific cognitive models, and for the relative importance of segregation and integration in functional anatomy.