This paper considers the temporal processing of complex sounds relevant to musical analysis. Functional imaging studies, using positron emission tomography (PET), functional magnetic resonance imaging (fMRI), and magnetoencephalography (MEG), and the psychophysical assessment of patients with lesions allow two different approaches to this. Functional imaging allows the determination of structures normally involved in temporal analysis, while patient studies allow inference about the necessary structures for temporal analysis. Both approaches suggest a hierarchal organization in the brain corresponding to the processing of music. The features of individual notes are analyzed in the pathway up to and including the auditory cortices, while higher-order patterns formed by those features are analyzed by distributed networks in the temporal lobe and frontal lobes distinct from the auditory cortices.