Decoding verbal and nonverbal emotional expressions is an important part of speech communication. Although various studies have tried to specify the brain regions that underlie different emotions conveyed in speech, few studies have aimed to specify the time course of emotional speech decoding. We used event-related potentials to determine when emotional speech is first differentiated from neutral speech. Participants engaged in an implicit emotional processing task (probe verification) while listening to emotional sentences spoken by a female and a male speaker. Independent of speaker voice, emotional sentences could be differentiated from neutral sentences as early as 200 ms after sentence onset (P200), suggesting rapid emotional decoding.