In the present study, the affective impact of earcons on stimulus classification is investigated. We show, using a picture-categorization task, that the affective connotation of earcons in major and minor mode (representing positive and negative valence, respectively) can be congruent or incongruent with response valence. Twenty participants classified pictures of animals and instruments in 256 trials, using positive and negative Yes or No responses. Together with the pictures, either a chord in major mode or minor mode was played. The affective valence of the chords either did or did not match the valence of responses. Response-time latencies show congruency effects of the matching and non matching sound and response valences, indicating that it is important to carefully investigate human-computer interfaces for potential affective congruency effects, as these can either facilitate or inhibit user performance.