Research on human action has extensively covered controlled and automatic processes in the transformation of stimulus information into motor action, and how conflict between both types of processes is solved. However, the question of how automatic stimulus-response (S-R) translation per se depends on top-down control states remains unanswered. The present study addressed this issue by manipulating top-down control state (instructed S-R mapping) and automatic bottom-up processing (retrieval of S-R memory traces) independently from each other. Using a color/shape task-switching paradigm, we compared cross-talk triggered by distractor stimuli, for which the instructed S-R mapping and the S-R associations compiled at the beginning of the experiment matched, with the cross-talk triggered by distractor stimuli, for which (re-)instructed mapping and compiled S-R associations did not match. We show that the latter distractors do not yield any cross-talk in RTs and even reversed cross-talk in error rates, demonstrating that automatic S-R retrieval is modulated by top-down control states.