Previous studies have shown that even in the context of fairly easy selection tasks, as is the case in a pop-out task, selection of the pop-out stimulus can be sped up (in terms of eye movements) when the target-defining feature repeats across trials. Here, we show that selection of a pop-out target can actually be delayed (in terms of saccadic latencies) and made less accurate (in terms of saccade accuracy) when the target-defining feature has recently been associated with distractor status. This effect was observed even though participants' task was to fixate color oddballs (when present) and simply press a button when their eyes reached the target to advance to the next trial. Importantly, the inter-trial effect was also observed in response time (time to advance to the next trial). In contrast, this response time effect was completely eliminated in a second experiment when eye movements were eliminated from the task. That is, when participants still had to press a button to advance to the next trial when an oddball target was present in the display (an oddball detection task experiment). This pattern of results closely links the "need for selection" in a task to the presence of an inter-trial bias of attention (and eye movements) in pop-out search.