Attention is often regarded as a mechanism by which attended objects become perceptually more salient, akin to increasing their contrast. We demonstrate that attention is better described as a mechanism by which task relevant information impacts on ongoing processing, while excluding task irrelevant information. We asked subjects to judge the orientation of a target relative to a reference, in a single and dual task setting. The target orientation percept was systematically influenced by the presentation of prior spatio-temporal context. We found that the sign of the context influence depended on target contrast, but its strength depended on the level of attention devoted to the task. Thus the effects of attention and contrast were fundamentally different; contrast influenced the sign of contextual interactions, while attention suppressed these interactions irrespective of their sign.