No matter how hard subjects concentrate on a task, their minds wander (Raichle et al., 2001; Buckner et al., 2008; Christoff et al., 2009; Killingsworth and Gilbert, 2010). Internal fluctuations cannot be measured behaviorally or from conventional neurophysiological measures, so their effects on performance have been difficult to study. Previously, we measured fluctuations in visual attention using the responses of populations of simultaneously recorded neurons in macaque visual cortex (Cohen and Maunsell, 2010). Here, we use this ability to investigate how attentional fluctuations affect performance. We found that attentional fluctuations have large and complex effects on performance, the sign of which depends on the difficulty of the perceptual judgment. As expected, attention greatly improves the detection of subtle changes in a stimulus. Surprisingly, we found that attending too strongly to a particular stimulus impairs the ability to notice when that stimulus changes dramatically. Our results suggest that all previously reported measures of behavioral performance should be viewed as amalgamations of different attentional states, whether or not those studies specifically addressed attention.