Many neurocognitive models of anxiety emphasize the importance of a hyper-responsive threat-detection system centered on the amygdala, with recent accounts incorporating a role for prefrontal mechanisms in regulating attention to threat. Here we investigated whether trait anxiety is associated with a much broader dysregulation of attentional control. Volunteers performed a response-conflict task under conditions that posed high or low demands on attention. High trait-anxious individuals showed reduced prefrontal activity and slower target identification in response to processing competition when the task did not fully occupy attentional resources. The relationship between trait anxiety and prefrontal recruitment remained after controlling for state anxiety. These findings indicate that trait anxiety is linked to impoverished recruitment of prefrontal attentional control mechanisms to inhibit distractor processing even when threat-related stimuli are absent. Notably, this deficit was observed when ongoing task-related demands on attention were low, potentially explaining the day-to-day difficulties in concentration that are associated with clinical anxiety.