It has been suggested over the decades that dysfunctional anxiety may be caused by distinct alterations in cognitive processing. To learn more about the relation between anxiety and cognitive functioning, two mouse strains that display either adaptive (BALB/c) or nonadaptive (129P2) anxiety, as reflected by their ability to habituate when repeatedly exposed to a novel environment, were tested for their cognitive performance in the modified hole board (mHB) task. In general, both strains showed successful acquisition of the task. The initially more anxious BALB/c mice revealed rapid habituation to the test setup, followed by decreased long-term and short-term memory errors across the experimental period and fast relearning after reversal of the task. By contrast, the nonadaptive 129P2 mice made more short-term memory errors and performed worse than the BALB/c animals after reversal of the test. The results confirm the proposed interaction of anxiety and cognition: In BALB/c mice, adaptive characteristics of anxiety were paralleled by more successful cognitive performance, while in 129P2 mice nonadaptive anxiety-related behaviour was accompanied by a higher level of short-term memory errors and less cognitive flexibility. Moreover, these results support our hypothesis that the nonadaptive anxiety phenotype in 129P2 mice may be the result of impaired cognitive control of emotional processes, resulting in impaired behavioural flexibility, for example in response to novelty.