Correlated spiking is often observed in cortical circuits, but its functional role is controversial. It is believed that correlations are a consequence of shared inputs between nearby neurons and could severely constrain information decoding. Here we show theoretically that recurrent neural networks can generate an asynchronous state characterized by arbitrarily low mean spiking correlations despite substantial amounts of shared input. In this state, spontaneous fluctuations in the activity of excitatory and inhibitory populations accurately track each other, generating negative correlations in synaptic currents which cancel the effect of shared input. Near-zero mean correlations were seen experimentally in recordings from rodent neocortex in vivo. Our results suggest a reexamination of the sources underlying observed correlations and their functional consequences for information processing.