Social interaction increases significantly the performance of a wide range of cooperative systems. However, evidence that natural swarms limit the number of interactions suggests potentially detrimental consequences of excessive interaction. Using a canonical model of collective motion, we find that the collective response to a dynamic localized perturbation-emulating a predator attack-is hindered when the number of interacting neighbors exceeds a certain threshold. Specifically, the effectiveness in avoiding the predator is enhanced by large integrated correlations, which are known to peak at a given level of interagent interaction. From the network-theoretic perspective, we uncover the same interplay between number of connections and effectiveness in group-level response for two distinct decision-making models of distributed consensus operating over a range of static networks. The effect of the number of connections on the collective response critically depends on the dynamics of the perturbation. While adding more connections improves the response to slow perturbations, the opposite is true for fast ones. These results have far-reaching implications for the design of artificial swarms or interaction networks.