Social insects rank among the most abundant and influential terrestrial organisms. The key to their success is their ability to form tightly knit social groups that perform work cooperatively, and effectively exclude non-members from the colony. An extensive body of research, both empirical and theoretical, has explored how optimal acceptance thresholds could evolve in individuals, driven by the twin costs of inappropriately rejecting true nestmates and erroneously accepting individuals from foreign colonies. Here, in contrast, we use agent-based modeling to show that strong nestmate recognition by individuals is often unnecessary. Instead, highly effective nestmate recognition can arise as a colony-level property from a collective of individually poor recognizers. Essentially, although an intruder can get by one defender when their odor cues are similar, it is nearly impossible to get past many defenders if there is the slightest difference in cues. The results of our models match observed rejection rates in studies of ants, wasps, and bees. We also show that previous research in support of the optimal threshold theory approach to the problem of nestmate recognition can be alternatively viewed as evidence in favor of the collective formation of a selectively permeable barrier that allows in nestmates (at a significant cost) while rejecting non-nestmates. Finally, this work shows that nestmate recognition has a stronger task allocation component than previously thought, as colonies can nearly always achieve perfect nestmate recognition if it is cost effective for them to do so at the colony level. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1007/s00265-010-1094-x) contains supplementary material, which is available to authorized users.