Individuals of many quite distantly related animal species find each other attractive and stay together for long periods in groups. We present a mechanism for mixed-species grouping in which individuals from different-looking prey species come together because the appearance of the mixed-species group is visually confusing to shared predators. Using an artificial neural network model of retinotopic mapping in predators, we train networks on random projections of single- and mixed-species prey groups and then test the ability of networks to reconstruct individual prey items from mixed-species groups in a retinotopic map. Over the majority of parameter space, cryptic prey items benefit from association with conspicuous prey because this particular visual combination worsens predator targeting of cryptic individuals. However, this benefit is not mutual as conspicuous prey tends to be targeted most poorly when in same-species groups. Many real mixed-species groups show the asymmetry in willingness to initiate and maintain the relationship predicted by our study. The agreement of model predictions with published empirical work, the efficacy of our modelling approach in previous studies, and the taxonomic ubiquity of retinotopic maps indicate that we may have uncovered an important, generic selective agent in the evolution of mixed-species grouping.