Endocytic recycling of internalized transmembrane proteins is essential for many important physiological processes. Recent studies have revealed that retromer-related Sorting Nexin family (SNX)-Bin/Amphiphysin/Rvs (BAR) proteins can directly recognize cargoes like cation-independent mannose 6-phosphate receptor (CI-MPR) and Insulin-like growth factor 1 receptor (IGF1R); however, it remains poorly understood how SNX-BARs select specific cargo proteins and whether they recognize additional ligands. Here, we discovered that the binding between SNX-BARs and CI-MPR or IGF1R is mediated by the phox-homology (PX) domain of SNX5 or SNX6 and a bipartite motif, termed SNX-BAR-binding motif (SBM), in the cargoes. Using this motif, we identified over 70 putative SNX-BAR ligands, many of which play critical roles in apoptosis, cell adhesion, signal transduction, or metabolite homeostasis. Remarkably, SNX-BARs could cooperate with both SNX27 and retromer in the recycling of ligands encompassing the SBM, PDZ-binding motif, or both motifs. Overall, our studies establish that SNX-BARs function as a direct cargo-selecting module for a large set of transmembrane proteins transiting the endosome, in addition to their roles in phospholipid recognition and biogenesis of tubular structures.