The Golgi-localized golgins golgin-97 and golgin-245 capture transport vesicles arriving from endosomes via the protein TBC1D23. The amino-terminal domain of TBC1D23 binds to the golgins, and the carboxyl-terminal domain of TBC1D23 captures the vesicles, but how it recognizes specific vesicles was unclear. A search for binding partners of the carboxyl-terminal domain unexpectedly revealed direct binding to carboxypeptidase D and syntaxin-16, known cargo proteins of the captured vesicles. Binding is via a threonine-leucine-tyrosine (TLY) sequence present in both proteins next to an acidic cluster. A crystal structure reveals how this acidic TLY motif binds to TBC1D23. An acidic TLY motif is also present in the tails of other endosome-to-Golgi cargo, and these also bind TBC1D23. Structure-guided mutations in the carboxyl-terminal domain that disrupt motif binding in vitro also block vesicle capture in vivo. Thus, TBC1D23 attached to golgin-97 and golgin-245 captures vesicles by a previously undescribed mechanism: the recognition of a motif shared by cargo proteins carried by the vesicle.