ADP ribosylation factors (Arf) regulate membrane trafficking at multiple intracellular sites by recruiting coat proteins to membranes. The site-specific regulation of Arf is thought to be mediated by regulatory proteins including the guanine nucleotide exchange factors (GEFs) and GTPase-activating proteins (GAPs). Here, we test this hypothesis by comparing the site of action of the Arf GAP AGAP2 to the closely related AGAP1. AGAP1 has previously been found to associate with the adaptor protein complex AP-3 and regulate the function of AP-3 endosomes. We found that AGAP2 directly interacted with AP-1. AGAP2 colocalized with AP-1, transferrin receptor and Rab4 on endosomes. Overexpression of AGAP2 changed the intracellular distribution of AP-1 and promoted Rab4-dependent fast recycling of transferrin. Based on these results, we concluded that the closely related Arf GAPs, AGAP1 and AGAP2, distinguish between these related heterotetrameric adaptor protein complexes to specifically regulate AP-3 endosomes and AP-1 recycling endosomes.