Proteins carrying EPS15 homology (EH) domains are present from yeast to mammals. The characterized members of this protein family are all involved in intracellular trafficking, typically endocytosis and endocytic recycling. We focused on two members of this family in Saccharomyces cerevisiae Irs4p and Tax4p, whose functions are less well characterized. We show that the deletion of IRS4 altered the function of a neighboring gene, VPS51, involved in endocytic recycling. The irs4Deltatax4Delta cells complemented for the loss of Vps51p (irs4Deltatax4Delta*) display no defects in endocytosis and endosomal recycling, clearly differentiating these two EH proteins from the other protein family members. Because Irs4p is phosphorylated when autophagy is induced, we studied the potential role of these two proteins in this latter process. We observed a loss of viability upon starvation in irs4Deltatax4Delta* cells because of a delay in bulk autophagy. Irs4p and Tax4p are also required for pexophagy but not for the cytoplasm-to-vacuole pathway. In growing cells, Irs4p and Tax4p colocalized to few cytoplasmic puncta distinct from endosomes and Golgi compartments. In conditions inducing autophagy, Irs4p and Tax4p partially localized to the pre-autophagosomal structure (PAS) and are required to efficiently recruit to the PAS Atg17p, a factor modulating the autophagic response. We propose that Irs4p and Tax4p are two redundant modulators of the autophagic processes acting upstream from Atg17p, possibly in the signaling events leading to the activation of the autophagic machinery in response to starvation.