Transient mesoscale oceanic eddies in Eastern Boundary Upwelling Systems are thought to strongly affect key regional scale processes such as ocean heat transport, coastal upwelling and productivity. Understanding how these can be modulated at low-frequency is thus critical to infer their role in the climate system. Here we use 26 years of satellite altimeter data and regional oceanic modeling to investigate the modulation of eddy kinetic energy (EKE) off Peru and Chile by ENSO, the main mode of natural variability in the tropical Pacific. We show that EKE tends to increase during strong Eastern Pacific (EP) El Niño events along the Peruvian coast up to northern Chile and decreases off central Chile, while it is hardly changed during Central Pacific El Niño and La Niña events. However the magnitude of the EKE changes during strong EP El Niño events is not proportional to their strength, with in particular the 1972/1973 El Niño event standing out as an extreme event in terms of EKE increase off Peru reaching an amplitude three times as large as that during the 1997/1998 El Niño event, and the 2015/2016 El Niño having instead a weak impact on EKE. This produces decadal changes in EKE, with a similar pattern than that of strong EP El Niño events, resulting in a significant negative (positive) long-term trend off Peru (central Chile).