The main asteroid belt lies between the orbits of Mars and Jupiter, but the region is not uniformly filled with asteroids. There are gaps, known as the Kirkwood gaps, in distinct locations that are associated with orbital resonances with the giant planets; asteroids placed in these locations will follow chaotic orbits and be removed. Here we show that the observed distribution of main belt asteroids does not fill uniformly even those regions that are dynamically stable over the age of the Solar System. We find a pattern of excess depletion of asteroids, particularly just outward of the Kirkwood gaps associated with the 5:2, the 7:3 and the 2:1 Jovian resonances. These features are not accounted for by planetary perturbations in the current structure of the Solar System, but are consistent with dynamical ejection of asteroids by the sweeping of gravitational resonances during the migration of Jupiter and Saturn approximately 4 Gyr ago.