Herbivory can negatively and selectively affect plant fitness by reducing growth, survival and reproductive output, thereby influencing plant population dynamics and evolution. Latitudinal variation in intensity of herbivory is common, but the extent to which it translates into corresponding variation in effects on plant performance is still poorly known. We tested the hypothesis that variation in the fitness-consequences of herbivory mirror differences in intensity of herbivory among three natural populations of the perennial herb Lythrum salicaria along a latitudinal gradient from southern to northernmost Sweden. We documented intensity of herbivory and examined its effect on survival, growth and reproductive output over two years by experimentally removing herbivores with insecticide. The intensity of herbivory and the effects of herbivory on plant fitness were strongest in the southern population, intermediate in the central population and weakest in the northern population. The mean proportion of the leaf area removed ranged from 11% in the southern to 3% in the northern population. Herbivore removal increased plant height 1.5-fold in the southern and 1.2-fold in the central population, the proportion plants flowering 4-fold in the southern and 2-fold in the central population, and seed production per flower 1.6-fold in the southern and 1.2-fold in the central population, but did not affect plant fitness in the northern population. Herbivore removal thus affected the relative fecundity of plants in the three populations: In the control, seed output per plant was 8.6 times higher in the northern population compared to the southern population, whereas after herbivore removal it was 2.5 times higher in the southern population. The results demonstrate that native herbivores may strongly affect the demographic structure of L. salicaria populations and thereby shape geographic patterns of seed production. They further suggest that the strength of herbivore-mediated selection varies among populations and decreases towards the north.