Progress in genetic improvement of crop yield potential has slowed since 1985. Simultaneously, more sustainable management of agricultural ecosystems is needed. A better understanding of natural selection can help solve both problems. We illustrate this point with two specific examples. First, the genetic legacy of crop plants has been refined by millions of years of natural selection, often driven by competition among plants. We therefore suggest that most simple, tradeoff-free options to increase competitiveness (e.g., increased gene expression, or minor modifications of existing plant genes) have already been tested by natural selection. Further genetic improvement of crop yield potential over the next decade will mainly involve tradeoffs, either between fitness in past versus present environments, or between individual competitiveness and the collective performance of plant communities. Eventually, we may develop the ability to predict the consequences of genetic alterations so radical that they have not yet been tested by natural selection. Second, natural selection acts mainly at the level of genes, individuals, and family groups, rather than ecosystems as a whole. Consequently, there is no reason to expect the structure of natural ecosystems (diversity, spatial, or temporal patterns) to be a reliable blueprint for agricultural ecosystems. Natural ecosystems are nonetheless an important source of information that could be used to improve agriculture.