A quantitative genetic model for the response of the distribution of a single metric trait to directional selection is investigated. Particular attention is paid to the performance of approximations that use only limited information on the initial state of the population, such as the mean, the variance, the skewness and the kurtosis. Selection is imposed according to an exponentially increasing fitness function, populations mate at random, have discrete generations, and all genetic effects are supposed to be additive. Neglecting random drift, qualitatively different predictions for the initial response of a large population that has previously been at a mutation-stabilizing selection balance are derived. These depend on different assumptions about the initial distribution and are compared to the exact dynamics of that model. Linkage disequilibrium can be ignored as long as linkage is not too tight. The mathematical analysis rests on the method of cumulants and of cumulant-generating functions and produces exact equations for the evolution of cumulants in this model. Small populations subject to random drift are shown to respond in a qualitatively different way. The case of small populations is treated, primarily, by Monte Carlo simulations. The consequences of qualitatively different assumptions about maintenance of variation through mutation for the initial response to exponential directional selection are discussed. It is concluded that a significant initial increase in variance is unlikely to be observed in selection experiments if the effective population size is not larger than 500 and if response is caused by additive genes. The present results also apply to weak truncation selection.