Evolutionary rescue occurs when a population that is threatened with extinction by an environmental change adapts to the change sufficiently rapidly to survive. Here we extend the mathematical theory of evolutionary rescue. In particular, we model evolutionary rescue to a sudden environmental change when adaptation involves evolution at a single locus. We consider adaptation using either new mutations or alleles from the standing genetic variation that begin rare. We obtain several results: i) the total probability of evolutionary rescue from either new mutation or standing variation; ii) the conditions under which rescue is more likely to involve a new mutation versus an allele from the standing genetic variation; iii) a mathematical description of the U-shaped curve of total population size through time, conditional on rescue; and iv) the time until the average population size begins to rebound as well as the minimal expected population size experienced by a rescued population. Our analysis requires taking into account a subtle population-genetic effect (familiar from the theory of genetic hitchhiking) that involves "oversampling" of those lucky alleles that ultimately sweep to high frequency. Our results are relevant to conservation biology, experimental microbial evolution, and medicine (e.g., the dynamics of antibiotic resistance).