Heart rate (HR) responds to exercise by increasing during exercise and recovering after exercise. As such, HR is an important predictor of mortality that researchers believe is modulated by the autonomic nervous system. However, the mechanistic basis underlying inter-individual differences has yet to be explained. Here, we perform a large-scale genome-wide analysis of HR increase and HR recovery in 58,818 UK Biobank individuals. Twenty-five independent SNPs in 23 loci are identified to be associated (p < 8.3 × 10-9) with HR increase or HR recovery. A total of 36 candidate causal genes are prioritized that are enriched for pathways related to neuron biology. No evidence is found of a causal relationship with mortality or cardiovascular diseases. However, a nominal association with parental lifespan requires further study. In conclusion, the findings provide new biological and clinical insight into the mechanistic underpinnings of HR response to exercise. The results also underscore the role of the autonomous nervous system in HR recovery.