Many living organisms contain biominerals and composites with finely tuned properties, reflecting a remarkable level of control over the nucleation, growth and shape of the constituent crystals. Peptides and proteins play an important role in achieving this control. But the general view that organic molecules affect mineralization through stereochemical recognition, where geometrical and chemical constraints dictate their binding to a mineral, seems difficult to reconcile with a mechanistic understanding, where crystallization is controlled by thermodynamic and kinetic factors. Indeed, traditional crystal growth models emphasize the inhibiting effect of so-called 'modifiers' on surface-step growth, rather than stereochemical matching to newly expressed crystal facets. Here we report in situ atomic force microscope observations and molecular modelling studies of calcite growth in the presence of chiral amino acids that reconcile these two seemingly divergent views. We find that enantiomer-specific binding of the amino acids to those surface-step edges that offer the best geometric and chemical fit changes the step-edge free energies, which in turn results in macroscopic crystal shape modifications. Our results emphasize that the mechanism underlying crystal modification through organic molecules is best understood by considering both stereochemical recognition and the effects of binding on the interfacial energies of the growing crystal.