The capacity of coral-dinoflagellate mutualisms to adapt to a changing climate relies in part on standing variation in host and symbiont populations, but rarely have the interactions between symbiotic partners been considered at the level of individuals. Here, we tested the importance of inter-individual variation with respect to the physiology of coral holobionts. We identified six genetically distinct Acropora palmata coral colonies that all shared the same isoclonal Symbiodinium 'fitti' dinoflagellate strain. No other Symbiodinium could be detected in host tissues. We exposed fragments of each colony to extreme cold and found that the stress-induced change in symbiont photochemical efficiency varied up to 3.6-fold depending on host genetic background. The S. 'fitti' strain was least stressed when associating with hosts that significantly altered the expression of 184 genes under cold shock; it was most stressed in hosts that only adjusted 14 genes. Key expression differences among hosts were related to redox signaling and iron availability pathways. Fine-scale interactions among unique host colonies and symbiont strains provide an underappreciated source of raw material for natural selection in coral symbioses.