Pleiotropy-when a single mutation affects multiple traits-is a controversial topic with far-reaching implications. Pleiotropy plays a central role in debates about how complex traits evolve and whether biological systems are modular or are organized such that every gene has the potential to affect many traits. Pleiotropy is also critical to initiatives in evolutionary medicine that seek to trap infectious microbes or tumors by selecting for mutations that encourage growth in some conditions at the expense of others. Research in these fields, and others, would benefit from understanding the extent to which pleiotropy reflects inherent relationships among phenotypes that correlate no matter the perturbation (vertical pleiotropy). Alternatively, pleiotropy may result from genetic changes that impose correlations between otherwise independent traits (horizontal pleiotropy). We distinguish these possibilities by using clonal populations of yeast cells to quantify the inherent relationships between single-cell morphological features. Then, we demonstrate how often these relationships underlie vertical pleiotropy and how often these relationships are modified by genetic variants (quantitative trait loci [QTL]) acting via horizontal pleiotropy. Our comprehensive screen measures thousands of pairwise trait correlations across hundreds of thousands of yeast cells and reveals ample evidence of both vertical and horizontal pleiotropy. Additionally, we observe that the correlations between traits can change with the environment, genetic background, and cell-cycle position. These changing dependencies suggest a nuanced view of pleiotropy: biological systems demonstrate limited pleiotropy in any given context, but across contexts (e.g., across diverse environments and genetic backgrounds) each genetic change has the potential to influence a larger number of traits. Our method suggests that exploiting pleiotropy for applications in evolutionary medicine would benefit from focusing on traits with correlations that are less dependent on context.