The histone chaperone Rtt106 binds histone H3 acetylated at lysine 56 (H3K56ac) and facilitates nucleosome assembly during several molecular processes. Both the structural basis of this modification-specific recognition and how this recognition informs Rtt106 function are presently unclear. Guided by our crystal structure of Rtt106, we identified two regions on its double-pleckstrin homology domain architecture that mediated histone binding. When histone binding was compromised, Rtt106 localized properly to chromatin but failed to deliver H3K56ac, leading to replication and silencing defects. By mutating analogous regions in the structurally homologous chromatin-reorganizer Pob3, we revealed a conserved histone-binding function for a basic patch found on both proteins. In contrast, a loop connecting two β-strands was required for histone binding by Rtt106 but was dispensable for Pob3 function. Unlike Rtt106, Pob3 histone binding was modification-independent, implicating the loop of Rtt106 in H3K56ac-specific recognition in vivo. Our studies described the structural origins of Rtt106 function, identified a conserved histone-binding surface, and defined a critical role for Rtt106:H3K56ac-binding specificity in silencing and replication-coupled nucleosome turnover.