Over the last century scientific research has become an increasingly collaborative endeavor. Commentators have pointed to different factors which contribute to this trend, including the specialization of science and growing need for diversity of interest and expertise areas in a scientific team. Very few studies, however, have precisely evaluated how the diversity of interest topics between researchers is related to the emergence of collaboration. Existing theoretical arguments suggest a curvilinear relationship between topic similarity and collaboration: too little similarity can complicate communication and agreement, yet too much overlap can increase competition and limit the potential for synergy. We test this idea using data on six years of publications across all disciplines at a large U.S. research university (approximately 14,300 articles, 12,500 collaborations, and 3,400 authors). Employing topic modelling and network statistical models, we analyze the relationship between topic overlap and the likelihood of coauthorship between two researchers while controlling for potential confounders. We find an inverted-U relationship in which the probability of collaboration initially increases with topic similarity, then rapidly declines after peaking at a similarity "sweet spot". Collaboration is most likely at low-to-moderate levels of topic overlap, which are substantially lower than the average self-similarity of scientists or research groups. These findings - which we replicate for different units of analysis (individuals and groups), genders of collaborators, disciplines, and collaboration types (intra- and interdisciplinary) - support the notion that researchers seek collaborators to augment their scientific and technical human capital. We discuss implications for theories of scientific collaboration and research policy.
Keywords: Collaboration; Exponential Random Graph Models; Latent Semantic Analysis; Science of Science; Team science; Topic modeling.