Recent advances in the estimation of prokaryotic diversity have brought us insight into two questions: what is the extent of prokaryotic diversity, and perhaps more importantly, why bother finding out. In this review, we highlight the insights about the extent of diversity that may be gained by considering patterns that occur, or are likely to occur, in the relative abundance of prokaryotic taxa. We posit that global reservoirs of diversity are an important driving force behind patterns in localised diversity seen in leaves, intestines and wastewater treatment reactors. Thus, where the reservoir community is very large and relatively even, chance alone will prevent physically identical communities from having the same, or sometimes even stable, communities. By contrast, communities that tend to be similar (even when not physically identical) and stable are observed where the source diversity is low. Thus the relationship between structure and function in a community can only be understood, predicted and engineered through an understanding of the source of diversity from which the community is drawn.