Contemporary researchers strive to understand complex physical phenomena that involve many constituents, may be influenced by numerous forces, and may exhibit unexpected or emergent behavior. Often such "complex systems" are macroscopic manifestations of other systems that exhibit their own complex behavior and obey more elemental laws. This article proposes that areas of mathematics, even ones based on simple axiomatic foundations, have discernible layers, entirely unexpected "macroscopic" outcomes, and both mathematical and physical ramifications profoundly beyond their historical beginnings. In a larger sense, the study of mathematics itself, which is increasingly surpassing the capacity of researchers to verify "by hand," may be the ultimate complex system.