The relative energetic investment in reproduction between the sexes forms the basis of sexual selection and life history theories in evolutionary biology. It is often assumed that males invest considerably less in gametes than females, but quantifying the energetic cost of gamete production in both sexes has remained a difficult challenge. For a broad diversity of species (invertebrates, reptiles, amphibians, fishes, birds, and mammals), we compared the cost of gamete production between the sexes in terms of the investment in gonad tissue and the rate of gamete biomass production. Investment in gonad biomass was nearly proportional to body mass in both sexes, but gamete biomass production rate was approximately two to four orders of magnitude higher in females. In both males and females, gamete biomass production rate increased with organism mass as a power law, much like individual metabolic rate. This suggests that whole-organism energetics may act as a primary constraint on gamete production among species. Residual variation in sperm production rate was positively correlated with relative testes size. Together, these results suggest that understanding the heterogeneity in rates of gamete production among species requires joint consideration of the effects of gonad mass and metabolism.