Fertilizer-induced changes in soil nutrients regulate nitrogen (N) fixation in the terrestrial biosphere, but the influences of N and phosphorus (P) fertilization on the diazotroph communities in successive crop seasons were unclear. In this study, we assessed the effects of N and P (high vs. low doses) on the abundance and structure of N2-fixation communities after wheat and soybean harvest in a long-term (34 and 35 years) fertilization experiment. In both seasons, long-term N addition significantly decreased the abundance of nifH genes and 16S rDNA; in addition, high doses of N and P fertilizer decreased the richness of diazotrophs, whereas low doses did not. The proportion of the dominant genus, Bradyrhizobium, in the soybean season (86.0%) was higher than that in the wheat season (47.9%). Fertilization decreased diazotroph diversity and the relative abundance of Bradyrhizobium in the wheat season, but had insignificant effects in the soybean season. The addition of N, but not P, significantly changed the communities of both diazotrophs (at the genus level) and rhizobia (at the species level) in the two seasons. Soil pH was positively associated with nifH abundance and diazotrophic richness; soil NO3- content was negatively correlated with diazotrophic richness and positively correlated with diversity. Soil pH and NO3- content were the two main drivers shaping the soil diazotrophic community. Overall, long-term inorganic N had a greater influence than P on both diazotrophic abundance and community composition, and diazotrophic diversity was more clearly affected by fertilization in the wheat season than in the soybean season.