Monitoring the effects of no-tillage (NT) in comparison with conventional tillage (CT) on soil microbes could improve our understanding of soil biochemical processes and thus help us to develop sound management strategies. The objective of this study was to compare the species composition and ecological function of soil arbuscular mycorrhizal (AM) fungi during the growth and rotation of crops under NT and CT. From late June 2009 to early June 2010, 32 topsoil (0-15 cm) samples from four individual plots per treatment (CT and NT) were collected at both the jointing and maturation stages of maize (Zea mays L.) and wheat (Triticum aestivum L.) from a long-term experimental field that was established in an Aquic Inceptisol in North China in June 2006. The AM fungal spores were isolated and identified and then used to calculate species diversity indices, including the Shannon- Wiener index (H'), Evenness (E), and Simpson's index (D). The root mycorrhizal colonization and soil alkaline phosphatase activity were also determined. A total of 34 species of AM fungi within nine genera were recorded. Compared with NT, CT negatively affected the soil AM fungal community at the maize sowing stage, leading to decreases in the average diversity indices (from 2.12, 0.79, and 0.82 to 1.79, 0.72, and 0.74 for H', E, and D, respectively), root mycorrhizal colonization (from 28% to 20%), soil alkaline phosphatase activity (from 0.24 to 0.19 mg/g/24 h) and available phosphorus concentration (from 17.4 to 10.5 mg/kg) at the maize jointing stage. However, reductions in diversity indices of H', E, and D were restored to 2.20, 0.81, and 0.84, respectively, at the maize maturation stage. CT should affect the community again at the wheat sowing stage; however, a similar restoration in the species diversity of AM fungi was completed before the wheat jointing stage, and the highest Jaccard index (0.800) for similarity in the species composition of soil AM fungi between CT and NT was recorded at the wheat maturation stage. Our results also demonstrated that NT resulted in the positive protection of the community structure of AM fungi and played an important role in maintaining their functionality especially for maize seedlings.