The extent of soil microbial diversity is seen to be critical to the maintenance of soil health and quality. Different agricultural practices are able to affect soil microbial diversity and thus the level of suppressiveness of plant diseases. In a 4-year field experiment, we investigated the microbial diversity of soil under different agricultural regimes. We studied permanent grassland, grassland turned into arable land, long-term arable land and arable land turned into grassland. The diversity of microbial communities was described by using cultivation-based and cultivation-independent methods. Both types of methods revealed differences in the diversities of soil microbial communities between different treatments. The treatments with higher above-ground biodiversity generally maintained higher levels of microbial diversity. Moreover, a positive correlation between suppression of Rhizoctonia solani AG3 and microbial diversity was observed. Permanent (species-rich) grassland and grassland turned into maize stimulated higher microbial diversities and higher levels of suppressiveness of R. solani AG3 compared with the long-term arable land. Effects of agricultural practices on Bacillus and Pseudomonas communities were also observed and clear correlations between the levels of suppressiveness and the diversities of these bacterial groups were found. This study highlighted the importance of agricultural management regime for soil microbial community structure and diversity as well as the level of soil suppressiveness.