Dietary composition largely influences pig's gastrointestinal microbiota and represents a useful prophylactic tool against enteric disturbances in young pigs. Despite the importance for host-microbe interactions and bacterial colonization, dietary responses of the mucosa-associated bacterial communities are less well investigated. In the present study, we characterized the mucosa-associated bacterial communities at the Pars non-glandularis of the stomach, ileum and colon, and identified shifts in these communities in response to different dietary calcium-phosphorus (Ca-P) contents (100% versus 190% of the Ca and P requirements) in combination with two basal diets (wheat-barley- or corn-based) in weaned pigs. Pyrosequencing of 16S rRNA genes from 93 mucosal samples yielded 447,849 sequences, clustering into 997 operational taxonomic units (OTUs) at 97% similarity level. OTUs were assigned to 198 genera belonging to 14 different phyla. Correlation-based networks revealed strong interactions among OTUs at the various gastrointestinal sites. Our data describe a previously not reported high diversity and species richness at the Pars non-glandularis of the stomach in weaned pigs. Moreover, high versus adequate Ca-P content significantly promoted Lactobacillus by 14.9% units (1.4 fold change) at the gastric Pars non-glandularis (P = 0.035). Discriminant analysis revealed dynamic changes in OTU composition in response to dietary cereals and Ca-P contents at all gastrointestinal sites which were less distinguishable at higher taxonomic levels. Overall, this study revealed a distinct mucosa-associated bacterial community at the different gut sites, and a strong effect of high Ca-P diets on the gastric community, thereby markedly expanding our comprehension on mucosa-associated microbiota and their diet-related dynamics in weaned pigs.