Glycosylation reactions mainly catalyzed by glycosyltransferases (Gts) occur almost everywhere in the biosphere, and always play crucial roles in vital processes. In order to understand the full potential of Gts, the chemical and structural glycosylation mechanisms are systematically summarized in this review, including some new outlooks in inverting/retaining mechanisms and the overview of GT-C superfamily proteins as a novel Gt fold. Some special features of glycosylation and the evolutionary studies on Gts are also discussed to help us better understand the function and application potential of Gts. Natural product (NP) glycosylation and related Gts which play important roles in new drug development are emphasized in this paper. The recent advances in the glycosylation pattern (particularly the rare C- and S-glycosylation), reversibility, iterative catalysis and protein auxiliary of NP Gts are all summed up comprehensively. This review also presents the application of NP Gts and associated studies on synthetic biology, which may further broaden the mind and bring wider application prospects.