Glycomics is the comprehensive study of glycan expression in an organism, cell, or tissue that relies on effective analytical technologies to understand glycan structure-function relationships. Owing to the macro- and micro-heterogeneity of oligosaccharides, detailed structure characterization has required an orthogonal approach, such as a combination of specific exoglycosidase digestions, LC-MS/MS, and the development of bioinformatic resources to comprehensively profile a complex biological sample. Liquid chromatography-electrospray ionization-mass spectrometry (LC-ESI-MS/MS) has emerged as a key tool in the structural analysis of oligosaccharides because of its high sensitivity, resolution, and robustness. Here, we present a strategy that uses LC-ESI-MS/MS to characterize over 200 N- and O-glycans from human saliva glycoproteins, complemented by sequential exoglycosidase treatment, to further verify the annotated glycan structures. Fragment-specific substructure diagnostic ions were collated from an extensive screen of the literature available on the detailed structural characterization of oligosaccharides and, together with other specific glycan structure feature ions derived from cross-ring and glycosidic-linkage fragmentation, were used to characterize the glycans and differentiate isomers. The availability of such annotated mass spectrometric fragmentation spectral libraries of glycan structures, together with such substructure diagnostic ions, will be key inputs for the future development of the automated elucidation of oligosaccharide structures from MS/MS data.