Galacto-N-biose/lacto-N-biose I phosphorylase (GLNBP) from Bifidobacterium longum, a key enzyme for intestinal growth, phosphorolyses galacto-N-biose and lacto-N-biose I with anomeric inversion. GLNBP homologues are often found in human pathogenic and commensal bacteria, and their substrate specificities potentially define the nutritional acquisition ability of these microbes in their habitat. We report the crystal structures of GLNBP in five different ligand-binding forms. This is the first three-dimensional structure of glycoside hydrolase (GH) family 112. The GlcNAc- and GalNAc-bound forms provide structural insights into distinct substrate preferences of GLNBP and its homologues from pathogens. The catalytic domain consists of a partially broken TIM barrel fold that is structurally similar to a thermophilic beta-galactosidase, strongly supporting the current classification of GLNBP homologues as one of the GH families. Anion binding induces a large conformational change by rotating a half-unit of the barrel. This is an unusual example of molecular adaptation of a TIM barrel scaffold to substrates.