O-linked N-acetylglucosamine (O-GlcNAc) modification of specific serines/threonines on intracellular proteins in higher eukaryotes has been shown to directly regulate important processes such as the cell cycle, insulin sensitivity and transcription. The structure, molecular mechanisms of catalysis, protein substrate recognition/specificity of the eukaryotic O-GlcNAc transferase and hydrolase are largely unknown. Here we describe the crystal structure, enzymology and in vitro activity on human substrates of Clostridium perfringens NagJ, a close homologue of human O-GlcNAcase (OGA), representing the first family 84 glycoside hydrolase structure. The structure reveals a deep active site pocket highly conserved with the human enzyme, compatible with binding of O-GlcNAcylated peptides. Together with mutagenesis data, the structure supports a variant of the substrate-assisted catalytic mechanism, involving two aspartic acids and an unusually positioned tyrosine. Insights into recognition of substrate come from a complex with the transition state mimic O-(2-acetamido-2-deoxy-D-glucopyranosylidene)amino-N-phenylcarbamate (Ki=5.4 nM). Strikingly, the enzyme is inhibited by the pseudosubstrate peptide Ala-Cys(-S-GlcNAc)-Ala, and has OGA activity against O-GlcNAcylated human proteins, suggesting that the enzyme is a suitable model for further studies into the function of human OGA.