SET domain enzymes represent a distinct family of protein lysine methyltransferases in eukaryotes. Recent studies have yielded significant insights into the structural basis of substrate recognition and the product specificities of these enzymes. However, the mechanism by which SET domain methyltransferases catalyze the transfer of the methyl group from S-adenosyl-L-methionine to the lysine epsilon-amine has remained unresolved. To elucidate this mechanism, we have determined the structures of the plant SET domain enzyme, pea ribulose-1,5 bisphosphate carboxylase/oxygenase large subunit methyltransferase, bound to S-adenosyl-L-methionine, and its non-reactive analogs Aza-adenosyl-L-methionine and Sinefungin, and characterized the binding of these ligands to a homolog of the enzyme. The structural and biochemical data collectively reveal that S-adenosyl-L-methionine is selectively recognized through carbon-oxygen hydrogen bonds between the cofactor's methyl group and an array of structurally conserved oxygens that comprise the methyl transfer pore in the active site. Furthermore, the structure of the enzyme co-crystallized with the product epsilon-N-trimethyllysine reveals a trigonal array of carbon-oxygen interactions between the epsilon-ammonium methyl groups and the oxygens in the pore. Taken together, these results establish a central role for carbon-oxygen hydrogen bonding in aligning the cofactor's methyl group for transfer to the lysine epsilon-amine and in coordinating the methyl groups after transfer to facilitate multiple rounds of lysine methylation.