Monomeric sarcosine oxidase (MSOX) is an inducible bacterial flavoenzyme that catalyzes the oxidative demethylation of sarcosine (N-methylglycine) and contains covalently bound FAD [8alpha-(S-cysteinyl)FAD]. This paper describes the spectroscopic and thermodynamic properties of MSOX as well as the X-ray crystallographic characterization of three new enzyme.inhibitor complexes. MSOX stabilizes the anionic form of the oxidized flavin (pK(a) = 8.3 versus 10.4 with free FAD), forms a thermodynamically stable flavin radical, and stabilizes the anionic form of the radical (pK(a) < 6 versus pK(a) = 8.3 with free FAD). MSOX forms a covalent flavin.sulfite complex, but there appears to be a significant kinetic barrier against complex formation. Active site binding determinants were probed in thermodynamic studies with various substrate analogues whose binding was found to perturb the flavin absorption spectrum and inhibit MSOX activity. The carboxyl group of sarcosine is essential for binding since none is observed with simple amines. The amino group of sarcosine is not essential, but binding affinity depends on the nature of the substitution (CH(3)XCH(2)CO(2)(-), X = CH(2) < O < S < Se < Te), an effect which has been attributed to differences in the strength of donor-pi interactions. MSOX probably binds the zwitterionic form of sarcosine, as judged by the spectrally similar complexes formed with dimethylthioacetate [(CH(3))(2)S(+)CH(2)CO(2)(-)] and dimethylglycine (K(d) = 20.5 and 17.4 mM, respectively) and by the crystal structure of the latter. The methyl group of sarcosine is not essential but does contribute to binding affinity. The methyl group contribution varied from -3.79 to -0.65 kcal/mol with CH(3)XCH(2)CO(2)(-) depending on the nature of the heteroatom (NH(2)(+) > O > S) and appeared to be inversely correlated with heteroatom electron density. Charge-transfer complexes are formed with MSOX and CH(3)XCH(2)CO(2)(-) when X = S, Se, or Te. An excellent linear correlation is observed between the energy of the charge transfer bands and the one-electron reduction potentials of the ligands. The presence of a sulfur, selenium, or telurium atom identically positioned with respect to the flavin ring is confirmed by X-ray crystallography, although the increased atomic radius of S < Se < Te appears to simultaneously favor an alternate binding position for the heavier atoms. Although L-proline is a poor substrate, aromatic heterocyclic carboxylates containing a five-membered ring and various heteroatoms (X = NH, O, S) are good ligands (K(d, X=NH) = 1.37 mM) and form charge-transfer complexes with MSOX. The energy of the charge-transfer bands (S > O >> NH) is linearly correlated with the one-electron ionization potentials of the corresponding heterocyclic rings.