The vitamin K dependent carboxylase of liver microsomes is involved in the posttranslational modification of certain serine protease zymogens which are critical components of the blood clotting cascade. During coupled carboxylation/oxygenation this carboxylase converts glutamate residues, dihydrovitamin K, CO2, and O2 to a gamma-carboxyglutamyl (Gla) residue, vitamin K (2R,3S)-epoxide, and H2O with a stoichiometry of 1:1 for all substrates and products. In this paper we investigate the role of molecular oxygen in the reaction by following the course of the oxygen atoms using 18O2. Two different mass spectroscopic techniques, electron ionization positive ion mass spectrometry and supercritical fluid chromatography-negative ion chemical ionization mass spectrometry, were used to quantitate the amount of 18O incorporation into the various oxygens of the vitamin K epoxide product. We found that 0.95 mol atoms of oxygen were incorporated into the epoxide oxygen, 0.05 mol atoms of oxygen were incorporated into the quinone oxygen of vitamin K epoxide, and the remaining ca. 1.0 mol atoms of oxygen were incorporated into H2O. No incorporation of oxygen into vitamin K epoxide from 50% H2(18)O was observed. Thus, the carboxylase operates as a dioxygenase 5% of the time during carboxylation/oxygenation. The relevance of these findings with respect to the nonenzymic "basicity enhancement" model proposed by Ham and Dowd [(1990) J. Am. Chem. Soc. 112, 1660-1661] is discussed.