The organoselenium compounds benzyl selenocyanate (BSC) and 1,4-phenylenebis(methylene)selenocyanate (p-XSC), as well as sodium selenite, are effective chemopreventive agents for various chemically induced tumors in animal models at both the initiation and postinitiation stages. The mechanisms involved at the postinitiation stage are not clear. Because several lines of evidence indicate that inhibition of excess DNA (cytosine-5)-methyltransferase (Mtase) may be a sufficient factor for the suppression or reversion of carcinogenesis, we examined the effects of sodium selenite, BSC, p-XSC and benzyl thiocyanate (BTC), the sulfur analog of BSC, on Mtase activity in nuclear extracts of human colon carcinomas, and of p-XSC on the Mtase activity of HCT116 human colon carcinoma cells in culture. For this purpose, we developed an improved Mtase assay, in which the incorporation of the methyl-[3H] group from S-adenosyl[methyl-3H]methionine into deoxycytidine of poly(dI-dC)-poly(dI-dC), is specifically determined by HPLC with radioflow detection after enzymatic hydrolysis, enhancing specificity and reliability. In a variation, using SssI methyltransferase and labeled S-adenosylmethionine, the overall methylation status of DNA in various tissues can also be compared. Selenite, BSC and p-XSC inhibited Mtase extracted from a human colon carcinoma with IC50s of 3.8, 8.1 and 5.2 microM, respectively; BTC had no effect. p-XSC also inhibited the Mtase activity and growth of human colon carcinoma HCT116 cells, with an IC50 of approximately 20 microM. The improved Mtase assay should prove to be a reliable method for screening potential Mtase inhibitors, especially using cells in culture. We suggest that inhibition of Mtase may be a major mechanism of chemoprevention by selenium compounds at the postinitiation stage of carcinogenesis.