We have recently observed that cigarette smoking affects plasma androgen concentrations. The effects of nicotine and cotinine, two products of cigarette smoking, on testosterone metabolism were determined. The activity of delta 4 steroid 5 alpha-reductase, which converts testosterone to 5 alpha-dihydrotestosterone (DHT) was measured in isolated dog prostate nuclei using testosterone (0-200 nM) as substrate and NADPH as cofactor. Activity of 3 alpha-hydroxysteroid dehydrogenase (HSD), which converts DHT to 3 alpha-androstanediol (3 alpha-diol) and is a reversible enzyme, was measured in isolated dog prostate microsomes with DHT (0-20 microM) as substrate and NADPH as cofactor. When microsomal fractions were incubated for 1 hour with and without nicotine (0-50 microM) and cotinine (0-100 microM), enzyme activity of HSD was significantly suppressed (p less than 0.001). The Vmax was not affected significantly (p greater than 0.60) and Km increased with increasing concentrations of nicotine and cotinine (p less than 0.05). Both nicotine and cotinine are competitive inhibitors of HSD in dog prostate microsomes with Ki's of 61 and 89 microM, respectively. The apparent 5 alpha-reductase activity was unaffected by nicotine and cotinine. The inhibitors produced a marked effect on activity of HSD when used in concentrations achieved in humans who smoke cigarettes. The results suggest that nicotine and cotinine are competitive inhibitors of the HSD, an important enzyme involved in the metabolism of DHT and produce an accumulation of DHT. These products of cigarette smoking could alter androgen action in tissue such as skin and prostate.