Mullerian inhibiting substance (MIS), a member of the TGFbeta superfamily, causes regression of the Mullerian duct in male embryos. The presence of MIS type II and type I receptors in tissues and cell lines derived from the prostate suggests that prostate is a likely target for MIS. In this report, we demonstrate that MIS inhibits androgen-stimulated growth of LNCaP cells and decreases their survival in androgen-deprived medium by preventing cell cycle progression and inducing apoptosis. Expression of dominant-negative Smad1 reversed the ability of MIS to decrease LNCaP cell survival in androgen-deprived medium but not androgen-stimulated growth, whereas abrogation of nuclear factor-kappaB (NFkappaB) activation ablated the suppressive effects of MIS on both androgen-stimulated growth and androgen-independent survival. The effect of MIS on androgen-induced growth was not due to changes in androgen receptor expression. However, MIS suppressed androgen-stimulated transcription of prostate-specific antigen; ablation of NFkappaB activation reversed MIS-mediated suppression of prostate-specific antigen. These observations suggest that MIS regulates androgen-induced gene expression and growth in prostate cancer cells through a NFkappaB-dependent but Smad1-independent mechanism. Thus, MIS, in addition to potentially regulating prostate growth indirectly by suppressing testicular testosterone synthesis, may also be a direct regulator of androgen-induced gene expression and growth in the prostate at the cellular level.