The physiological androgens testosterone and 5α-dihydrotestosterone regulate the development and maintenance of primary and secondary male sexual characteristics through binding to the androgen receptor (AR), a ligand-dependent transcription factor. In addition, a number of nonreproductive tissues of both genders are subject to androgen regulation. AR is also a central target in the treatment of prostate cancer. A large number of studies over the last decade have characterized many regulatory aspects of the AR pathway, such as androgen-dependent transcription programs, AR cistromes, and coregulatory proteins, mostly in cultured cells of prostate cancer origin. Moreover, recent work has revealed the presence of pioneer/licensing factors and chromatin modifications that are important to guide receptor recruitment onto appropriate chromatin loci in cell lines and in tissues under physiological conditions. Despite these advances, current knowledge related to the mechanisms responsible for receptor- and tissue-specific actions of androgens is still relatively limited. Here, we review topics that pertain to these specificity issues at different levels, both in cultured cells and tissues in vivo, with a particular emphasis on the nature of the steroid, the response element sequence, the AR cistromes, pioneer/licensing factors, and coregulatory proteins. We conclude that liganded AR and its DNA-response elements are required but are not sufficient for establishment of tissue-specific transcription programs in vivo, and that AR-selective actions over other steroid receptors rely on relaxed rather than increased stringency of cis-elements on chromatin.