The lineage of dental mesenchymal cells originates in the cranial neural crest, and after sequential determination and differentiation, gives rise to all structures of the tooth and its supporting tissues, except the enamel. Reciprocal interactions between the epithelial and mesenchymal tissues are conceivably the most important regulators of dental mesenchymal cell differentiation. The molecular mechanisms of this epigenetic regulation are not known at present. In order to examine the mechanisms of regulation of gene expression in the lineage of dental mesenchymal cells, information is needed on the molecular changes that accompany advancing differentiation. By using the molar tooth germ of mouse embryos as a model system, the changes in the expression of some molecules have been analysed by immunohistological localization and in situ hybridization, and the roles of tissue interactions in this process examined. This has shown that syndecan, a recently characterized cell surface proteoglycan, and tenascin, a matrix glycoprotein, appear in the condensing dental mesenchyme during the bud stage of tooth development. During the cap stage, dental mesenchyme is characterized by continued intense expression of syndecan, but this is lost during terminal differentiation of odontoblasts. Tenascin and syndecan may mediate cell-matrix interactions during condensation of dental mesenchymal cells. Expression of the Int-2 proto-oncogene, coding for a fibroblast growth factor-related molecule, can be detected by in situ hybridization in dental mesenchyme at the cap stage. This expression persists in cuspal mesenchyme at the bell stage but is lost from odontoblasts and from pulpal mesenchyme at progressive stages of tooth development. The advancement of tooth morphogenesis from cap to bell stage is accompanied by expression of alkaline phosphatase in the cuspal mesenchyme. Also tenascin, which is only weakly expressed during the cap stage, appears in the cuspal areas and shows codistribution with alkaline phosphatase. These observations indicate that the sequential determination and differentiation of the dental mesenchymal cells are characterized by a cascade of specific molecular changes. The cell surface proteoglycan syndecan and the Int-2 proto-oncogene are specific and transient markers of early dental mesenchymal cell differentiation. This information allows studies on the mechanisms of developmental regulation. These experimental tissue recombination studies indicate that the expression of syndecan and tenascin in the early dental mesenchyme is induced by the presumptive dental epithelium.(ABSTRACT TRUNCATED AT 400 WORDS)