Modality-specific sensory inputs from individual sense organs are processed in parallel in distinct areas of the neocortex. For each sensory modality, input follows a cortico-thalamo-cortical loop in which a 'first-order' exteroceptive thalamic nucleus sends peripheral input to the primary sensory cortex, which projects back to a 'higher order' thalamic nucleus that targets a secondary sensory cortex. This conserved circuit motif raises the possibility that shared genetic programs exist across sensory modalities. Here we report that, despite their association with distinct sensory modalities, first-order nuclei in mice are genetically homologous across somatosensory, visual, and auditory pathways, as are higher order nuclei. We further reveal peripheral input-dependent control over the transcriptional identity and connectivity of first-order nuclei by showing that input ablation leads to induction of higher-order-type transcriptional programs and rewiring of higher-order-directed descending cortical input to deprived first-order nuclei. These findings uncover an input-dependent genetic logic for the design and plasticity of sensory pathways, in which conserved developmental programs lead to conserved circuit motifs across sensory modalities.