The brain plays a central role in controlling energy, glucose, and lipid homeostasis, with specialized neurons within nuclei of the mediobasal hypothalamus, namely the arcuate (ARC) and ventromedial (VMH), tasked with proper signal integration. Exactly how the exquisite cytoarchitecture and underlying circuitry becomes established within these nuclei remains largely unknown, in part because hypothalamic developmental programs are just beginning to be elucidated. Here, we demonstrate that the Retina and anterior neural fold homeobox (Rax) gene plays a key role in establishing ARC and VMH nuclei in mice. First, we show that Rax is expressed in ARC and VMH progenitors throughout development, consistent with genetic fate mapping studies demonstrating that Rax+ lineages give rise to VMH neurons. Second, the conditional ablation of Rax in a subset of VMH progenitors using a Shh::Cre driver leads to a fate switch from a VMH neuronal phenotype to a hypothalamic but non-VMH identity, suggesting that Rax is a selector gene for VMH cellular fates. Finally, the broader elimination of Rax throughout ARC/VMH progenitors using Six3::Cre leads to a severe loss of both VMH and ARC cellular phenotypes, demonstrating a role for Rax in both VMH and ARC fate specification. Combined, our study illustrates that Rax is required in ARC/VMH progenitors to specify neuronal phenotypes within this hypothalamic brain region. Rax thus provides a molecular entry point for further study of the ontology and establishment of hypothalamic feeding circuits.