The identification of specific functional roles for the numerous long noncoding (nc)RNAs found in eukaryotic transcriptomes is currently a matter of intense study amid speculation that these ncRNAs have key regulatory roles. We have identified a pair of cis-interfering ncRNAs in yeast that contribute to the control of variegated gene expression at the FLO11 locus by implementing a regulatory circuit that toggles between two stable states. These capped, polyadenylated ncRNAs are transcribed across the large intergenic region upstream of the FLO11 ORF. As with mammalian long intervening (li)ncRNAs, these yeast ncRNAs (ICR1 and PWR1) are themselves regulated by transcription factors (Sfl1 and Flo8) and chromatin remodelers (Rpd3L) that are key elements in phenotypic transitions in yeast. The mechanism that we describe explains the unanticipated role of a histone deacetylase complex in activating gene expression, because Rpd3L mutants force the ncRNA circuit into a state that silences the expression of the adjacent variegating gene.