The novel Rfm3 locus causing undesired fertility restoration in the msm1 cytoplasm of winter barley is located on the short arm of chromosome 6H. Undesired fertility restoration of cytoplasmic male sterile (CMS) mother lines in absence of the functional Rfm1 restorer gene is a significant problem for hybrid breeding in winter barley. Here, we describe that a novel restorer locus on the short arm of chromosome 6H, designated Rfm3, is closely linked to two mitochondrial transcription termination factor family (mTERF) protein coding genes. Genome-wide association studies in a multiparental mapping population revealed that two of the most significantly associated markers are located very close to these genes, with one marker lying directly within one mTERF gene sequence. Sequences of the candidate genes in the parental lines, segregating individuals and an independent set of breeding lines clearly revealed haplotypes discriminating completely sterile, partially fertile and Rfm1-restorer lines. The haplotypes segregate for several single nucleotide polymorphisms, a 6 bp insertion-deletion (InDel) polymorphism and another 2 bp InDel. CMS-unstable genotypes carrying haplotypes associated with undesired fertility restoration showed significantly higher grain setting on bagged spikes when plants were subjected to elevated temperatures during anthesis, indicating a temperature influence on pollen fertility. SNPs associated with desirable Rfm3 haplotypes can be implemented in marker-assisted selection of stable CMS mother lines.