Biochemical reaction networks are subjected to large fluctuations attributable to small molecule numbers, yet underlie reliable biological functions. Thus, it is important to understand how regularity can emerge from noise. Here, we study the stochastic dynamics of a self-repressing gene with arbitrarily long or short response time. We find that when the mRNA and protein half-lives are approximately equal to the gene response time, fluctuations can induce relatively regular oscillations in the protein concentration. To gain insight into this phenomenon at the crossroads of determinism and stochasticity, we use an intermediate theoretical approach, based on a moment-closure approximation of the master equation, which allows us to take into account the binary character of gene activity. We thereby obtain differential equations that describe how nonlinearity can feed-back fluctuations into the mean-field equations to trigger oscillations. Finally, our results suggest that the self-repressing Hes1 gene circuit exploits this phenomenon to generate robust oscillations, inasmuch as its time constants satisfy precisely the conditions we have identified.