The effect of quenched sequence disorder on the thermodynamics of RNA secondary structure formation is investigated for two- and four-letter alphabet models using the constrained annealing approach, from which the temperature behavior of the free energy, specific heat, and helicity is analytically obtained. For competing base pairing energies, the calculations reveal reentrant melting at low temperatures, in excellent agreement with numerical results. Our results suggest an additional mechanism for the experimental phenomenon of RNA cold denaturation.