This study fine mapped and validated a QTL on rice chromosome 4 that increases spikelet fertility under high temperature (over 37 °C) at the flowering stage. Climate change has a negative effect on crop production and food security. Understanding the genetic mechanism of heat tolerance and developing heat-tolerant varieties is essential to cope with future global warming. Previously, we reported on a QTL (qHTSF4.1) from an IR64/N22 population responsible for rice spikelet fertility under high-temperature stress at the flowering stage. To further fine map and validate the effect of qHTSF4.1, PCR-based SNP markers were developed and used to genotype BC2F2, BC3F2, BC3F3, and BC5F2 populations from the same cross. The interval of the QTL was narrowed down to about 1.2 Mb; however, further recombination was not identified even with a large BC5F2 population that was subsequently developed and screened. The sequence in the QTL region is highly conserved and a large number of genes in the same gene family were observed to be clustered in the region. The QTL qHTSF4.1 consistently increased spikelet fertility in all of the backcross populations. This was confirmed using 24 rice varieties. Most of the rice varieties with the QTL showed a certain degree of heat tolerance under high-temperature conditions. In a BC5F2 population with clean background of IR64, QTL qHTSF4.1 increased spikelet fertility by about 15%. It could be an important source for enhancing heat tolerance in rice at the flowering stage. PCR-based SNP markers developed in this study can be used for QTL introgression and for pyramiding with other agronomically important QTLs/genes through marker-assisted selection.