Hyperpolarization-activated cyclic nucleotide-gated (HCN) channels are activated by membrane hyperpolarizations that cause an inward movement of the positive charges in the fourth transmembrane domain (S4), which triggers channel opening. The mechanism of how the motion of S4 charges triggers channel opening is unknown. Here, we used voltage clamp fluorometry (VCF) to detect S4 conformational changes and to correlate these to the different activation steps in spHCN channels. We show that S4 undergoes two distinct conformational changes during voltage activation. Analysis of the fluorescence signals suggests that the N-terminal region of S4 undergoes conformational changes during a previously characterized mode shift in HCN channel voltage dependence, while a more C-terminal region undergoes an additional conformational change during gating charge movements. We fit our fluorescence and ionic current data to a previously proposed 10-state allosteric model for HCN channels. Our results are not compatible with a fast S4 motion and rate-limiting channel opening. Instead, our data and modeling suggest that spHCN channels open after only two S4s have moved and that S4 motion is rate limiting during voltage activation of spHCN channels.