We propose a proton range verification technique for passive scattering proton therapy systems where spread out Bragg peak (SOBP) fields are produced with rotating range modulator wheels. The technique is based on the correlation of time patterns of the prompt gamma ray emission with the range of protons delivering the SOBP. The main feature of the technique is the ability to verify the proton range with a single point of measurement and a simple detector configuration. We performed four-dimensional (time-dependent) Monte Carlo simulations using TOPAS to show the validity and accuracy of the technique. First, we validated the hadronic models used in TOPAS by comparing simulations and prompt gamma spectrometry measurements published in the literature. Second, prompt gamma simulations for proton range verification were performed for the case of a water phantom and a prostate cancer patient. In the water phantom, the proton range was determined with 2 mm accuracy with a full ring detector configuration for a dose of ~2.5 cGy. For the prostate cancer patient, 4 mm accuracy on range determination was achieved for a dose of ~15 cGy. The results presented in this paper are encouraging in view of a potential clinical application of the technique.