We present experimental and theoretical studies on terahertz surface plasmon (TSP) propagation on slit and rectangular aperture arrays in an aluminum sheet. Terahertz waves are coupled onto the plasmonic structures via a parallel plate waveguide. Long-lasting oscillations are observed in the temporal pulse shape after propagating through the periodic structure, whose Fourier transformation into the frequency domain results in Bragg-resonance spectral features. We show that the interference between the incident wave and the radiation reflected from both the aperture array and the waveguide block is responsible for this Bragg-resonance behavior. The reflection coefficient for a single slit is deduced to be 0.017 +/- 0.002.