The local structure of liquid water as a function of temperature is a source of intense research. This structure is intimately linked to the dynamics of water molecules, which can be measured using Raman and infrared spectroscopies. The assignment of spectral peaks depends on whether they are collective modes or single-molecule motions. Vibrational modes in liquids are usually considered to be associated to the motions of single molecules or small clusters. Using molecular dynamics simulations, here we find dispersive optical phonon-like modes in the librational and OH-stretching bands. We argue that on subpicosecond time scales these modes propagate through water's hydrogen-bond network over distances of up to 2 nm. In the long wavelength limit these optical modes exhibit longitudinal-transverse splitting, indicating the presence of coherent long-range dipole-dipole interactions, as in ice. Our results indicate the dynamics of liquid water have more similarities to ice than previously thought.