Plasmon polaritons are hybrid excitations of light and mobile electrons that can confine the energy of long-wavelength radiation at the nanoscale. Plasmon polaritons may enable many enigmatic quantum effects, including lasing 1 , topological protection2,3 and dipole-forbidden absorption 4 . A necessary condition for realizing such phenomena is a long plasmonic lifetime, which is notoriously difficult to achieve for highly confined modes 5 . Plasmon polaritons in graphene-hybrids of Dirac quasiparticles and infrared photons-provide a platform for exploring light-matter interaction at the nanoscale6,7. However, plasmonic dissipation in graphene is substantial 8 and its fundamental limits remain undetermined. Here we use nanometre-scale infrared imaging to investigate propagating plasmon polaritons in high-mobility encapsulated graphene at cryogenic temperatures. In this regime, the propagation of plasmon polaritons is primarily restricted by the dielectric losses of the encapsulated layers, with a minor contribution from electron-phonon interactions. At liquid-nitrogen temperatures, the intrinsic plasmonic propagation length can exceed 10 micrometres, or 50 plasmonic wavelengths, thus setting a record for highly confined and tunable polariton modes. Our nanoscale imaging results reveal the physics of plasmonic dissipation and will be instrumental in mitigating such losses in heterostructure engineering applications.