The observation of strongly interacting many-body phenomena in atomic gases typically requires ultracold samples. Here we show that the strong interaction potentials between Rydberg atoms enable the observation of many-body effects in an atomic vapor, even at room temperature. We excite Rydberg atoms in cesium vapor and observe in real time an out-of-equilibrium excitation dynamics that is consistent with an aggregation mechanism. The experimental observations show qualitative and quantitative agreement with a microscopic theoretical model. Numerical simulations reveal that the strongly correlated growth of the emerging aggregates is reminiscent of soft-matter type systems.