Dark matter (DM) could be a relic of freeze-in through a light mediator, where the DM is produced by extremely feeble, IR-dominated processes in the thermal standard model plasma. In the simplest viable models with DM lighter than 1 MeV, the DM has a small effective electric charge and is born with a nonthermal phase-space distribution. This DM candidate would cause observable departures from standard cosmological evolution. In this work, we combine data from the cosmic microwave background (CMB), Lyman-α forest, quasar lensing, stellar streams, and Milky Way satellite abundances to set limits on freeze-in DM masses up to ∼20 keV, with the exact constraint depending on whether the DM thermalizes in its own sector. We perform forecasts for the CMB-S4 experiment, the Hydrogen Epoch of Reionization Array, and the Vera Rubin Observatory, finding that freeze-in DM masses up to ∼80 keV can be explored.