The late development of fast brain activity in infancy restricts initial processing abilities to slow information. Nevertheless, infants acquire the short-lived speech sounds of their native language during their first year of life. Here, we trace the early buildup of the infant phoneme inventory with naturalistic electroencephalogram. We apply the recent method of deconvolution modeling to capture the emergence of the feature-based phoneme representation that is known to govern speech processing in the mature brain. Our cross-sectional analysis uncovers a gradual developmental increase in neural responses to native phonemes. Critically, infants appear to acquire those phoneme features first that extend over longer time intervals-thus meeting infants' slow processing abilities. Shorter-lived phoneme features are added stepwise, with the shortest acquired last. Our study shows that the ontogenetic acceleration of electrophysiology shapes early language acquisition by determining the duration of the acquired units.