Human development is thought to evolve from the dynamic interchange of biological dispositions and environmental provisions; yet the effects of specific biological and environmental birth conditions on the trajectories of cognitive and social-emotional growth have rarely been studied. We observed 126 children at six time-points from birth to 5 years. Intelligence, maternal sensitivity, and child social engagement were repeatedly tested. Effects of neonatal vagal tone (VT) and maternal postpartum depressive symptoms on growth-rates were assessed. Cognitive development showed a substantial growth-spurt between 2 and 5 years and social engagement increased rapidly across the first year and more gradually thereafter. VT improved cognitive and social-emotional growth-rates across the first year, whereas maternal depressive symptoms interfered with growth from 2 to 5 years. Differences between infants with none, one, or two non-optimal birth conditions increased with age. Findings shed light on the dynamics of early development as it is shaped by biological and environmental initial conditions.