Plasticity in the brain reaches far beyond a mere changing of synaptic strengths. Recent time-lapse imaging in the living brain reveals ongoing structural plasticity by forming or breaking of synapses, motile spines, and re-routing of axonal branches in the developing and adult brain. Some forms of structural plasticity do not follow Hebbian- or anti-Hebbian paradigms of plasticity but rather appear to contribute to the homeostasis of network activity. Four decades of lesion studies have brought up a wealth of data on the mutual interdependence of neuronal activity, neurotransmitter release and neuronal morphogenesis and network formation. Here, we review these former studies on structural plasticity in the context of recent experimental studies. We compare spontaneous and experience-dependent structural plasticity with lesion-induced (reactive) structural plasticity that occurs during development and in the adult brain. Understanding the principles of neural network reorganization on a structural level is relevant for a deeper understanding of long-term memory formation as well as for the treatment of neurological diseases such as stroke.