Synapses between cochlear nerve terminals and hair cells are the most vulnerable elements in the inner ear in both noise-induced and age-related hearing loss, and this neuropathy is exacerbated in the absence of efferent feedback from the olivocochlear bundle. If age-related loss is dominated by a lifetime of exposure to environmental sounds, reduction of acoustic drive to the inner ear might improve cochlear preservation throughout life. To test this, we removed the tympanic membrane unilaterally in one group of young adult mice, removed the olivocochlear bundle in another group and compared their cochlear function and innervation to age-matched controls one year later. Results showed that tympanic membrane removal, and the associated threshold elevation, was counterproductive: cochlear efferent innervation was dramatically reduced, especially the lateral olivocochlear terminals to the inner hair cell area, and there was a corresponding reduction in the number of cochlear nerve synapses. This loss led to a decrease in the amplitude of the suprathreshold cochlear neural responses. Similar results were seen in two cases with conductive hearing loss due to chronic otitis media. Outer hair cell death was increased only in ears lacking medial olivocochlear innervation following olivocochlear bundle cuts. Results suggest the novel ideas that 1) the olivocochlear efferent pathway has a dramatic use-dependent plasticity even in the adult ear and 2) a component of the lingering auditory processing disorder seen in humans after persistent middle-ear infections is cochlear in origin.