Communicating with one's environment requires efficient neural interaction between action and perception. Neural substrates of sound perception and production are connected by the arcuate fasciculus (AF). Although AF is known to be involved in language, its roles in non-linguistic functions are unexplored. Here, we show that tone-deaf people, with impaired sound perception and production, have reduced AF connectivity. Diffusion tensor tractography and psychophysics were assessed in tone-deaf individuals and matched controls. Abnormally reduced AF connectivity was observed in the tone deaf. Furthermore, we observed relationships between AF and auditory-motor behavior: superior and inferior AF branches predict psychophysically assessed pitch discrimination and sound production perception abilities, respectively. This neural abnormality suggests that tone deafness leads to a reduction in connectivity resulting in pitch-related impairments. Results support a dual-stream anatomy of sound production and perception implicated in vocal communications. By identifying white matter differences and their psychophysical correlates, results contribute to our understanding of how neural connectivity subserves behavior.