Our previous studies have shown that the peripheral auditory system of the toadfish encodes the direction of a sound source. Here, we compare directional responses of peripheral saccular afferents, cells in the descending octaval nucleus (DON) of the medulla, and the torus semicircularis (TS) of the midbrain. Recording locations in the brain were labeled with neurobiotin to confirm the site. To compare directional responses among cells, we calculated an index [sharpening ratio (SR)] that weights the relative strength of responses to the best direction for that cell and to the adjacent stimulus angles tested. Unsharpened saccular afferents tend to have a cosinusoidal directional response pattern (DRP) with an expected SR of 0.87. In DON, more than 60% of the cells exhibited directional sharpening (defined as SR <0.8). In TS, more than 80% of the cells exhibited directional sharpening. We conclude that directional auditory sharpening first occurs in DON and some additional sharpening occurs in the ascending pathway to the midbrain, particularly in azimuth. The sharpening of directional selectivity is likely to be an important component of the neural computations underlying directional hearing.