An efficient coding hypothesis links sparsity and selectivity of neural responses

Abstract

To what extent are sensory responses in the brain compatible with first-order principles? The efficient coding hypothesis projects that neurons use as few spikes as possible to faithfully represent natural stimuli. However, many sparsely firing neurons in higher brain areas seem to violate this hypothesis in that they respond more to familiar stimuli than to nonfamiliar stimuli. We reconcile this discrepancy by showing that efficient sensory responses give rise to stimulus selectivity that depends on the stimulus-independent firing threshold and the balance between excitatory and inhibitory inputs. We construct a cost function that enforces minimal firing rates in model neurons by linearly punishing suprathreshold synaptic currents. By contrast, subthreshold currents are punished quadratically, which allows us to optimally reconstruct sensory inputs from elicited responses. We train synaptic currents on many renditions of a particular bird's own song (BOS) and few renditions of conspecific birds' songs (CONs). During training, model neurons develop a response selectivity with complex dependence on the firing threshold. At low thresholds, they fire densely and prefer CON and the reverse BOS (REV) over BOS. However, at high thresholds or when hyperpolarized, they fire sparsely and prefer BOS over REV and over CON. Based on this selectivity reversal, our model suggests that preference for a highly familiar stimulus corresponds to a high-threshold or strong-inhibition regime of an efficient coding strategy. Our findings apply to songbird mirror neurons, and in general, they suggest that the brain may be endowed with simple mechanisms to rapidly change selectivity of neural responses to focus sensory processing on either familiar or nonfamiliar stimuli. In summary, we find support for the efficient coding hypothesis and provide new insights into the interplay between the sparsity and selectivity of neural responses.

Figure 1. Efficient coding of cochlear spectrograms.

(A) Schematic of the auditory pathway in the songbird forebrain. Auditory input to the pallial field L is provided by the thalamic nucleus ovoidalis (Ov). From field L, auditory information is relayed to the caudal mesopallium (CM), and from there to the nucleus interface of the nidopallium (NIf) and to HVC. (B) Network model. At time , the auditory input to the network is a 50-ms window of the sound spectrogram. This input is multiplied by synaptic weights to result in total synaptic currents onto neurons. stands for whitening and dimensionality reduction (principal component analysis), and stands for a sparseness transformation. Neural firing rates are given by rectified synaptic currents. (C) Cost function for a threshold . Subthreshold synaptic currents are punished quadratically and suprathreshold currents are punished linearly. The parameter defining the subthreshold current eliciting minimal cost is set to the expected subthreshold current . Synaptic currents are reported in units of mean-subtracted standard deviations (z-scores). A threshold implies that suprathreshold currents are depolarizing (positive), whereas subthreshold currents are hyperpolarizing (negative). .

Figure 2. Spectral temporal receptive fields.

(A)Spectral temporal receptive fields (STRFs) of neurons, arranged by nearest-neighbor similarity (circular boundary conditions). Neurons tend to be either temporally tuned (vertical stripes, top right), spectrally tuned (horizontal stripes, middle rows), or display more complex spectro-temporal patterns. Spectral resolution is 172 Hz, offset between subsequent cochlear inputs is 1.5 ms. (B) STRFs obtained with a linear cost on synaptic weight magnitudes. The linear cost forces many synaptic weights to be close to zero (green), leading to low-density STRFs most of which contain a smaller number of excitatory and inhibitory subfields than in A. Interestingly, excitatory and inhibitory subfields tend to be close to each other and aligned horizontally or vertically, similar to observations in field L neurons. The 100 presented STRFs were randomly chosen out of the total 800. .

Figure 3. Receptive fields and neurogram.

(A) Power spectrogram of a bird's own song (BOS). (B) STRFs of eight representative neurons (). The horizontal alignment of STRFs with the spectrogram in A is such that the trailing edges of the STRFs correspond to the respective peak times of synaptic currents. The temporal axis of the STRFs is inverted for better comparison with the BOS spectrogram. (C) Stack plot of synaptic currents of representative neurons in B in response to ten different versions of BOS, vertically aligned to A. (D) Neurogram of synaptic currents in response to the BOS in A. The neurons are sorted according to the peak times of their synaptic currents. Fewer neurons display synaptic current peaks during syllable gaps (blue arrows) than during syllables.

Figure 4. Neurons encode behavioral variability, for example song pitch.

(A) Two receptive fields formed by training a network on all songs produced by a bird on a single day. (B) Spectrograms of a song syllable containing a harmonic stack. The left version has median pitch 1024 Hz, the right version 1138 Hz. (C) Stack plot of synaptic currents in the two neurons elicited by 813 syllable renditions. The stack plots have been sorted identically to reveal that for a given syllable rendition either the left or right neuron exhibits a peak in synaptic current, but not both. Peaks in synaptic currents are computed in intervals indicated by red bars on the bottom. (D) Scatter plot of peak synaptic currents in the two neurons. The distribution is sparse (‘L’-shaped). (E) Median synaptic current in same intervals versus median song pitch of the harmonic stack. The two neurons are detectors of low and high pitch versions of the stack, respectively. Red and blue lines are linear regressions (Neuron 1: , , Neuron 2: , ), , .

Figure 5. Actual STRFs and STRFs estimated using reverse correlation.

(A) A selection of twelve STRFs obtained after convergence of the algorithm (). (B) Estimated STRFs (reverse correlation) based on the predicted firing rates . Shown are only estimated STRFs for neurons associated with a correlation coefficient between predicted and actual firing rates of . . .

Figure 6. Probability density of total synaptic currents.

(A) The probability density of total synaptic currents averaged over all neurons has a heavy tail on the positive side. Shown are the densities for BOS (blue), CON (green), and REV (black). Near zero synaptic currents, the curves are approximatively unit Gaussian (red), though their excessive peaks are slightly shifted to the negative side (inset, arrow). The curves cross each other such that large positive synaptic currents are preferentially elicited by the BOS and small positive currents by REV and CON. , . (B) The distributions of synaptic currents for sparse STRFs (Figure 2B) are qualitatively similar to (A). The only noticeable difference is that the distribution for REV is closer to BOS, reflecting a lower selectivity for temporal order. .

Figure 7. Probability densities of mean firing rates.

Mean firing rates in response to (A) a BOS stimulus and (B) a CON stimulus. For each cell we computed the mean firing rate to one stimulus trial. Our simulation data (blue asterisks) are better fit by log-normal densities (red) than by exponential densities (black). Firing rates are plotted in arbitrary units. Fit parameters for log-normal densities were determined by the mean and variance of logarithmic firing rates, and for exponential densities they were determined by the mean firing rates. Thresholds varied from to . Noise amplitude .

Figure 8. Sparsification reduces firing dependences.

(A) The sparseness transformation renders the size distribution of coactive neuron groups (whitening+sparseness) closer to binomial. The probability of the binomial distribution (that a neuron is active per unit time) was estimated in terms of the firing density (the fraction of suprathreshold events over all neurons and training stimuli). Probabilities were nearly identical for whitening and whitening+sparseness when ( and during learning). (B) The Kullback–Leibler divergence between size distributions is smaller when comparing the whitening+sparseness model to the binomial model than when comparing the whitening model to the binomial model, for nearly all firing densities tested.

Figure 9. Reconstructing the cochlear spectrograms from firing rates.

(A) Firing-rate of one example neuron in response to BOS for increasing firing thresholds ( to ). The BOS spectrogram is shown on top. This neuron is tuned to a feature present in introductory notes and responds to it up to thresholds higher than seven. For each threshold, ten different responses are plotted, corresponding to ten different instantiations of synaptic noise. . (B) The reconstruction of a BOS spectrogram (orig., top) using all neurons, based on a firing threshold of minus infinity (whi., 2nd from top) is fairly complete with little information loss (arising from dimensionality reduction). With increasing thresholds (below), more and more syllables are lost in the reconstruction, but the reconstructed spectro-temporal patterns remain clearly recognizable. The arrow points to a down-sweep syllable. (C) Reconstructions of REV (flipped horizontally for comparison with B) are worse than reconstructions of BOS at the same threshold; for example the down-sweep syllable is not well reconstructed (arrow), presumably because zebra finches produce almost no up-sweeps. (D) The fraction of active neurons (averaged over all BOS stimuli) decreases with increasing threshold such that at about 1% of neurons are active on average. This fraction decreases to 0.1% at about . (E) The reconstruction errors averaged over different stimulus ensembles are monotonic functions of the firing threshold. For a given positive threshold, reconstruction errors increase from BOS to CON to REV. threshold-linear neurons.

Figure 10. Smart suppression of electrical noise affecting the recordings.

(A) The STRFs of five neurons that encoded monitor noise. (B) Original BOS spectrogram. The noise is manifest as gray horizontal bands (black arrows) during syllable gaps. (C) Reconstruction of the BOS (Equation 14) from elicited responses in the network. The thresholds of all neurons were set to , with exception of the five neurons in A in which the thresholds were set to . The monitor noise has vanished in the reconstructions, without affecting the birdsong signal.

Figure 11. selectivity for BOS reverses at high firing thresholds.

(A) Example model neuron with reversing BOS-CON selectivity. This neuron's STRF (inlay) codes for an up-sweep from 500 to 800 Hz over 20 ms. The resulting BOS-CON selectivity is negative for low thresholds and turns positive for thresholds . (B) Example cumulative distributions of BOS-REV (blue) and BOS-CON (red) selectivities across neurons for (solid lines) and (dashed lines). For the selectivities are biased towards negative values, whereas for the distributions are biased towards positive values. (C) Bar plot summarizing BOS-REV (blue) and BOS-CON (red) selectivities for a wide range of firing thresholds. The colored bars indicate the median selectivity and the error bars delimit the first and third quartiles. Selectivity reverses at around (REV) and (CON).

Figure 12. Selectivity for CON vs. different artificial stimuli.

Depicted are median selectivities quartiles. PIP = tone-pip stimuli, WN = white noise.

Figure 13. Selectivity in two layers.

Median and quartile selectivities in a network of two layers for various firing thresholds in the first layer and in the second layer. In each simulation, second-layer responses were evaluated using the first-layer threshold applied during training. As can be seen, BOS preference in the second layer is restricted to the high-sparseness regime there (right part of the three subplots).