Maximally efficient prediction in the early fly visual system may support evasive flight maneuvers

Abstract

The visual system must make predictions to compensate for inherent delays in its processing. Yet little is known, mechanistically, about how prediction aids natural behaviors. Here, we show that despite a 20-30ms intrinsic processing delay, the vertical motion sensitive (VS) network of the blowfly achieves maximally efficient prediction. This prediction enables the fly to fine-tune its complex, yet brief, evasive flight maneuvers according to its initial ego-rotation at the time of detection of the visual threat. Combining a rich database of behavioral recordings with detailed compartmental modeling of the VS network, we further show that the VS network has axonal gap junctions that are critical for optimal prediction. During evasive maneuvers, a VS subpopulation that directly innervates the neck motor center can convey predictive information about the fly's future ego-rotation, potentially crucial for ongoing flight control. These results suggest a novel sensory-motor pathway that links sensory prediction to behavior.

Fig 1. Predictive information is the dominant information source about visual inputs during evasive flight maneuvers.

(A) Upon emergence of a threat (shown as the red star), dashed arrow represents the visual-motor delay of 60 ms from the onset of threat to the start of evasive maneuvers. After this sensory-motor delay, the position of the threat is known. The fly performs an evasive maneuver by changing its heading through a banked turn (arrows show a rotation at direction $\overrightarrow{\theta }$ and its respective counterrotation). During evasive maneuvers, visual predictions can provide motion information throughout the entire duration, i.e. without delay (shown as the yellow zone), whereas the haltere feedback is only available after 20 ms (shown as the green zone) and the visual feedback is only available after 30 ms (shown as the shaded zone). The arrow leading to the haltere system illustrates how visual information might regulate haltere activity (as recently shown in [13]): because of the 30 ms sensory processing lag, haltere activity must be regulated by visual prediction. (3D fly rendering courtesy of D. Allan Drummond.) (B) This histogram compares how much information the visual prediction (shown in blue) can encode about ego-rotation (I(θ, V)) during evasive maneuvers with their respective entropy (shown as S(θ) in gray). We use the ego-rotation distribution at Δt = 10ms into the evasive maneuver to compute this entropy. Its distribution is shown in S2(A) Fig. Note that the VS output encodes almost half of the entropy of a future ego-rotation. (C) The Mercator map of a randomly generated natural scene background. To generate this map, we first randomly generate a natural scene environment. We then generate a movie mimicking an evasive flight in the natural environment by rotating this natural scene environment according to the respective measured rotations. We project this movie to a unit sphere that represents the fly’s retina, see details in S1 Fig. There are 5,000 local motion detectors (LMD) on this unit sphere as on the fly’s retina. The responses of these LMDs are then integrated as the input current $\mathcal{I}$ to the VS network (shown as an arrow to D). (D) A biophysically detailed model of the VS network, based on known neural circuitry [50, 51]. Note that because the soma is decoupled in VS cells (only connecting to the rest of the cell via a soma fiber), we leave out the soma in this VS model. We highlight the outputs to the neck motor center here, the axonal voltages of the VS 5-6-7 triplet. This is the only known readout that directly connects to motor pathways. (E) A cartoon showing how the information bottleneck problem is setup for prediction in this system: using the general correlation between a past input (either the ego-rotation $\overrightarrow{\theta }$ or the corresponding dendritic input $\mathcal{I}$, this information bottleneck finds a compact representation V (i.e. the bottleneck defines how much information about the input is ‘squeezed out’ when V is generated) of the past (the past input to the VS network: ${\mathcal{I}}_{\text{past}}$) that retains predictive components about the future (θ_future or ${\mathcal{I}}_{\text{future}}$).

Fig 2. The capacity of the VS network to encode predictive information varies with the anatomical locations of the gap junction between VS cells.

A) The predictive information about the future input current, I_future encoded in four different schemes: 1) the past dendritic input current (solid line, this is the limit $I_{\text{future}}^{\text{max}}$. It is also the upper bound of I_future), 2) the past axonal voltage when the gap junctions are present between VS axons (dashed line), 3) when the gap junctions are present between VS dendrites (dotted line) and 4) in the absent of gap junctions (dash-dotted line). All gap junctions = 1000 nS for both settings when they are present. Only their locations differ, i.e. axon vs. dendrite. Note that when the gap junctions are present between VS cell axons, the output voltages preserve almost the entire amount of the predictive information available at the inputs (red). (See also Materials and methods.) Such encoding is not because of linear correlation. As shown in, there is negligible linear correlation between the past and future egorotation at Δt = 30ms or Δt = 40ms. B) The presence of axonal GJs improves the encoding of predictive information more than instantaneous input. We compare how the VS network encodes the predictive information in two scenarios (i.e. I_future and I_future(θ, Δt) with Δt = 10 ms) and the instantaneous egomotion θ with axonal GJs (cyan bars) and without GJs (red bars). The encoding of instantaneous constant egomotion I(θ_t; V_t) (without prediction forward in time) is compiled from the previous work [46]. I(θ_t; V_t) was defined as the mutual information between a constant rotation θ and the transient axonal voltages of the VS network (integrated for Δt = 10 ms).

Fig 3. Near-optimal prediction of the input to the VS network.

(A) The encoding of predictive information about the future current input to the VS network is near-optimal 10 ms after evasive maneuvers starts (Δt = 10 ms). Such performance is present for using both the entire VS network and the triplets. The dark blue curve traces out optimum encoding of future input to the VS network given varying amounts of information retained about the past input (also see Materials and methods). This curve also divides the plane into allowed (blue shaded region) and forbidden regions. No encoding can exist in the forbidden region because it cannot have more information about its future inputs than the input correlation structure allows, given causality and the data processing inequality. In addition, the maximal amount of information (shown as the highest point of the information curve) that is available as predictive information is limited by the correlation structure of the input (current), itself. We then plot the amount of information the axonal voltages of VS network (we show with axonal gap junctions in pink and without gap junctions in black) encode about the future input (the input current at time t + Δt) versus the information they retain about the past input (the input current at time t) (with all 120 triplets (crosses) and the whole network (circle)). The information efficiency, compared to the bound, contained in a particular encoding scheme corresponds to a single point in this (I_past, I_future) plane, which shows how much information it encodes about the past input vs. how much it encodes about the future. A particular VS encoding could occupy any point within the blue shaded region, but those that get close to the bound $I_{\text{future}}^{*}(\mathcal{I},\Delta t)$ for a particular I_past are the maximally informative predictors of the future input. (B) I_future for all VS (light bars) vs. triplets (dark bars, with error bars) throughout the time span of evasive maneuvers (Δt = 10 ms, 20 ms, 30 ms, 40 ms).

Fig 4. Encodings based on the axonal voltages of triplets are near-optimal in predicting the future ego-rotation.

(A) Histogram showing that the triplets (we use the output triplet VS 5-6-7 triplet here) encode nearly as much information about the future ego-rotation (shown in dark bars) vs. the entire VS network (shown in light bars), throughout evasive maneuvers. The solid line shows the mutual information within ego-rotation themselves: between the prior heading upon the detection of visual threat θ_past and different future ego-rotation throughout evasive maneuvers. The dashed line shows the mutual information between past input to the VS network and different future ego-rotation, all using the same past input ${\mathcal{I}}_{\text{past}}$ corresponding to the previous θ_past. This is also the limit of prediction in the information bottleneck framework. (B) The encoding of predictive information for the θ_future at 10 ms after the start of evasive maneuvers (Δt = 10 ms). The dark blue curve traces out the optimum encoding for the future ego-rotation (I(V_past; θ_future)) given varying amounts of information retained about the past input ($I(V_{past};{\mathcal{I}}_{\text{past}})$). The cyan cross corresponds to how much information each of all possible 120 triplets encodes about the future ego-rotation vs. how much information they retain from the past input.

Fig 5. The predictive information encoded by the VS network supports fine scale discrimination of future ego-rotation.

(A) The predictive representation of four future ego-rotations in the same quadrant of roll and pitch, e.g. an up-tilt and a clockwise roll. This representation maps the axonal voltage of the entire VS network to future ego-rotation through a latent feature space. The dimensions in this latent feature space (shown as VIB D1 and VIB D2) are VIB-learned predictive features based on the output of the VS network. All ego-rotation correspond to vectors within the 1st quadrant of the fly’s coronal plane. The inset shows a polar histogram in grey and the four selected ego-rotations in color. (B) Similar to A but using the axonal voltages of the VS 5-6-7 triplet. (C) Similar to A, but ego-rotation are all counter-clockwise roll and up-tilt, corresponding to vectors in the 4th quadrant (between 270° and 360°) of the fly’s coronal plane. (D) Similar to C, but obtained using the axonal voltages of the VS 5-6-7 triplet as the VIB input. Note that although the overall correlation is high for the VIB solution using the axonal voltages of the VS 5-6-7 triplet, the VIB D1 and VIB D2 encode different information about θ_future: VIB D1 encodes 1 bits about θ_future and VIB D2 encodes an additional 0.3 bits.