Precise visuomotor transformations underlying collective behavior in larval zebrafish

Abstract

Complex schooling behaviors result from local interactions among individuals. Yet, how sensory signals from neighbors are analyzed in the visuomotor stream of animals is poorly understood. Here, we studied aggregation behavior in larval zebrafish and found that over development larvae transition from overdispersed groups to tight shoals. Using a virtual reality assay, we characterized the algorithms fish use to transform visual inputs from neighbors into movement decisions. We found that young larvae turn away from virtual neighbors by integrating and averaging retina-wide visual occupancy within each eye, and by using a winner-take-all strategy for binocular integration. As fish mature, their responses expand to include attraction to virtual neighbors, which is based on similar algorithms of visual integration. Using model simulations, we show that the observed algorithms accurately predict group structure over development. These findings allow us to make testable predictions regarding the neuronal circuits underlying collective behavior in zebrafish.

Fig. 1. Group structure depends on visual interactions and develops with age.

a Left: Experimental system. Multiple cameras capture the behavior of multiple groups and individuals swimming freely in separate arenas. Right: Example images and trajectories of groups of 5 larvae at 7, 14, and 21 dpf. Different colors represent different fish in the groups. b Example trajectories (recorded over 5 min) of groups of 5 larvae at age 7 dpf, swimming together in the light (top) or in total darkness (bottom). Different colors represent different fish. c Left: Normalized dispersion values for one group swimming together in the light (blue) and in the dark (black). Zero represents the average dispersion value expected when fish do not interact, and positive values represent overdispersed distributions (“Methods”). Right: groups of 7 dpf fish are more dispersed than is expected by chance (p = $8\times {10}^{-10}$, N_light = 48 groups, two-sided t test; t = 7.7) and are also more dispersed than groups swimming in total darkness (p = 0.0016, N_dark = 16 groups; two sided t test, $t=3.3$; Cohen’s d = 0.95). Bars represent mean over groups; error bars are SEM. d Left: Example dispersion values for groups at 7, 14, and 21 dpf. Right: At 14 and 21 dpf group are significantly less dispersed (more aggregated) than chance (p₁₄ = $3\times {10}^{-5}$, p₂₁ = $6\times {10}^{-5}$; N₁₄ = 21, N₂₁ = 10; two sided t test; $t_{14}=-5.3,t_{21}=-7$), and dispersion also decreases significantly over development (p = $6\times {10}^{-26}$, one-way ANOVA; $F=137$). Bars represent mean over groups; error bars are SEM. e Top: Image showing the total retinal occupancy that neighbors cast on each of the eyes of a focal fish. Bottom: Example traces of the total retinal occupancy for each of the eyes over 8 s. f Probability to turn right (per bout) as a function of the difference in retinal occupancy experienced by each eye (negative values - higher occupancy to the left). At 7 dpf, larvae tend to turn away from the more occupied side and do not respond to neighbors in total darkness (Top row). At 14 dpf, fish begin turning towards the more occupied side, and this tendency increases at 21 dpf (Bottom row). Bold lines represent turning probability calculated as the fraction of right turns out of all turns collected from all fish in 5^o bins; error bars are the 95% confidence interval of the fitted Binomial distribution to the events in each bin.

Fig. 2. Virtual reality reveals the algorithms fish use to integrate visual social information.

a Left: Testing fish social interactions using closed-loop projection of simplified moving objects mimicking neighbors (“Methods”). Right: A pinhole model of the retina is used to estimate the shape, size and position of the image of the projected object on the retina of the fish (black shapes). Both retinae are modeled as spheres, red dots are the center of the back of the retina and red lines represent the horizon line (“Methods”). b Examples of the cumulative turning angles for one fish responding to a single dot (36^o) mimicking a neighbor moving in bouts in the left visual field (red lines) and in the right one (gray lines) over 40 trials. Bold lines represent averages over trials; vertical lines represent times when stimulus is turned on and off during the trials. c Left: Probability to turn right per bout when a single moving dot of different sizes is presented to the left of the fish (N = 24 fish, ““Methods””). Right: Probability to turn away from moving dots of different sizes presented to the left of the fish, calculated over the entire stimulus duration. At 7 dpf, larvae consistently turn away from the side of the projected image. d Left: Probability to turn right per bout in response to ellipses of increasing vertical size (perpendicular to the plane of the eye), while the horizontal size remains constant at 9^o (N = 32 fish). Inset shows the image of the vertical ellipse on the retina. Right: Probability to turn right per bout in response to ellipses of increasing horizontal sizes (parallel to the plane of the eye), while the vertical size remains constant at 9^o (N = 32 fish). Inset shows the image of the horizontal ellipse on the retina. e Probability to turn right per bout in response to two images presented together to the left visual field (green line); to each of the images presented alone (blue lines) and the prediction based on the weighted average of the responses to each stimulus presented alone (red line, “Methods”), where weights represent the relative sizes of the stimuli (N = 32 fish). f Probability to turn right per bout in response to two dots presented simultaneously to each eye of the fish (green line); to each dot presented alone (blue lines) and the prediction based on the linear summation of the (competing) recorded response biases to each dot presented alone (red line, “Methods”)(N = 24 fish). In panels (c–f), probability to turn right per bout is calculated as the fraction of right turns out of all turns in 1.25 s time bins. Bold lines are average; shaded areas are SEM.

Fig. 3. Older larvae use similar algorithms to integrate visual social information.

a Left: Probability to turn right per bout in response to dots of different sizes presented to the left visual field. At 14 dpf, fish show both attraction to small angular sizes and repulsion from larger sizes (N = 32 fish). Right: Probability to turn away from dots of different sizes, presented to the left of the fish, calculated over the entire stimulus duration. Gray shaded area marks retinal occupancy values for which fish exhibit attraction. b Same as in A, only for 21 dpf larvae (“Methods”). c Left: Probability of 21 dpf larvae to turn right per bout, in response to ellipses of increasing vertical size (perpendicular to the plane of the eye), while horizontal sizes remain constant at 23^o (N = 32 fish). Right: Same but in response to ellipses of increasing horizontal sizes (parallel to the plane of the eye), while vertical sizes remain constant at 23^o (N = 32 fish). d Left: Probability to turn right per bout in response to two images presented together to the left visual field of 14 dpf larvae (green line); to each of the images presented alone (blue lines) and the prediction based on the weighted average of the responses to each stimulus presented alone (red line, “Methods”), where the weights of the responses are equal (N = 32 fish). Right: Same but for 21 dpf larvae. e Left: Probability to turn right per bout in response to two dots presented simultaneously to both eyes of 14 dpf larvae (green line); for each dot presented alone (blue lines) and the prediction based on the linear summation of the recorded response biases to each dot presented alone (red line, “Methods”)(N = 32 fish). Right: same but for 21 dpf larvae. In all panels probability to turn right per bout is calculated as the fraction of right turns out of all turns in 1.25 s time bins (“Methods”). Bold lines are average; shaded areas are SEM.

Fig. 4. Social interactions extracted from VR capture the behavior of real groups.

a Models are based on the vertical visual occupancy casted by neighbors on the retina of the focal fish and the algorithms and response functions extracted from VR experiments (Eq. 1 and “Methods”). Left: Each neighbor is represented by its estimated vertical size at specific visual angels. Middle: The vertical sizes casted by neighbors elicit a turning bias based on the age dependent response functions observed in VR experiments $p\left(turnright\mid v_i\right)$. Right: The turning biases are then averaged on each side (yellow bars) and compared between the eyes to elicit a turing probability $p\left(turnright\right)$ (black bar) for the next bout performed by the fish (“Methods”). b Examples of simulated trajectories for 7, 14, and 21 dpf larvae in arena sizes that match real group experiments. c Left: Normalized dispersion values for one simulated group of 5 fish at 7 dpf, with and without social interactions. Zero represents the expected average dispersion when fish do not interact, and positive values represent overdispersed distributions (“Methods”). Right: Simulated groups of 5 fish at 7 dpf modeled with social interactions show higher dispersion levels than chance and are also more dispersed than simulated groups modeled without social interactions. Bars represent means; error bars are SEM (N = 50 simulations). Experimental data of fish swimming in the light and in the dark is plotted for comparison (same as Fig. 1c). d Same as in (c), only for simulations of different age groups. Simulated groups switch from overdispersed to clustered groups from 7 to 21 dpf. Experimental data of different age groups is plotted for comparison (same as Fig. 1d). e Probability to turn right as a function of the difference in total angular occupancy experienced by each eye (negative values - higher occupancy to the left) in the simulations. (Similar to Fig. 1e, f for group experiments). Bold lines represent turning probability calculated as the fraction of right turns out of all turns collected from all simulated fish in 5^o bins; error bars are the 95% confidence interval of the fitted Binomial distribution to the events in each bin.

Fig. 5. Conceptual circuit model describing visuomotor transformation underlying social interactions.

a Images casted by neighboring fish on the retina of the focal fish are represented as a two-dimensional grid of retinal ganglion cells mapping the vertical height of the neighbors at different viewing angles $\Theta$. b Retinal ganglion cells selectively project to two separate populations in visual areas: the ‘Rep’ population, which ultimately elicits turning responses away from the stimulated eye, and the ‘Att’ population, which elicits turning towards the stimulated eye at older ages. c The activity of all ganglion cells at a visual angle ϴ_i are combined by specific integrating units to represent the vertical height of the stimulus at that angle. These units relay their activity to an output population (brown triangles show activation, gray triangles represent inactive connections); we show three such example units, which represent the corresponding visual angles in (a). Active integrating units also stimulate an additional inhibitory population I (red), which suppresses the output population, and performs averaging of the inputs from the integrator units (see text). d, e. Each population in the visual areas can excite (green) and inhibit (red) motor centers on the ipsi and contralateral sides. At 7 dpf (d) responses are dominated by repulsion from the stimulated eye, while at 14 and 21 dpf (e) the balance of both the attractive and repulsive populations will determine the turning direction of the fish. Line thickness represents the strength of the excitation/inhibition, gray color represents inactive connections.