A Neurocomputational Model of Goal-Directed Navigation in Insect-Inspired Artificial Agents

Abstract

Despite their small size, insect brains are able to produce robust and efficient navigation in complex environments. Specifically in social insects, such as ants and bees, these navigational capabilities are guided by orientation directing vectors generated by a process called path integration. During this process, they integrate compass and odometric cues to estimate their current location as a vector, called the home vector for guiding them back home on a straight path. They further acquire and retrieve path integration-based vector memories globally to the nest or based on visual landmarks. Although existing computational models reproduced similar behaviors, a neurocomputational model of vector navigation including the acquisition of vector representations has not been described before. Here we present a model of neural mechanisms in a modular closed-loop control-enabling vector navigation in artificial agents. The model consists of a path integration mechanism, reward-modulated global learning, random search, and action selection. The path integration mechanism integrates compass and odometric cues to compute a vectorial representation of the agent's current location as neural activity patterns in circular arrays. A reward-modulated learning rule enables the acquisition of vector memories by associating the local food reward with the path integration state. A motor output is computed based on the combination of vector memories and random exploration. In simulation, we show that the neural mechanisms enable robust homing and localization, even in the presence of external sensory noise. The proposed learning rules lead to goal-directed navigation and route formation performed under realistic conditions. Consequently, we provide a novel approach for vector learning and navigation in a simulated, situated agent linking behavioral observations to their possible underlying neural substrates.

Keywords: artificial intelligence; insect navigation; neural networks; path integration; reward-based learning.

Figure 1

Schematic diagram of the modular closed-loop control for vector navigation. (A) The model consists of a neural path integration (PI) mechanism (1), reward-modulated vector learning (2), random search (3), and action selection (4). Vector information for guiding navigation is computed and represented in the activity of circular arrays. The home vector (HV) array is the output of the PI mechanism and is applied for homing behavior and as a scaffold for global vector (GV) learning. These three vector representations and random search are integrated through an adaptive action selection mechanism, which produces the steering command to the CPG-based locomotion control. (B) Spatial representation of the different vectors used for navigation. The HV is computed by PI and gives an estimate for the current location of the agent. In general, GVs connect the nest to a rewarding location. Using vector addition, the agent is able to compute, how to orient from its current location toward the feeder.

Figure 2

Multilayered neural network of the proposed path integration (PI) mechanism. (A) Sensory inputs from a compass sensor (ϕ) and odometer (s) are provided to the mechanism. (B) Neurons in the head direction (HD) layer encodes the sensory input from a compass sensor using a cosine response function. Each neuron encodes a particular preferred direction enclosing the full range of 2π. Note that the figure depicts only six neurons for simplicity. (C) An odometric sensory signal (i.e., walking speed) is used to modulate the HD signals. (D) The memory layer accumulates the signals by self-recurrent connections. (E) Cosine weight kernels decode the accumulated directions to compute the output activity representing the home vector (HV). (F) The difference between the HV angle and current heading angle is used to compute the homing signal (see Equation 11).

Figure 3

Example of vector representations based on the neural activities of the decoding layer (see Figure 2E) in the path integration (PI) mechanism for a square trajectory. The agent runs for 5 m in one of the four directions (180°, 270°, 0°, 90°), thus finally returning to the starting point of its journey. The coarse encoding of heading orientations lead to a correct decoding of memory layer activities. Thus, the activities of the decoding layer in the PI mechanism (see inlay) represent the home vector (HV), where the position of the maximum firing rate is the angle and the amplitude of the maximum firing rate is the length of the vector. Note that, as the agent returns to the home position, the output activities are suppressed to zero resulting from the elimination of opposite directions.

Figure 4

Canonical vector learning rule involves associations of path integration (PI) states with context-dependent and reward signals. Global vector memories are acquired and expressed by this learning circuit. The home vector array activities are associated with the food reward given an active foraging state (outward journey). For details, see text below.

Figure 5

Path integration (PI) accuracy under the influence of external noise. (A) Example trajectories of the simulated agent during random foraging (light gray) and homing behavior (dark gray) for different sensory, correlated noise levels: 1, 2, and 5%. The red point marks the starting point at the nest, and the blue point indicates the return, when the agent switches to its inward state. Using only path integration, the agent successfully navigates back to the nest with a home radius (green circle) of 0.2 m. (B) We evaluate the accuracy of the proposed PI mechanism by using the mean positional error averaged over each time step during each trial. Distribution of positional errors for different sensory, correlated noise levels: 1, 2, and 5%. (C) Examples of population-coded HD activities with correlated and uncorrelated noise. Filled dots are activities of individual neurons, while the dashed line is a cosine response function. (D) Mean position errors 〈δr〉 (± S.D.) in PI with respect to fully correlated, sensory noise levels averaged over 1,000 trials (fixed number of 18 neurons per layer). (E) Mean position errors 〈δr〉 (± S.D.) in PI with respect to uncorrelated, neural noise levels averaged over 1,000 trials (fixed number of 18 neurons per layer).

Figure 6

Mean positional errors 〈δr〉 (± S.D.) in path integration (PI) with respect to number of neurons per layer averaged over 1, 000 trials for three different sensory noise level (0, 2, and 5%). In all three cases, the error reaches a minimum plateau between 16 and 32 neurons (colored area), which corresponds to the number of functional columns in the ellipsoid body of the insect central complex (Wolff et al., 2015).

Figure 7

Systematic errors δθ of desert ant homing are reproduced by leaky integration of path segments. Müller and Wehner (1988) tested the ants how accurate they return to the nest after following the two connected, straight channels with 10 and 5 m length to the feeder (sketch modified from Müller and Wehner, 1988). The second channel angle α was varied in 2.5° intervals for the simulation results. In our model, the leak rate λ in the self-recurrent connections is used to fit the behavioral data (Müller and Wehner, 1988). We found that values λ ≈ 0.0075 accurately describe the observed systematic errors in desert ants.

Figure 8

Learning walks of the simulated agent for a feeder placed L_feed = 10 m away from the nest. (A) Trajectories of the agent for five trials with a feeder in 10 m distance and 90° angle to the nest. Each trial number is color-coded (see colorbar). Inward runs are characterized by straight paths controlled only by PI. See text for details. (B) Synaptic strengths of the GV array changes due to learning over time (of the five trials). The estimated angle θ_GV (cyan-colored solid line) to the feeder is given by the position of the maximum synaptic strength. (C) Exploration rate and food reward signal with respect to time. The exploration rate decreases as the agent repeatedly visits the feeder and receives reward.

Figure 9

Longer foraging durations during global vector (GV) learning increase the average goal success rate, but decrease the ratio of learned global vector and nearest feeder distance. (A) Mean exploration rate and running mean goal success and homing rate (± S.D.) with respect to trials averaged over 100 cycles of randomly generated environments (foraging time t_forage = 1, 000 s). Goal success is defined by whether a feeder was visited per trial. The homing rate is determined by the agent's return to the nest within the given total trial duration T. (B) Mean goal success rate after 100 trials with respect to foraging time t_forage averaged over 100 cycles. (C) Mean ratio of learned GV distance and nearest feeder distance with respect to foraging time t_forage averaged over 100 cycles.