Optimal foraging and the information theory of gambling

Abstract

At a macroscopic level, part of the ant colony life cycle is simple: a colony collects resources; these resources are converted into more ants, and these ants in turn collect more resources. Because more ants collect more resources, this is a multiplicative process, and the expected logarithm of the amount of resources determines how successful the colony will be in the long run. Over 60 years ago, Kelly showed, using information theoretic techniques, that the rate of growth of resources for such a situation is optimized by a strategy of betting in proportion to the probability of pay-off. Thus, in the case of ants, the fraction of the colony foraging at a given location should be proportional to the probability that resources will be found there, a result widely applied in the mathematics of gambling. This theoretical optimum leads to predictions as to which collective ant movement strategies might have evolved. Here, we show how colony-level optimal foraging behaviour can be achieved by mapping movement to Markov chain Monte Carlo (MCMC) methods, specifically Hamiltonian Monte Carlo (HMC). This can be done by the ants following a (noisy) local measurement of the (logarithm of) resource probability gradient (possibly supplemented with momentum, i.e. a propensity to move in the same direction). This maps the problem of foraging (via the information theory of gambling, stochastic dynamics and techniques employed within Bayesian statistics to efficiently sample from probability distributions) to simple models of ant foraging behaviour. This identification has broad applicability, facilitates the application of information theory approaches to understand movement ecology and unifies insights from existing biomechanical, cognitive, random and optimality movement paradigms. At the cost of requiring ants to obtain (noisy) resource gradient information, we show that this model is both efficient and matches a number of characteristics of real ant exploration.

Keywords: Bayesian methods; Lévy foraging; Markov chain Monte Carlo; collective behaviour; movement ecology.

Figure 1.

A comparison of the Kelly strategy with an expected return matching strategy, over the long term (identical one-step pay-offs for a ‘win' in both cases). (a) The proportion of ants ‘bet' (yellow bars) matches the probability of success (grey). (b) The proportion of ants is allocated by the expected return (probability × pay-off). The Kelly strategy increasingly outperforms any other strategy as time goes by ((c), example simulation). (Online version in colour.)

Figure 2.

Performance of the M–H model as it generates a sample distribution q that approximates the target distribution p, the location of resources in the environment. The minimum cross-entropy, where q = p, is shown as a dotted line. (Online version in colour.)

Figure 3.

Performance of the HMC and PMR models, compared with that of M–H. In general, HMC and PMR outperform M–H because random walk type exploration of probability space is avoided, by following local gradient information and making larger steps. Their performance depends on the nature of the target distribution and choosing suitable values for step length ε and number of steps L. (Online version in colour.)

Figure 4.

Comparison of example trajectories from real ants (100 s) and for the three MCMC models (100 simulated time steps). The model trajectories become increasingly superdiffusive.

Figure 5.

The distribution of direction changes between steps in real ants (N = 18) and the three MCMC models (simulated for N = 18 ‘ants' for 1000 time steps). (Online version in colour.)

Figure 6.

The (apparently) power law distributed step lengths for both real ants and simulated PMR walkers. (Online version in colour.)