Dynamic analysis of learning in behavioral experiments

Abstract

Understanding how an animal's ability to learn relates to neural activity or is altered by lesions, different attentional states, pharmacological interventions, or genetic manipulations are central questions in neuroscience. Although learning is a dynamic process, current analyses do not use dynamic estimation methods, require many trials across many animals to establish the occurrence of learning, and provide no consensus as how best to identify when learning has occurred. We develop a state-space model paradigm to characterize learning as the probability of a correct response as a function of trial number (learning curve). We compute the learning curve and its confidence intervals using a state-space smoothing algorithm and define the learning trial as the first trial on which there is reasonable certainty (>0.95) that a subject performs better than chance for the balance of the experiment. For a range of simulated learning experiments, the smoothing algorithm estimated learning curves with smaller mean integrated squared error and identified the learning trials with greater reliability than commonly used methods. The smoothing algorithm tracked easily the rapid learning of a monkey during a single session of an association learning experiment and identified learning 2 to 4 d earlier than accepted criteria for a rat in a 47 d procedural learning experiment. Our state-space paradigm estimates learning curves for single animals, gives a precise definition of learning, and suggests a coherent statistical framework for the design and analysis of learning experiments that could reduce the number of animals and trials per animal that these studies require.

Figure 1.

Example of the filter algorithm (A) and the smoothing algorithm (B) applied in the analysis of a simulated learning experiment. The correct and incorrect responses are shown, respectively, by black and gray marks above the panels. The probability of a correct response occurring by chance is 0.25 (dashed horizontal line). Black lines are the learning curve estimates, and the gray lines are the associated 90% confidence intervals. The 90% confidence intervals are defined by the upper and lower 95% confidence bounds. The learning trial is defined as the trial on which the lower 95% confidence bound exceeds 0.25 and remains above 0.25 for the balance of the experiment. The filter algorithm identified trial 27 as the learning trial (arrow in A), whereas the smoothing algorithm, which used all of the data, identified trial 23 as the ideal observer learning trial with level of certainty 0.95 (arrow in B). The confidence limits at a given trial were constructed from the probability density of a correct response at that trial using Equations 2.4 and B.4. The probability densities of the probability of a correct response at the learning trial and the trial immediately preceding the learning trial are shown in C for both the filter (solid lines) and smoothing (dashed lines) algorithms. For the filter algorithm, the learning trial was 27 (C, solid black line) and the preceding trial was 26 (solid gray line), whereas for the smoothing algorithm, the IO (0.95) learning trial was trial 23 (C, dashed black line) and the preceding trial was 22 (dashed gray line). D shows the level of certainty the ideal observer has that the animal's performance is better than chance at each trial. From trial 23 on, the ideal observer is 0.95 certain that the performance is better chance, whereas this observer can be 0.90 certain of performance better than chance from trial 12 on.

Figure 2.

A, Family of sigmoidal curves (Eq. 3.1) used to simulate the learning experiments. Learning curves were constructed using three values of the initial probability of a correct response p₀ (0.125, 0.25, and 0.5), five values of the final probability of correct response p_f (0.6, 0.7, 0.8, 0.9, and 1), and three values of γ (0.2, 0.3, and 0.4), which governs the rate of rise or learning rate of the curves. For each of the 3 × 5 × 3 = 45 learning curves, we simulated 100 learning experiments for a total of 4500. All of the curves increase monotonically from p₀, indicating that performance is better than chance on all trials, i.e., learning starts immediately. B, MISE for the filter algorithm (gray dots), the smoothing algorithm (black dots), and the moving average method (squares) for each of the 45 learning curves in A plotted as a function of p_f - p₀. The smoothing algorithm MISE is smaller than those of the filter algorithm and the moving average method for all values of p_f - p₀ above 0.1.

Figure 3.

Analysis of three simulated learning experiments by the filter algorithm, the smoothing algorithm, and the moving average method. A-C, Delayed rapid learning. D-F, Immediate rapid learning. G-I, Learning after initially declining performance. We compared the 100 estimated learning curves (green), the true learning curve (black), and the 90% confidence intervals (red) using the filter algorithm (first column, A, D, and G), the smoothing algorithm (second column, B, E, and H), and the moving average method (third column, C, F, and I). The moving average method estimates fluctuate more, do not provide confidence intervals, and do not track well the true learning curves. The filter algorithm learning curve estimates consistently lag behind the true learning curves. The smoothing algorithm gives the best overall estimates of the learning curves with the narrowest confidence intervals.

Figure 4.

Analysis of learning trial estimation for the family of sigmoidal learning curves in Figure 2.A shows the number of learning trials identified by the smoothing algorithm [IO (0.95); black dots], the change-point test (CPT; crosses), the consecutive correct response method (CR; squares), and the moving average method (MA; plus signs) as a function of p_f - p₀, the difference between the final and the initial probabilities of a correct response. Each point is the number of learning trials identified by a given method for one of the 15 combinations of p_f - p₀ summed over the three values of γ (Fig. 2). The maximum value of each point would be 3 × 100 = 300, if the method identified a learning trial for each simulated experiment. The IO (0.95) identified a learning trial in more experiments than the other three methods, for all values of p_f - p₀ except when p_f - p₀ was 0.1. Scatterplots of the learning trial estimates of the change-point test versus the IO (0.95) (B), the consecutive correct responses method versus the IO (0.95) (C), and the moving average method versus the IO (0.95) (D), for the 2892 of the 4500 simulated learning experiments (Fig. 2) in which all three methods identified a learning trial. Because in all of the simulated experiments true performance was greater than chance from the outset, the method that identified the earliest learning trial performed the best. The 45° line indicates that the two methods being compared identified the same learning trial. The change-point learning trial estimate was later than the IO (0.95) estimate in 2237 of 2892 experiments (77%), at the same trial in 233 of 2892 (8%), and earlier in 422 of 2892 (15%). The consecutive correct response method estimate of the learning trial was later than the IO (0.95) estimate in 2875 of 2892 experiments (>99%), at the same trial as in 0 of 2970, and earlier in 17 of 2970 (<1%). The moving average method estimated the learning trial after the IO (0.95) criterion in 2810 of 2892 experiments (97%), at the same trial as the moving average in 70 of 2892 (<3%), and earlier in 12 of 2892 (<1%) of the simulated experiments.

Figure 5.

Comparison of learning trials estimated by the IO (0.95), change-point test, the consecutive correct response method, and the moving average method for the three simulated learning experiments in Figure 3. A compares learning trial estimates from the change-point test (CPT) and the IO (0.95) for delayed rapid learning (gray dots), immediate rapid learning (black dots), and learning after declining performance (squares). The number of learning trial estimates above and below the 45° line are marked in the panel. Trial 72 (dashed lines) is the trial on which the learning curve in the learning after declining performance simulations crossed p₀ = 0.5, the line for the probability of a correct response by chance. For the delayed rapid learning curve (gray dots, A), there were 50 simulated experiments in which the change-point test estimated learning later than the IO (0.95), 40 experiments in which it estimated learning earlier, and eight experiments in which the change-point test and IO (0.95) estimated identical learning trials. For the immediate rapid learning (black dots, A), the change-point test estimated learning earlier than the IO (0.95) for the majority of simulated experiments (59 of 81) because of the observation-dependent null hypothesis in the change-point test (see Results). For the learning after declining performance (squares, A), the change-point test estimated learning earlier than the IO (0.95) in all of the simulated experiments. However, the majority of the estimates (61 of 99) occurred before trial 72, indicating that the change-point test incorrectly identified learning before the true learning curve was above chance. B compares the learning trial estimated from the consecutive correct responses method (CR) and the IO (0.95) for the same three simulated learning curves using the same symbol definition as in A. The IO (0.95) estimated learning earlier than the consecutive responses method in nearly all (292 of 294) simulated experiments. For the learning after declining performance curve (open squares, B), the IO (0.95) only estimated learning in advance of trial 72 in one experiment as trial 69. The consecutive correct responses method never detected learning before trial 72. C compares learning trials estimated with the moving average method (MA) and the IO (0.95) criterion. In most cases, the moving average estimates of the learning trial were later (295 of 299) than the IO (0.95) learning trial estimates and were identical to the IO (0.95) estimates in 4 of 299.

Figure 6.

Performance of the smoothing algorithm compared with the moving average method in analyzing association learning by a macaque monkey. The monkey performed a 55-trial location-scene association task (A, B) and a 67-trial location-scene association task (C, D). The incorrect and correct responses of the monkey in each experiment are shown, respectively, as gray and black tick marks above A-D. The solid black curves are the learning curve estimates for the smoothing algorithm, and the gray curves are the associated 90% confidence intervals in A and C. In B and D, the gray line shows the moving average estimate of the probability of a correct response. The asterisks denote the nine-trial windows in which the number of correct responses is significantly more than would be predicted by chance if there were no learning based on a local binomial probability distribution function. In both experiments, the probability of a correct response occurring by chance was 0.25 (horizontal dashed line). In the 55-trial experiment, the monkey gave only two correct responses in the first 24 trials (A, B). The IO (0.95) criterion identified learning at trial 24 (arrow in A). The moving average method (B) identified the learning trial as occurring in the window centered at trial 24 (arrow in B). In the 67-trial experiment, the monkey gave its first correct response at trial 13 and subsequently performed better than chance but with a smaller total proportion of correct responses compared with the 55-trial experiment. The IO (0.95) identified the learning trial as trial 16 (arrow in C), whereas the moving average method identified the learning trial as occurring near trial 60 (arrow in D). E summarizes learning trial estimates for the two experiments using the IO (0.95) criterion, the moving average method (MA), the consecutive correct responses method (CR), and change-point test (CPT).

Figure 7.

Performance of a rat during a 47 d T-maze procedural learning task. The rat performed 40 trials per day, except on day 1 (20 trials) and day 46 (15 trials), for a total of 1835 trials. The data were analyzed using the smoothing algorithm and the IO (0.95) criterion in a trial-by-trial analysis (A) and in a day-by-day analysis using the smoothing algorithm with the binomial observation model (Eq. 3.2) and the IO (0.95) criterion and the number of correct responses on each day as the observed data (B, C). The learning curve estimates for each analysis (solid black lines) and the associated 90% confidence intervals (gray lines) are shown. In B, the black dots connected by the gray lines are the proportions of correct responses on each day. The probability that correct response occurred by chance was 0.5 in this experiment (dashed horizontal lines in all panels). The learning trial estimates for the trial-by-trial analysis in A were as follows: IO (0.95), trial 341 on day 9; the change-point test (CPT), trial 953 on day 24; and the fixed criterion of 15 consecutive correct responses (CR), trial 567 on day 14. The estimates of the learning day for the day-by-day analysis in B were as follows: IO (0.95), day 11; and the empirical criterion (EC) of Jog et al. (1999), day 13. C, The data were also analyzed on a day-by-day basis as the experiment progressed by computing the learning curve from the start to the end of the data series as the end was moved from day 3 to day 47. The 45 learning curves (black lines) and their associated 90% confidence bounds (gray lines) are plotted in C with the proportion of correct responses on each day (black dots). The “whispy” appearance of the plot is attributable to the fact that each learning curve and its associated confidence intervals were estimated with a different number of days of data, and, hence, each has different end points. In each analysis from day 12 to 47, the IO (0.95) criterion identified the learning day as day 11. The fact that the upper 95% confidence bound fell below 0.5 at trial 4 (B, C) suggests that the animal might have had a response bias at the start of the experiment.