Constant Sub-second Cycling between Representations of Possible Futures in the Hippocampus

Abstract

Cognitive faculties such as imagination, planning, and decision-making entail the ability to represent hypothetical experience. Crucially, animal behavior in natural settings implies that the brain can represent hypothetical future experience not only quickly but also constantly over time, as external events continually unfold. To determine how this is possible, we recorded neural activity in the hippocampus of rats navigating a maze with multiple spatial paths. We found neural activity encoding two possible future scenarios (two upcoming maze paths) in constant alternation at 8 Hz: one scenario per ∼125-ms cycle. Further, we found that the underlying dynamics of cycling (both inter- and intra-cycle dynamics) generalized across qualitatively different representational correlates (location and direction). Notably, cycling occurred across moving behaviors, including during running. These findings identify a general dynamic process capable of quickly and continually representing hypothetical experience, including that of multiple possible futures.

Keywords: CA1; CA2; CA3; decision-making; hippocampus; imagination; place cells; planning; synchrony; theta rhythm.

Figure 1.. Study rationale and initial observation.

(A) A visualization of two types of representation: localizing vs. generative. For illustration, the representational correlate is here taken to be the animal’s (rat) location. Given this correlate, veridical (localizing) representation refers to actual location, while hypothetical (generative) representation refers to possible locations; for example, spatial paths projecting from the animal’s actual location. (B) Diagram of hippocampal recording sites. CA2 recording sites were those near the cytoarchitectural locus of CA2 (dotted lines). (C) Diagram of task maze and generative scenarios (possible futures). Separate diagrams are shown for two types of maze passes: those in which subjects (rats) choose left (L; grey) vs. right (R; pink) maze arms. Actual maze was not colored differently. In the maze task (Figure S1A), subjects were rewarded for choosing correctly between L vs. R. When the subject is located in the center maze arm before crossing the choice boundary (CB, dotted line), the L and R maze arms constitute possible future locations; moreover, for a given maze pass, entry into the maze arm not subsequently chosen necessarily constitutes a possible future scenario. (D) Identifying generative neural activity: competing hypotheses. For each hypothesis, a schematic of neural activity during a single maze pass is shown; a colored bar indicates neural activity encoding one of two locations: L (grey) vs. R (pink) maze arms. The time at which the subject crosses the choice boundary (CB) is indicated by a dotted line. Neural activity is in reference to a particular brain region of study (e.g. the hippocampus). Note that neural activity is generative only if the unchosen maze arm (alternative) is encoded, as in (iii), (iv), and (v). (i) Pure localizing. Neural activity encodes no information about possible experience. (ii) Pure prediction. Before the CB, neural activity only encodes the subsequently chosen maze arm (i.e. fully anticipated experience). (iii) Generative continuous. Before the CB, neural activity encoding the alternative occurs without interruption. (iv) Generative irregular. Before the CB, neural activity encoding the alternative occurs only during irregular intervals. Irregularity can be defined as having a wide incident frequency and also lacking entrainment to any narrow-frequency pattern. Two irregular patterns associated with generative neural representation are sharp-wave ripples (Buzsaki, 2015; Joo and Frank, 2018) and behavioral head scanning events (vicarious-trial-and-error (Redish, 2016)). (v) Generative cycling. Before the CB, neural activity encoding the alternative occurs at regular intervals (in cycles); an internal dynamic process having a characteristic time course (diagrammed as a wave) is implied. (E and F) Firing maps of two example cell pairs. Each row corresponds to a cell. Data from outbound maze passes. Left column: positions visited (grey) and positions where the cell fired (colored points; cell 1: black; cell 2: red). Total number of spikes is reported at upper right. Right column: time-averaged firing map. Peak firing rate is reported at upper right. Recording regions: cell 1: CA3; cell 2: CA2; cell 3: CA3; cell 4: CA3. (G and H) Firing rasters of the two cell pairs from (E) and (F), respectively, during three maze passes. Plotted above each pass is theta-filtered LFP (θ, 5–11 Hz from CA3). Periods when subject was located in an outer maze arm are indicated above plots by colored bar (grey: left; pink: right). In (G), a portion of the data is expanded to help show the firing pattern (grey boxing; scale bar: 125 ms). Note the firing alternation between cells at the ~125 ms (8 Hz) timescale. (I and J) Firing cross-correlogram (XCG) of the two cell pairs from (E) and (F), respectively. Cell 1 (and 3) spikes are aligned to Cell 2 (and 4) spikes (t = 0 s). Each XCG (5-ms bins) is smoothed with a Gaussian kernel (σ = 10 ms) and peak-normalized; total number of spikes in XCG is reported at top. (K) Firing XCGs of anti-synchronous cell pairs (in rows; see STAR Methods for criteria). Greyscale value indicates firing density. Data from outbound maze passes. Additional cell pair types and data conditions in Figure S2.

Figure 2.. Constant cycling (8 Hz) in the hippocampus.

Example cell pairs showing constant cycling firing at 8 Hz. (A-D) Four example cell pairs with differing locational representations (left (grey) vs. right (pink) arm; schematic at far left). Plotting conventions are the same as Figures 1E–J. Data from outbound maze passes, with data from left (grey) vs. right (pink) passes plotted separately. In (A) and (B), a portion of the data is expanded to help show the firing pattern (grey boxing; scale bar: 125 ms). (E-J) Six example cell pairs with differing directional representations (outbound (grey, Out) vs. inbound (pink, In) direction; schematic at far left). Data from inbound vs. outbound maze passes plotted separately. Plotting conventions are the same as Figures 1E–J but with the following differences: black corresponds to outbound-preferring cells, while red corresponds to inbound-preferring cells; below, linearized (rather than 2D) time-averaged firing maps are plotted (arms plotted in linearized maps correspond to positions plotted as light grey in raw firing maps); in rasters, maze pass times are indicated above plots by colored bars. In (E) and (F), a portion of the data is expanded to help show the firing pattern (grey boxing; scale bar: 125 ms). (K and L) Two example cell pairs with similar locational and directional representations. Data from inbound (pink) vs. outbound (grey) maze passes (schematic at far left, shared with that of E-J), with passes of each direction plotted separately. Plotting conventions are the same as (E-J), but firing data from cell 26 is shown in dark blue to denote outbound preference (same preference as cell 25), and firing data from cell 27 is colored in dark red to denote inbound preference (same preference as cell 28). In the example in (L), two XCGs are plotted, one for each (directional) condition; notably, anti-synchronous cycling firing is seen only in one condition. Note that (K) and (L) are suggestive of a representational correlate, different from location and direction, that may be associated with cycling firing. Recording regions: CA1: cells 3–6, 11–14, 16–17, 19; CA2: cells 9, 18, 23; CA3: cells 1–2, 7–8, 10, 15, 20–22, 24.

Figure 3.. Two correlates of cycling.

(A-D) Time-averaged firing maps (left) and rasters (right) of four example cells (recording regions at far left). Data plotted is from a single type of maze pass (colored arrows; grey: outbound; pink: inbound). In rasters, maze pass times are indicated above plots by colored bars; plotted above each pass is theta-filtered LFP (θ, 5–11 Hz from CA3). Firing maps, left column: positions visited (grey) and positions where the cell fired (black points). Total number of spikes is reported at upper right. Firing maps, right column: time-averaged firing map. Peak firing rate is reported at upper right. For the cell in (B), data from two types of maze passes are shown on separate rows to illustrate that firing patterns for an individual cell could depend on condition. (E) Firing auto-correlograms (ACGs) of the four cells from (A-D). Each ACG (5-ms bins; zero bin excluded) is smoothed with a Gaussian kernel (σ = 10 ms) and peak-normalized. Total number of spikes in ACG is reported at top. For the cell from (B), data from each of the two types of maze passes are shown on two separate rows. (F) Firing ACGs of all cell samples (in rows) across recording regions (CA1, CA2, CA3). Greyscale indicates firing density. Each cell sample corresponds to data from a single cell for one type of maze pass. Cell samples are ordered by cycle skipping index (CSI; high to low plotted top to bottom); for each region, red arrowheads indicate division between cell samples with CSI > 0 (above division) vs. < 0 (below division). CSI < 0 corresponds to classical firing (firing on adjacent cycles), while CSI > 0 corresponds to cycle skipping. (G) Cycle skipping index (CSI) by hippocampal recording region. Top, diagram of sub-regions with higher CSI values (CA2 and CA3; yellow zone). Bottom, histograms of CSI values across cell samples. Total number of cell samples is indicated at upper right. Values in CA2/CA3 were higher than in CA1. (H) CSI by behavioral condition. Top, schematic of the conditions (choice passed: periods when subject was leaving the choice point; choice imminent: periods when subject was approaching the choice point). Bottom, histograms of CSI values across cell samples. Total number of cell samples is indicated at upper right. For every recording region (CA1, CA2, CA3), CSI values were higher for choice imminent vs. choice passed. Rank-sum tests. P-values reported in main text. **, P < 0.01. ***, P < 0.001.

Figure 4.. Constant cycling (8 Hz) of possible future locations.

Three example maze passes (A, B, C) from a single recording epoch. (At left) Behavior plot. Position (green) and head angle (black lines; sampling period in plot: 133 ms) are overlaid on positions visited by the subject in epoch (color-coded by maze arm; grey: C (center); blue: L (left); red: R (right)). (At right) Data and decoded representation. Top section: LFP (θ, 5–11 Hz; CA3). Second section: decoded output (y-axis: linearized position); probability density is plotted as color values and colored by maze arm (black: C; blue: L; red: R); green line indicates actual position of subject. In (A), several example instances of ~100 ms spatial sequences within the center arm (schematized in Figure S1C) are highlighted by open arrowheads. Third section: multi-unit spiking activity (MUA; smoothed with Gaussian kernel (σ = 20 ms)). Bottom section: linear (light grey fill trace) and angular (dark grey fill) speed. (D) Prevalence of constant cycling in observed (red line) vs. shuffled data (histogram, 10000 permutations; study-wide shuffle). Plotted is the total number of cycles in detected constant cycling periods. P = 0.0034 (34 out of 10000 shuffles had equal or higher prevalence of cycles). (E) P-values of individual constant cycling periods (individual period shuffle). Shaded area enclosed by dotted line indicates criterion (P < 0.05) for individual periods analyzed subsequently in (F) and (G). Plotted separately are individual periods that occurred exclusively during movement (>4 cm/s) (black bars) versus those that overlapped with low speed periods (<4 cm/s for <0.5 s) (stacked white bars). (F) Histogram of durations (in cycles) of individual constant cycling periods. Bar plot convention follows that of panel (E). (G) Behavioral speed during individual constant cycling periods. Individual periods that occurred exclusively during movement (>4 cm/s) (black dots) versus those that overlapped with low speed periods (<4 cm/s for <0.5 s) (open grey circles) are plotted separately. Observed periods commonly occurred when angular speed was low (<10 deg/s), indicating that constant cycling could occur in the absence of overtly deliberative behavior (e.g. head scanning (Johnson and Redish, 2007; Redish, 2016)). (H) Theta phase histograms of decoded location representation (n = 1683 maze passes across 7 subjects; SEM omitted from plots due to minimal size). For each maze pass, posterior density (across positions) was pooled across time bins, subdivided into three maze arms (center, choice, and alternative (alt)), then histogramed by theta phase. Decoded data restricted to the center arm. Note that both choice and alternative arms tended to be represented in the second half of theta (0 to π). (Left) 12-bin histogram: center, choice, and alternative (alt). (Upper right) 2-bin histogram: center vs. choice. (Lower right) 2-bin histogram: center vs. alternative (alt). Signed-rank tests. P-values reported in main text. ***, P < 0.001.

Figure 5.. Intra-cycle coding of hypotheticals.

Single-cell examples of hypothetical coding; examples grouped by representational correlate (A, outbound path; B, direction; C, inbound path). Across all plots, firing data are colored based on the condition in which they occurred: preferred (black) vs. non-preferred (blue). Note that, regardless of representational correlate, firing in the non-preferred (vs. preferred) condition is shifted to the second half of theta (0 to π), consistent with encoding of hypothetical (vs. current) experience. (A) Outbound path-coding cells. Each example cell is plotted in a column. For outbound path coding, preferred vs. non-preferred conditions correspond to whether the maze path subsequently taken by the subject was to the left vs. right. In each plot, maze locations where quantification of theta phase of firing (Figure 6) was performed are indicated (light grey; other locations in dark grey). Top section, firing maps. Two maps are shown: one from preferred path (black arrow) maze passes and one from non-preferred path (blue arrow) maze passes. Firing locations during each path pass type are shown as colored points (preferred: black; non-preferred: blue). Second section, time-averaged linearized firing map. Bottom section, theta phase of firing by position. Firing is plotted separately whether occurring in the preferred (black) vs. non-preferred (blue) condition. Arrow at upper left of plots indicates the subject’s heading direction. Total number of spikes is indicated at lower right. Recording regions, left to right: CA3, CA2, CA2, CA1. (B) Direction-coding cells. Plotting conventions are the same as in (A), with total spike counts indicated at lower left. Preferred vs. non-preferred conditions correspond to opposite heading directions. Recording regions, top row, left to right: CA3, CA3, CA1, CA3; bottom row, left to right: CA3, CA3, CA1, CA1. (C) Inbound path-coding cells. Plotting conventions are the same as in (A). Preferred vs. non-preferred conditions correspond to whether the path previously taken by the subject was from the left vs. right. Recording regions, left to right: CA1, CA2, CA1, CA2.

Figure 6.. Intra-cycle coding of hypotheticals: summary.

Additional examples and study-wide quantification of outbound path (A-D) and direction (E-H) representation in single cells. Across all plots, firing data are colored based on the condition in which they occurred: preferred (black) vs. non-preferred (blue). (A) Outbound path-coding cells. Plotting conventions are the same as in Figure 5A. Recording regions, left to right: CA3, CA2, CA1. (B) Scatter of mean theta phase (n = 132 cell samples). For each cell sample, the theta phases of spikes in the preferred and non-preferred conditions were separately collected and the circular mean calculated. Cell samples analyzed were restricted to those with at least 20 spikes in the non-preferred condition, and with non-uniform phase histograms (Rayleigh tests at P < 0.05) in both conditions. (C) Theta phase histogram (12-bin). Mean ± SEM (n = 132 cell samples). (D) Theta phase histogram (2-bin). Mean ± SEM (n = 132 cell samples). (Note that preferred vs. non-preferred comparison would be expected to depend partly on locations of cells’ spatial firing fields.) Firing was higher in the 2^nd half of the theta cycle in the non-preferred condition. (E) Direction-coding cells. Plotting conventions follow (A). The cell in the middle column is same as that in the middle column in (A), instancing a cell with both location (left vs. right arms; Figures S1D-F) and direction (outbound vs. inbound direction; Figures S1G-I) selectivity. Note that, as in (A), spikes that occur in the non-preferred condition are shifted to the second half of theta (0 to π). Recording regions, left to right: CA1, CA2, CA1. (F-H), Directional firing theta phase quantification (n = 665 cell samples). Plotting conventions and comparisons follow (B-D). As in location (B-D), firing was higher in the 2^nd half of the theta cycle in the non-preferred condition. Signed-rank tests. P-values reported in main text. ***, P < 0.001.

Figure 7.. Constant cycling of heading direction.

(A-C) Three example maze passes showing various levels of half-theta cycling. Examples in (A) and (B) are from same recording epoch. In example (C), an individual period of constant half-theta cycling is highlighted (yellow bar). (Left) Behavior plot. Position (green) and head angle (black lines; sampling period in plot: 133 ms) are overlaid on locations visited by the subject in the recording epoch (light grey: locations analyzed; dark grey: other locations). (Right) Data and decoded representation. Top section: LFP (θ, 5–11 Hz from CA3). Second section: binary decoded output (red: inbound; blue: outbound). Third section: continuous-valued decoded output (−1: inbound; 0: non-directional; 1: outbound) (filled circle: 1^st half theta; open circle: 2^nd half theta; connecting lines shown for visual clarity); green line denotes actual direction of subject. Fourth section: multi-unit firing activity (MUA) smoothed with Gaussian kernel (σ = 20 ms). Fifth section: linear (light grey fill trace) and angular (dark grey fill) speed of rat. (D) Prevalence of constant (half-theta) cycling in observed (red line) vs. shuffled data (histogram, 10000 permutations; study-wide shuffle). Plotted is the total number of cycles participating in detected constant cycling periods. P < 0.0001 (0 out of 10000 shuffles had equal or greater prevalence of cycles). (E) P-values of individual constant (half-theta) cycling periods (individual period shuffle). Shaded area enclosed by dotted line indicates criterion (P < 0.05) for individual periods analyzed subsequently in (F) and (G). Individual periods that occurred exclusively during movement (>4 cm/s) (black bars) versus those that overlapped with low speed periods (<4 cm/s for <0.5 s) (stacked white bars) are plotted separately. (F) Histogram of durations (in half-cycles) of individual constant half-theta cycling periods. Bar plot convention follows that of (E). (G) Behavioral speed during individual constant (half-theta) cycling periods. Data plotted as a 2D histogram, where greyscale value corresponds to count density (767 total periods plotted). Observed periods commonly occurred when angular speed was low (<10 deg/s), indicating that constant cycling could occur in the absence of overtly deliberative behavior (e.g. head scanning (Redish, 2016)).