Dynamic hidden states underlying working-memory-guided behavior

Abstract

Recent theoretical models propose that working memory is mediated by rapid transitions in 'activity-silent' neural states (for example, short-term synaptic plasticity). According to the dynamic coding framework, such hidden state transitions flexibly configure memory networks for memory-guided behavior and dissolve them equally fast to allow forgetting. We developed a perturbation approach to measure mnemonic hidden states in an electroencephalogram. By 'pinging' the brain during maintenance, we show that memory-item-specific information is decodable from the impulse response, even in the absence of attention and lingering delay activity. Moreover, hidden memories are remarkably flexible: an instruction cue that directs people to forget one item is sufficient to wipe the corresponding trace from the hidden state. In contrast, temporarily unattended items remain robustly coded in the hidden state, decoupling attentional focus from cue-directed forgetting. Finally, the strength of hidden-state coding predicts the accuracy of working-memory-guided behavior, including memory precision.

Figure 1. Experiment 1 task structure, behavioural performance and attention-related alpha band activity.

a. Trial schematic. Two memory items were presented (randomly oriented grating stimuli), and participants were instructed to memorize both orientations. A retro-cue then indicated which item would actually be tested at the end of the current trial (100% valid). The impulse stimulus (high contrast, task-irrelevant visual input) was then presented during the subsequent delay while participants should have only the cued item in WM. At the end of the trial, a forced-choice probe was presented at the centre of the screen. Participants indicated whether the probe was rotated clockwise or anti-clockwise relative to the orientation of the cued item. b. Boxplots show WM accuracy as a function of the absolute angular difference (in degrees) between the memory item and the probe. Data points outside of the 1.5 * interquartile range are shown separately (small crosses). c. Time-frequency representation of the difference between the contra- and ipsilateral posterior electrodes relative to the cued hemifield. The highlighted cluster in the alpha frequency band (8-12 Hz) indicates significant contralateral desynchronization (permutation test, n = 30, cluster-forming threshold p < 0.05, corrected significance level p < 0.05). The coloured bars under the x-axis represent the timings of the corresponding stimuli illustrated on top.

Figure 2. Orientation decoding in EEG and pinging hidden states of WM.

a-d. Decoding procedure. a. The dissimilarity in the neural pattern between a single trial and all other trials is computed as a function of orientation difference (binned: 30 degrees). b. Average distance to template of all trials for each time-point during and after memory item presentation, plotted separately for the left and the right memory item (upper/lower respectively). Distances are mean centred and sign reversed (high = small distance/high similarity) for visualization. c. A cosine is convolved with the data. d. The vector mean of the convolved tuning curves (i.e., decoding accuracy) over time, averaged over left and right items. The black bar indicates significant decoding (permutation test, n = 30, cluster-forming threshold p < 0.05, corrected significance level p < 0.05). Error shading is the 95 % C.I. of the mean. e. Pinging hidden states. Analogy to active sonar: differences in hidden state are inferred from differences in the measured response to a well-characterised impulse. f. Decoding results in the impulse epoch. The blue bar indicates significant decoding of the cued item. The purple bar indicates significant difference in decodability between the cued and uncued item (permutation test, n = 30, cluster-forming threshold p < 0.05, corrected significance level p < 0.05). Error shading is the 95 % C.I. of the mean. The boxplots and superimposed circles with error-bars (mean and 95 % C.I. of the mean) represent average decoding from 100 to 500 ms after impulse onset. Data points outside of the 1.5 * interquartile range are shown separately (small crosses). Significant average decoding and significant difference in average decodability between the cued and uncued item are marked by asterisks (permutation test, n = 30, p < 0.05).

Figure 3. Relationship between item-specific impulse decoding and WM accuracy.

a. Difference in overall WM task performance between high and low cued item decoding trials (left). Proportion clockwise response for high and low decoding trials as a function of the angular difference between the memory item and the probe (right). Inset shows the difference in the slope parameter (a measure of memory precision) between high and low decoding trials. Data points outside of the 1.5 * interquartile range are shown separately in the boxplots (small crosses). Superimposed circles and error-bars are the mean and 95% C. I. of the mean. b. The same convention as in a. but for the decoding of the uncued item. Significant differences in accuracy/precision between high and low decoding trials are highlighted by asterisks (permutation test, n = 30, p < 0.05).

Figure 4. Experiment 2 task structure, behavioural performance and attention-related alpha band activity.

a. Trial schematic. Two memory items were presented. Participants were instructed to maintain both items and were told at the start of each block which order the items would be tested. The first impulse was presented within the first memory delay (maintain both items, but attend the prioritised item), after which the prioritised item was probed. The second impulse was presented during the subsequent memory delay (maintain and attend only the now-prioritised item), after which the remaining item was probed. b. Boxplots show the accuracy of the early and late tested item as a function of the absolute angular difference (in degrees) between the memory item and the probe. Data points outside of the 1.5 * interquartile range are shown separately in the boxplots (small crosses). c. Time-frequency representation of the difference between the contra- and ipsilateral posterior electrodes relative to the presentation side of the early tested memory items. Highlighted areas indicate significant difference (permutation test, n = 19, cluster-forming threshold p < 0.05, corrected significance level p < 0.05).

Figure 5. Priority-dependent encoding and maintenance in WM.

a. Decodability of the item that is tested early (blue) and the item that is tested late (red) during memory item presentation. Blue and red bars indicate significant decoding clusters for the early- and late-tested item, respectively (permutation test, n = 19, cluster-defining threshold p < 0.05, corrected significance level p < 0.05). Error shading is 95% C.I. of the mean. Boxplots and superimposed circles with error bars (mean and 95 % C.I. of the mean) represent average decodability from 100 ms after stimulus onset until the end of the epoch. Significant average decoding and average difference between the decodability of the early and late item are marked by an asterisk (permutation test, n = 19, p < 0.05). b. Cross-temporal decoding matrices of the early (left) and late-tested (middle) item derived from training and testing on all time-point combinations, and the difference between the decoding of the early and late tested item (right). The grey outline indicates time-points of significantly lower decoding relative to both equivalent time-points along the diagonal, which is taken as evidence for dynamic coding (permutation test, n = 19, cluster-defining threshold p < 0.05, corrected significance level p < 0.05). The black outline (right) indicates significantly higher decodability of the early compared to the late tested item (permutation test, n = 19, cluster-defining threshold p < 0.05, corrected significance level p < 0.05).

Figure 6. Attended and unattended WM items in early and late epochs and relationship to behavioural performance.

a. Item decoding of the early (blue) and late tested item (red) during the first impulse epoch. Coloured bars on top indicate significant decoding clusters of the corresponding items (permutation test, n = 19, cluster-defining threshold p < 0.05, corrected significance level p < 0.05). Error shading is 95% C.I. of the mean. Boxplots and superimposed circles with error bars (mean and 95% C.I. of the mean) represent average decodability from 100 ms after stimulus onset until the end of the epoch. Significant average decoding and average difference between the decodability of the early and late item are marked by an asterisk (permutation test, n = 19, p < 0.05). b. Item decoding during the second impulse epoch, same conventions as a. c. Boxplot and superimposed circles and error-bars represent the difference in overall WM task performance between high and low early-tested item decoding trials during the first impulse (left). Proportion of clockwise responses for high and low decoding trials as a function of the angular difference between the memory item and the probe (right). Inset shows the boxplot and error-bar of the difference in the slope parameter (a measure of memory precision) between high and low decoding trials. d. The same convention as in a. but for the decoding of the late-tested item during the late impulse. Significant differences in accuracy/precision between high and low decoding trials are highlighted by asterisks (permutation test, n = 19, p < 0.05, two-sided and one-sided for accuracy and precision tests, respectively). Data points outside of 1.5 * interquartile range are shown separately in the boxplots (small crosses).