A model of bi-directional interactions between complementary learning systems for memory consolidation of sequential experiences

The standard theory of memory consolidation posits a dual-store memory system: a fast-learning fast-decaying hippocampus that transfers memories to slow-learning long-term cortical storage. Hippocampal lesions interrupt this transfer, so recent memories are more likely to be lost than more remote memories. Existing models of memory consolidation that simulate this temporally graded retrograde amnesia operate only on static patterns or unitary variables as memories and study only one-way interaction from the hippocampus to the cortex. However, the mechanisms underlying the consolidation of episodes, which are sequential in nature and comprise multiple events, are not well-understood. The representation of learning for sequential experiences in the cortical-hippocampal network as a self-consistent dynamical system is not sufficiently addressed in prior models. Further, there is evidence for a bi-directional interaction between the two memory systems during offline periods, whereby the reactivation of waking neural patterns originating in the cortex triggers time-compressed sequential replays in the hippocampus, which in turn drive the consolidation of the pertinent sequence in the cortex. We have developed a computational model of memory encoding, consolidation, and recall for storing temporal sequences that explores the dynamics of this bi-directional interaction and time-compressed replays in four simulation experiments, providing novel insights into whether hippocampal learning needs to be suppressed for stable memory consolidation and into how new and old memories compete for limited replay opportunities during offline periods. The salience of experienced events, based on factors such as recency and frequency of use, is shown to have considerable impact on memory consolidation because it biases the relative probability that a particular event will be cued in the cortex during offline periods. In the presence of hippocampal learning during sleep, our model predicts that the fast-forgetting hippocampus can continually refresh the memory traces of a given episodic sequence if there are no competing experiences to be replayed.

Introduction

Studies on patients with lesions of the hippocampus report deficits both in forming new memories and in retrieving recent memories (Scoville and Milner, 1957). Based on these and subsequent studies, the hippocampus was found to be necessary for these functions (Squire, 1992; Izquierdo and Medina, 1997). It learns rapidly due to its ability to quickly develop synaptic connections (Malenka, 1994), but fast learning also means fast decay as older memories are overwritten by new ones—which would mean catastrophic interference in a unitary memory (Grossberg, 1980; McCloskey and Cohen, 1989; Ratcliff, 1990). This motivates the need for a two-stage memory system: the hippocampal engram gets strong quickly but also decays quickly, and the task of systems-level consolidation is to gradually train the more stable, slower learning neocortex (Buzsáki, 1989; Zola-Morgan and Squire, 1990; Marr et al., 1991; Alvarez and Squire, 1994; McClelland and O'Reilly, 1995; Squire and Alvarez, 1995; Dudai, 2004; Gais and Born, 2004; Káli and Dayan, 2004; Marshall and Born, 2007; Born and Wilhelm, 2012).

Replays of neural activity during sleep, which were first directly observed in the rat hippocampus (Wilson and McNaughton, 1994; Skaggs and McNaughton, 1996; Buzsáki, 1998), have been hypothesized to play a key role in memory consolidation (Alvarez and Squire, 1994; McClelland and O'Reilly, 1995; Káli and Dayan, 2004). During slow-wave sleep (SWS), the cortex exhibits synchronized slow-wave oscillations (SWOs) that are characterized by a rhythmic alternation between depolarized (“UP states”) and hyperpolarized states (“DOWN states”) (Sanchez-Vives, 2020). Replays generally occur during SWS (Nádasdy et al., 1999) and are coordinated between the hippocampal and cortical UP states (Ji and Wilson, 2007). Hippocampal replays can boost the weaker cortical representations and strengthen them until, eventually, they become independent of the hippocampus. Moreover, replays occur in a time-compressed manner (Nádasdy et al., 1999; Euston et al., 2007; Ji and Wilson, 2007) to likely facilitate more efficacious strengthening of synaptic connections among pertinent neurons in the cortex for memory consolidation (Nádasdy et al., 1999) through spike timing dependent plasticity (STDP; Bi and Poo, 1998; Caporale and Dan, 2008; Cai et al., 2016). The likelihood that a particular memory is replayed during limited SWS UP states appears to be dependent on saliency factors such as novelty (Cheng and Frank, 2008), recency, emotional involvement, and reward (Singer and Frank, 2009; McNamara et al., 2014).

In the last 15 years, several animal and human studies have demonstrated improved post-nap or post-sleep memory performance by either boosting cortical SWOs with sensory or transcranial electrical stimulation (Marshall et al., 2006; Cox et al., 2014; Ketz et al., 2018), or by applying cues during SWOs that were previously used to tag associations during learning (Rasch et al., 2007; Rudoy et al., 2009; Antony et al., 2012; Pilly et al., 2019). These demonstrations suggest that even though cortex learns slowly, it can nonetheless be modulated to boost the memory consolidation process at various levels of specificity. Some experimental evidence suggests that neural activity in the cortex during SWO UP states leads and can potentially bias the content of the time-compressed replays in the hippocampus, which then in turn drive coordinated replays in the cortex (Ji and Wilson, 2007).

Existing computational models of memory consolidation (Alvarez and Squire, 1994; McClelland and O'Reilly, 1995; Káli and Dayan, 2004) primarily simulate the behavioral effects of lesioning the hippocampus at different times after learning. But they use only static patterns or unitary variables to represent memories, though the episodes that we experience on a daily basis are sequential in nature and have a temporal aspect to them. Moreover, the effect of hippocampal learning during sleep replays on the persistence of memory traces in the supposedly short-term storage in the hippocampus has not been examined. In this regard, none of the existing models have investigated any scheme to prioritize the reactivation of individual events during the limited offline periods. And as such, there is no computational theory for how memory replays of sequential experiences emerge dynamically during offline periods and how they are coordinated between the hippocampus and neocortex to facilitate the stabilization of episodic memories (Ji and Wilson, 2007).

In order to overcome these limitations, we developed a model of memory encoding, consolidation, and recall for storing temporal sequences that explores the waking and sleep dynamics of the bi-directional interactions between the fast-learning hippocampus and the slow-learning cortex. Details of the model are in the Materials and methods section. Our model simulates episodic memories as a temporal sequence of activations of items in both regions (Tulving et al., 1972; Tulving and Markowitsch, 1998), where each item represents the conjunctive activation of a pool of neurons sensitive to the salient features of a particular event in the experience (Liu et al., 2012; Ramirez et al., 2013). The Results section provides results from four simulation experiments. First, we demonstrate how a five-item temporal sequence can be initially encoded in the hippocampus and subsequently consolidated in the cortex. We show how items can be probabilistically reactivated in the cortex during simulated UP states of SWS based on their recency and frequency of use (i.e., salience metric), how fast replays in the hippocampus can be triggered by these item reactivations in the cortex, and how they cascade into coordinated replays in the cortex that drive efficacious learning for long-term storage. Second, we replicate the phenomenon of retrograde amnesia with lesions of feedback projections from the hippocampus to the cortex. Third, we show how a more salient five-item sequence is preferentially replayed and consolidated compared to another five-item sequence. Fourth, we investigate if hippocampal learning during offline replays is critical for episodic memory consolidation. Finally, the Discussion section compares and contrasts our model with previous models of memory consolidation and also makes several testable predictions.

Materials and methods

Model of memory encoding, consolidation, and recall

Our core model consists of a hippocampal module and a cortical module (Figure 1), which are both simulated as recurrent networks capable of learning sequences of items. Items, which are constituents of various sequential experiences, are represented by individual neurons in either module. There is all-to-all excitatory and inhibitory connectivity within each module. In particular, each neuron receives fixed inhibitory projections from all the other neurons, and each neuron can learn directional excitatory projections to other neurons in response to the sequential presentation of items from the environment. As proposed by the complementary learning systems theory (McClelland and O'Reilly, 1995), intra-hippocampal excitatory projections have a much higher rate of learning compared to the intra-cortical excitatory projections. In other words, the cortex is slow to encode sequences of items compared to the hippocampus. Further, the memory traces (namely, weights of excitatory projections) in the hippocampus decay to their baseline levels (zero) at a much faster rate in the hippocampus compared to the cortex. Neurons also have self-excitation as well as delayed self-inhibition to regulate activations. Further, there are item-specific bi-directional fixed excitatory projections between the hippocampus and the cortex. Each waking sensory experience is represented as a sequence of items in an input register, providing bursts of excitation that coincide with their presentation to the corresponding neurons in the cortex. It is the spread of activation through the learned excitatory projections that is responsible for recall of a previously experienced sequence. In the remainder of this paper, for simplicity, we use “item” to also refer to the cortical and hippocampus neurons, which each represent a particular perceptual item.

FIGURE 1

Figure 1. An illustration of the cortico-hippocampal memory model. Encoding, consolidation, and recall of sequential experiences are simulated. When a percept is entered into the input register, it energizes a matching cortical item. If a matching cortical item is not found, a new one is allocated. Activation is spread from cortical items to their counterparts in hippocampus and back to the cortex through fixed inter-module links or bi-directional projections (black arrows). Directional intra-module links (red arrows) develop between any simultaneously active items within each memory module as a function of their dynamic activations. The illustration shows an input three-item sequence (“A”-“B”-“C”) with directional links having formed first from “A” to “B.” When “C” is encountered, if “A” and “B” are still active, links are instantiated from “A” to “C” and from “B” to “C.” The learning rate in the hippocampus is an order of magnitude faster than in the cortex.

Figure 2A is a plot of a training session in which the percepts “A,” “B,” “C,” “D,” and “E” (applied to the input register for 2 s each) sequentially activate the corresponding items in the cortex, and thereby in the hippocampus. Figure 2B shows a recall cued by applying the single item “A” to the input register for 1.5 s, which spreads to activate items “B” and “C” in turn. Item activations in either memory module are ephemeral, falling soon after they rise to a high level. In the example shown, the network has not been trained enough to develop strong links to items “D” and “E” for a full recall. Figure 4 shows cued recall in more detail.

FIGURE 2

Figure 2. An example of item activations during (A) training and (B) cued recall. (A) A five-item sequence is trained by presenting a series of inputs (the colored rectangles in cortex) that stimulate activation of matching items in cortex, and in turn activate the matching items in hippocampus. Directional excitatory projections between items within each memory module are learned as a function of their mutual activation dynamics. (B) Dynamics of neuronal activations when item “A” is cued in the input register after training. The activation of “A” in hippocampus lags behind the cued activation in cortex as cortex is driving it, but subsequent items are activated more quickly in hippocampus due to its stronger excitatory projections (owing to faster learning). In the example shown, neither module's weights have strengthened enough to spread activation through the entire sequence, so only “A,” “B,” and “C” become activated.

Simulation of memory encoding and recall during wake

The activation of a given item x in either cortex or hippocampus (a_x) is bounded between 0 and 1 and governed by shunting dynamics in a recurrent competitive network (Grossberg, 1976) as follows:

$\begin{array}{ll}{\sigma }_a\frac{d{a}_x}{dt}=-\frac{a_x}{{\tau }_a}+(1-a_x)E_x-a_x{I}_x,& (1)\end{array}$

where σ_a modulates the rate of the neuron's temporal integration (the smaller σ_a is, the faster the temporal dynamics are), τ_a modulates the rate of passive decay of the neuron's activation (the smaller τ_a is, the faster the decay is), and E_x and I_x are the net excitatory and inhibitory inputs to the neuron, respectively. Note that the time constant of the dynamics is equal to σ_a × τ_a. All model parameters are summarized in Table 1 along with default values. All equations were numerically integrated using Euler's forward method with a fixed time step Δt = 1 ms. The excitatory input to each neuron, E_x, is a sum of the feedforward input F (from items in the input register to the cortex, and from items in the cortex to the hippocampus), the adaptive excitatory projections from other neurons in the module, and self-excitation (modeled using a sigmoidal function), as follows:

$\begin{array}{ll}{E}_x=\left[\mu F+\gamma {\displaystyle \sum _{x\ne y}{w}_{yx}}{a}_y+\frac{\alpha a_x^m}{a_x^m+t_a^m}\right].& (2)\end{array}$

F = 0.1 for cortex (in response to the pertinent item in the input register) and F = c_x for hippocampus, where c_x is the activation of the corresponding item in the cortex. Cortical neurons are additionally excited by hippocampal feedback ($\zeta {\left[h_x\right]}^{+}$), where h_x is the activation of the corresponding item in the hippocampus). The inhibitory input to each neuron, I_x, is a sum of the fixed inhibitory projections from other neurons in the module and delayed self-inhibition (modeled using a sigmoidal function of an inactivation current g_x), as follows:

$\begin{array}{ll}{I}_x=\left[\beta {\displaystyle \sum _{y\ne x}{a}_y}+\frac{\theta g_x^n}{g_x^n+t_h^n}\right].& (3)\end{array}$

The inactivation current g_x simulates the inhibitory effect on active neurons that have been depolarized for a prolonged period (Mickus et al., 1999; Koch, 2004) as follows:

$\begin{array}{ll}{\sigma }_g\frac{d{g}_x}{dt}=-\frac{g_x}{{\tau }_g}+(1-g_x)\left[\kappa a_x\right],& (4)\end{array}$

where σ_g modulates the speed of temporal dynamics and τ_g modulates the rate of passive decay. The inactivation current g_x evolves and decays at a slower rate than the corresponding neuron's activation a_x (σ_g>σ_a and τ_g>τ_a), providing delayed inhibition on the item when it gets large. The effect of self-excitation in Equation (2) is to raise the item's activation quickly to a high level once activation reaches a certain threshold. However, the inhibitory input specified in Equation (3) ensures that few items will remain active at any time, and none for any prolonged period. All item activations a_x and inactivation currents g_x are initialized to 0 at the start of each experiment. Note that associative links between any two arbitrary items are instantiated at the first co-occurrence of their activations, as illustrated in Figure 1.

TABLE 1

Table 1. Model parameters.

While an arbitrary item y is active, its activity spreads across weighted directional projections w_yx (in Equation 2) to other items x, and across the fixed links between the two memory modules (see Figure 2B). These directional projections are causal, based on the order in which the linked items are experienced. As a projection becomes stronger, more excitation is transmitted to the post-synaptic neuron, which becomes activated more quickly. This results behaviorally in faster memory recalls. Thus, the strength of the excitatory directional projections between the component items of a memory in both hippocampus and cortex are predictive of the ability to recall the sequential experience.

The adaptive weight of the excitatory directional projection from item x to another item y in either cortex or hippocampus (w_xy), bounded between 0 and 1, is governed by a novel rate-based learning rule:

$\begin{array}{ll}\frac{d{w}_{xy}}{dt}=\eta \left\{-\frac{w_{xy}}{{\tau }_w}+a_x(1-w_{xy})\left[{\left[\frac{d{a}_y}{dt}\right]}^{+}-Q{\left[-\frac{d{a}_y}{dt}\right]}^{+}\right]\right\},& (5)\end{array}$

where η is the learning rate (the bigger η is, the faster the weights change) , τ_w modulates the rate of passive decay, and Q is a parameter that controls the emergence of non-casual (reverse) links. Equation (5) models how causal associative links are formed as a result of overlapping activation of the pre-synaptic neuron (item x) and the rate of change in the activation of the post-synaptic neuron (item y; cf., Cai et al., 2016; Yokose et al., 2017). If the activation of item x reliably increases the activation of item y, then x is assumed to be causal and as a result the connection from x to y (namely, w_xy) increases in strength. On the other hand, if the activation of item y reliably declines despite strong activation of item x, then x is assumed to be non-causal and as a result w_xy reduces in strength. The reduction in weight is a fraction Q of the increase, which not only ensures that associative links in the causal direction are created, but also is necessary for the learning of connections between simultaneous events. At the default value for Q of 0.5, reverse links are eliminated. In the model, the hippocampus learns (η_h>>η_c) as well as forgets (τ_{w, h} < < τ_{w, c}) faster compared to the cortex. With the default values for τ_w in Table 1, hippocampal weights undergo natural decay on the order of about 10 days and cortical weights on the order of about 2 years. Each weight is initialized to 0 when the corresponding connection is instantiated. Weights in both directions are updated at the same time, as illustrated in Figure 3. Existing connections are strengthened during recall, but new connections can only be created during training when a new item is presented while recently activated items are still active. Once associative links are learned between items in either memory module, a sequential memory can be reactivated by a relevant cue from the environment during waking or randomly during sleep. Note that by default, learning is enabled in both the hippocampus and cortex at all times including SWS.

FIGURE 3

Figure 3. Weights develop as a function of item activations. Weights learn to be directional from earlier items to later items in a sequence.

Memory recall metrics

Figure 4 illustrates the recall metrics used in the experiments discussed below, with an example of a cortical recall when cued by “A” after being exposed to the sequence “A”-“B”-“C”-“D”-“E” (or ABCDE for short) for 20 trials. In this case, the cortical representation was strong enough that every item became activated in turn. An item is considered recalled when its activation level in the cortex rises above a threshold of 0.01. Recall accuracy is the proportion of sequence items successfully recalled in the correct order. So if all five items in the sequence reach the threshold in the correct order, the accuracy is 100%. However, if any item crosses the threshold out of order, no credit is given for it or any further items. Thus, if the activation level of the first and second items crossed the threshold in order, followed by the fourth then third and fifth, the accuracy would be 40% as two of the five items crossed the threshold in the correct order before an incorrect item recall. For a recall trial with 100% accuracy, the recall time is defined as the interval between the beginning of cue stimulation to the moment at which the final item crosses the threshold, provided this occurs before the arbitrary ceiling of 30 s. If the accuracy is <100%, the recall time is assigned the arbitrary ceiling value of 30 s.

FIGURE 4

Figure 4. Spreading activation recalls a cued sequence during waking. After multiple training exposures, a presentation of the “A” cue (input activation of 0.1 for 1.5 s shown as a blue rectangle) causes a recall cascade in the cortex due to spreading activation through causal links. Each item in the sequence is considered recalled when it reaches the recall threshold of 0.01, shown as the blue dotted line.

Simulation of memory consolidation during sleep

As described in the Introduction section, memories are consolidated in long-term cortical memory during SWS, when events experienced in the daytime can be reactivated or “replayed” during the UP states of SWOs. We simulate these emergent replays in each UP state by selecting an item probabilistically and activating it as described below. Figure 5 plots emergent replays during a SWS simulation. An item cued in this way excites other items along existing directed connections, just as in waking, but the parameter for speed of dynamics (σ_a in Equation 1) is set so the activations rise and fall about 20 times as fast in sleep compared to waking. Sleep replays in rats are reported to occur between 6 and 20 times faster than waking experiences in rodents (Nádasdy et al., 1999; Lee and Wilson, 2002; Euston et al., 2007; Rasch and Born, 2013). There is also some evidence of time-compressed episodic memory replays in humans (Michelmann et al., 2019). The actual speed of a given replay depends on the strength of the causal associative links between the constituent items of the corresponding sequence. The change in a given weight is primarily driven by the overlap in the activation of the pre-synaptic neuron and the rate of change in the activation of the post-synaptic neuron (see Equation 5), which is higher when the replay is compressed in time. Thus, learning in the model is generally more efficacious during sleep replays compared to waking training. This is most important in the slower-learning cortex, and accounts for why sleep replays are so critical for strengthening connections in the cortex over time with no additional waking training.

FIGURE 5

Figure 5. A series of time-compressed replays during UP states in SWS. The last six replays on Night 1 after training the sequence ABCDE for 10 trials are plotted. The initial item in each replay is selected from a weighted distribution based upon the recent activation of each item (i.e., the salience), and its activity spreads to the subsequent items of the pertinent sequence stored in memory. In this example, six replays are generated by cueing “E,” “A,” “C,” “B,” “D,” and “A” in turn. The duration of the cue to initiate the replay in each UP state is 12.5 ms, and the amplitude is 0.2. Note a 1 Hz SWO is superimposed for illustration.

During UP states of SWOs, any item in either memory module can be randomly chosen to cue a replay by sampling from a weighted distribution called “salience.” The salience s_x of an item x is essentially a moving average of its activation during wake, which is defined as follows.

$\begin{array}{ll}\frac{d{s}_x}{dt}=-\frac{s_x}{{\tau }_s}+\lambda a_x,& (6)\end{array}$

where τ_s modulates the rate of passive decay and λ scales the input of the corresponding activation, which is active only during wake. Activations during sleep replays do not contribute to salience in order to prevent a self-activation feedback that would lead to a few items dominating. Since any item can be selected, not just the first item in a sequence, replays are typically fragmentary. Note that all item activations a_x and inactivation currents g_x are reset to zero at the end of each UP state, which is also the beginning of the next DOWN state. There is supporting in vivo evidence for hippocampal and cortical interneurons to be active during UP states and silent during DOWN state (Hahn et al., 2006; Zucca et al., 2017).

Results

Experimental details

Figure 6 shows typical timings for presentation of input percepts and recall cues in the experiments that follow. Training trials present a sequence of input stimuli in the input register, such as the sequence ABCDE shown in Figure 2A. Each percept during waking stimulates the input register at the level of 0.1 for 2 s, with an inter-stimulus interval of 5 ms. Trials are separated by 1 min. When a recall test is required, it is separated from training by at least 1 min, and the recall cue is presented in the input register for 1.5 s. As noted above, the recall is measured based on the spreading activations of the pertinent items in the cortex.

FIGURE 6

Figure 6. Typical experiment timings. Cue presentation times are illustrated for training and recall during waking. Training cues are presented for 2 s each in sequences, with 5 ms inter-stimulus interval. Recall cues last 1.5 s each.

For our simulation experiments, without loss of generality, each UP state during SWS randomly activates an item in the cortex with probability proportional to its corresponding salience with a brief pulsed cue in the input register (amplitude = 0.2, duration = 12.5 ms). In order to incorporate the possibility of no replay, a null item is added to the list of items with a fixed salience of 0.5. This ensures that the likelihood of selecting the null item is higher if the total salience of the other items is low (i.e., the probability of no replay increases when no items have been active recently). For simplicity and with loss of any generality, SWS during each night is simulated as a continuous block of SWOs for 50 s with a dominant frequency of 1 Hz. This manifests as 50 UP states per night, each with a duration of 0.5 s, to facilitate memory consolidation.

Salience and memory replays

To show how different sequential experiences can interfere even when they do not share any items (Robertson, 2012) based on differences in practice during wake, we trained two different sequences (ABCDE and FGHIJ), 10 trials each on Day 1, and then only ABCDE for 10 trials per day on Days 2, 3, and 4. Each night, 50 replays were run during SWS using the salience algorithm described above to select from the 10 items (along with the null item) to cue for replay. Figure 7 shows simulations of the last full replay for both sequences in each night, which started with the pertinent first item in each sequence. The inset histograms show the number of replays cued by each item in the two sequences during each night.

FIGURE 7

Figure 7. Representative replays of two separate sequences during a four-day, four-night simulation. Sequences ABCDE and FGHIJ were trained for 10 trials each on Day 1, and then only ABCDE was trained for 10 trials per day on Days 2, 3, and 4. 50 cycles of 1 Hz SWO were run each night. The last full replay for each of the four nights (cued by the first item in each sequence) within the respective 500 ms UP state is plotted here. (A) ABCDE replay in the hippocampus each night. (B) ABCDE replay in the cortex with inset histogram of the number of replays cued by each item during the night. (C) FGHIJ replay in the hippocampus each night. (D) FGHIJ replay in the cortex with inset histogram of the number of replays of each item during the night. Due to lower salience, replays of FGHIJ die out after Day 2. Note the first item is activated by the replay cue. The duration of replay is printed at the top of each plot, which is calculated only when all items in the sequence get activated.

Due to the extra daytime training on Day 2, the salience of the items in the sequence ABCDE rises above that for items of sequence FGHIJ, and so the sequence ABCDE is replayed more on subsequent nights (74% on Night 2, 94% on Night 3, and 98% on Night 4), and the activation spreads through the sequence faster with each night. Recall time of the replays is assessed at the end of the SWS period in each night by cueing with the first item for either sequence. The two sequences do not directly conflict in terms of sharing one or more constituent items, but they compete indirectly with each other for replay opportunities during SWS period in each night based on salience of the individual items. Additional training is certainly responsible for the increased number of replays for the sequence ABCDE on subsequent nights. Since there was no practice on the sequence FGHIJ after the first day, the salience of those items monotonically decayed; thus, the more salient ABCDE items were replayed more in the subsequent nights.

Hippocampal influence on cortical replays

We use our model to explore the necessity of the hippocampus for the long-term consolidation of sequential experiences in the cortex using two simulation experiments. The first experiment reveals how the baseline network of intact hippocampus and cortex operates, and the second experiment focuses on what happens if the feedback projections from the hippocampus to the cortex are lesioned prior to the initial encoding of a sequence. For both experiments, the network was trained on the sequence ABCDE for 10 trials on Day 1, followed by memory assessment over the succeeding 4 days. This included four nights with a SWS period at the start of each night consisting of 50 UP state events where one item was chosen to cue the replay using the salience algorithm described above.

The dynamics of hippocampal and cortical weights are plotted in Figure 8 for the baseline case with both the hippocampus and cortex intact. Here the hippocampal weights grow to over 75% strength when trained on Day 1 and then quickly decay, whereas the cortical weights rise to <40%; however, the cortical weights suffer only imperceptible decay. At the beginning of each night, the hippocampal weights recover due to replays during SWS, and the cortical weights rise as well. Replays occur in the hippocampus and the cortex on each night because the hippocampal weights are strong enough to propagate the activations through the sequence within the hippocampus, as well as within the cortex owing to item-to-item feedback projections from the hippocampus. During these replays, cortical connections become much stronger. Since there is no training after Day 1, the differences in the dynamics of weight strengths within each memory module over the subsequent days is exclusively due to the random emergent replays during the SWS period in each of the four nights; so, for example, in the cortex the weight A→B does not get as strong as the others because the fragment A→B within the sequence ABCDE is the least likely to be reactivated, compared to the other fragments, in response to saliency-based item selection for cueing replays during the SWS periods. See Supplementary Figures 1, 2 for replication of the baseline networks with a higher recall threshold of 0.05, which is five times greater than the default value, and with all weights randomly initialized using a uniform distribution on the interval [0, 0.05], respectively, to demonstrate the robustness of our simulation of memory consolidation.

FIGURE 8

Figure 8. Baseline network. (A) A sequence ABCDE was trained and tested on Day 1 as illustrated in the timeline. On the next four nights (highlighted with cyan patches) there were 50 non-deterministic replays biased by salience, followed by morning tests (HC, hippocampus; CTX, cortex). The middle row plots the two memory recall metrics based on each of the five waking tests: (B) Recall time, where 30 s is default for incomplete recalls. (C) Recall accuracy. The bottom row plots the dynamics of weight strengths between items of the sequence ABCDE in the cortex and hippocampus during the same period. (D) Weight strengths in CTX. (E) Weight strengths in HC.

If the hippocampal projections to the cortex are lesioned prior to Day 1, we find that cortical replays cannot occur (Figure 9). Replays can still occur in hippocampus, but connections between the items in the cortex are not strong enough after the initial training to trigger replays on their own, and the cortical weights do not grow (cf., Figure 8).

FIGURE 9

Figure 9. Lesions of hippocampal feedback projections before training. (A) With feedback from hippocampus to cortex turned off before training, the cortical representation cannot develop. Hippocampal connections are re-learned from replay activity during the SWS periods, but little growth is seen in the cortical connections (HC, hippocampus; CTX, cortex). (B) Recall time, where 30 s is default for incomplete recalls. (C) Recall accuracy. (D) Weight strengths in CTX. (E) Weight strengths in HC.

This is in agreement with the data on temporally graded retrograde amnesia with hippocampal lesions (Squire and Alvarez, 1995), where the longer the hippocampus is intact after training (with intervening offline periods), the stronger is the long-term consolidation in the cortex. The hippocampal projections to the cortex are initially necessary to generate replays in the cortex during sleep, and thereby consolidate sequential memories in long-term storage.

Competing sequences

We now focus on the effects of competition among sequential experiences for limited replay opportunities during sleep on memory recall. In this experiment (see Figure 10A), two non-overlapping sequences ABCDE and FGHIJ were trained for 10 trials each on Day 1, followed by continued training for the sequence FGHIJ only on subsequent Days 2–4 (10 trials per day). This has the same design as the experiment reported in Figure 7. Memory recall for the sequence ABCDE was assessed on each of Days 1–5 to focus on the effect of the more practiced sequence on the less practiced sequence. Just as the simulation experiments reported above, there were 50 replay opportunities at the start of each of the four intervening nights. On Nights 2–4, the items belonging to the sequence FGHIJ have a much higher chance (compared to the items belonging to the sequence ABCDE) of getting selected to cue the replays during the SWS period, as they have higher salience due to the additional practice on Days 2–4.

FIGURE 10

Figure 10. Effects of competition for limited replay opportunities on memory recall. (A) This experiment is similar to Figure 8, but competition from the more practiced sequence FGHIJ reduces and then eliminates replays of the sequence ABCDE (HC, hippocampus; CTX, cortex). There is a growth of cortical connections in Night 1, but very little in Night 2 and beyond. (B) Recall time for the sequence ABCDE, where 30 s is default for incomplete recalls. (C) Recall accuracy for the sequence ABCDE. (D) Weight strengths in CTX for the sequence ABCDE. (E) Weight strengths in HC for the sequence ABCDE.

In Figure 10, it can be seen that the cortical and hippocampal weights for the sequence ABCDE increased similar to Figure 8 on Day 1 and Night 1. But starting from Night 2, the preferential replays cued by items belong to the other sequence FGHIJ result in the indirect suppression of consolidation of the sequence ABCDE in the cortex. By Night 3, there were almost no replays related to the sequence ABCDE. Consequently, the sequence ABCDE is not recalled on Days 3–5 with expected minimum recall accuracy and maximum default recall time.

We also performed another simulation experiment where the model is exposed to two sequences that overlap in the third item (namely, ABCDE and FGCIJ). Both sequences were trained for 10 trials each on 4 consecutive days (Days 1–4). And 50 cycles of 1 Hz SWO were run each of the four nights (Nights 1–4). Memory recall tests were conducted at the end of each day. Supplementary Figure 3 shows the waking recalls for either sequence, cued by the respective first item, in the hippocampus and cortex. We can observe that the model is able to consolidate both sequences, but not without learning associations between subsequences “C”-“D”-“E” and “C”-“I”-“J.” Further model development would be needed to learn long-range connections between non-adjacent items (e.g., “A” -> “C,” “A” -> “D”) to enforce a more contextual recall as opposed to item-based recall. For instance, when cued by item “A” and following the recall of items “B” and “C,” the dynamic context of the subsequence “A”-“B”-“C” should bias the succeeding recall of item “D” over item “I.”

Hippocampal learning disabled during sleep

There is evidence that hippocampal plasticity is suppressed during SWS (Leonard et al., 1987; Bramham and Srebro, 1989). This might serve the purpose of allowing transfer to the cortex without further modifying the hippocampal representation (Squire and Alvarez, 1995). We repeated the experiment reported in Figure 8 with hippocampal learning disabled during sleep (see Figure 11). During Day 1 training, the hippocampus learned the sequence as evidenced by the high weights, high recall accuracy, and low recall time. Hippocampal replays occurred during Night 1, but without learning in sleep. Unlike Figure 8E, the hippocampal weights decayed continuously following the initial encoding except for small bumps during daytime recall testing. The feedback from hippocampus to cortex was still intact; so even as the hippocampal weights decayed without the benefit of the overnight replays, there was enough replay activity to strengthen the cortical weights during SWS to the point that the cortical representation became independent of the hippocampus. Compare with Figure 9 to see the difference between lesioning the hippocampal projections to the cortex vs. having an intact hippocampus with learning disabled during sleep. We also assessed the duration of offline replays in the hippocampus across the four nights (see Supplementary Figure 4). The hippocampal replays did not progressively become faster (cf., Figure 7) even as the process of memory consolidation in the cortex continued. Overall, this simulation experiment shows that while the hippocampal projections to the cortex are critical for long-term consolidation, learning in the hippocampus during SWS may not be critical.

FIGURE 11

Figure 11. Intact network with hippocampal learning turned off during sleep. (A) Hippocampal replays occur during sleep, but with learning turned off, the hippocampal weights do not benefit and decay continuously. However, sufficient weight strengths are built up in the cortex during Night 1 such that the cortex is able to sustain replays through the subsequent nights. In this case, the cortex can still consolidate the sequence and become independent of the hippocampus. (B) Recall time, where 30 s is default for incomplete recalls. (C) Recall accuracy. (D) Weight strengths in CTX. (E) Weight strengths in HC.

Discussion

We have presented a model of the cortico-hippocampal system that quickly learns causal associations between items in sequential experiences as a function of the overlap of their dynamically changing activations for the short term, and over a longer period of time develops a stable long-term representation. The model produces memory consolidation due to emergent replay activity during SWS periods (Zola-Morgan and Squire, 1990; Squire and Alvarez, 1995; Gais and Born, 2004; Born and Wilhelm, 2012; see Figures 8, 11). As illustrated in Figure 1, our model predicts the key role of the bi-directional cortico-hippocampal dialogue to strengthen and consolidate sequential memories for long-term storage in the cortex. Figure 8 shows that with hippocampal learning intact during sleep if the model doesn't experience new episodes, the hippocampal representation of a given sequence can sustain itself almost indefinitely (or at least until the salience of the items belonging to the sequence decays to an extremely low level). Figure 9 shows that if hippocampal projections to the cortex are lesioned, the cortex is unable to develop a long-term representation. For the case that the hippocampal projections to the cortex remain intact, if there is increased competition from another sequence (due to additional practice) for replay opportunities during SWS periods before the cortical representation is developed, the less practiced sequence is not replayed enough in the hippocampus to facilitate consolidation (see Figure 10). On the other hand, plasticity is suppressed in hippocampus during SWS (Leonard et al., 1987; Bramham and Srebro, 1989), possibly to prevent changing the hippocampal representation while its replays are driving the transfer to the cortex. If the network remains intact but hippocampal learning is turned off during sleep, the cortical representation can develop even as the hippocampal representation decays (see Figure 11).

Replays and salience

Replays are primarily observed during the UP states of SWOs during SWS (Nádasdy et al., 1999), which we have simulated as shown in Figure 5. Replays have also been detected during quiet waking (Diba and Buzsáki, 2007; Carr et al., 2011). However, it has been observed that replays during SWS are more effective for consolidation than waking replays (Barnes and Wilson, 2014). This may be due to powerful bursts of activation during sharp-wave ripples in the hippocampus during SWS (Chrobak and Buzsáki, 1994), which speed up the dynamics of the sleep replays as much as 20 times over the original waking experiences (Nádasdy et al., 1999; Lee and Wilson, 2002; Euston et al., 2007; Ji and Wilson, 2007; Rasch and Born, 2013). In the brain, the temporal compression of replays means a tighter overlap between the activations of the items leading to more efficacious learning of the pertinent connections in the cortex. It also allows longer range connections, so that events that were too temporally separated during waking can be associated during sleep (Buzsáki, 2005; Lerner and Gluck, 2019). Plasticity is suppressed in the hippocampus during SWS (Leonard et al., 1987; Bramham and Srebro, 1989), but not in the cortex, so replays during sleep give the slower learning cortex more exposures to a memory at a higher effective learning rate than during the original experience. Note that in the experiment of Figure 11, even with hippocampal plasticity disabled during sleep, the memory can still be transferred to cortex successfully.

Do replays start at the beginning of a sequence, or more often in the middle due to random reactivation? Do they continue reliably to completion? Existing physiology data (Wilson and McNaughton, 1994; Nádasdy et al., 1999; Diba and Buzsáki, 2007; Ji and Wilson, 2007; Wikenheiser and Redish, 2013) correlate neural spiking during replays with entire movement sequences and can tell whether a waking experience was replayed in the forward or reverse direction, but does not clarify whether partial sequences were replayed. Our model simulates replays by activating an item in memory with a probability proportional to its salience metric (see Equation ??). If instead the items were randomly chosen to cue replays and the activation always spread in one direction to the end of the sequence, the weights of the later links would grow stronger than the earlier links since they are more likely to be adapted over the course of many replays (Lerner et al., 2019).

Without the influence of salience, old and new memories would all replay with equal frequency, reducing the opportunities for a new memory to become consolidated. The salience metric defined in Equation (??) makes it more likely for newer memories to replay. In our model, the salience of a memory is only updated during waking experience; if salience were also updated during sleep replays, runaway feedback would prevent new memories from competing for consolidation. We know that salience is not a simple function of recency and can be influenced by neuromodulators (Atherton et al., 2015). It is possible that the likelihood of replays may be reduced due to physiological factors during encoding such as low attention or high fatigue, or increased by novelty (Cheng and Frank, 2008), high emotional content (Yonelinas and Ritchey, 2015), or immediate reward (Fuchs et al., 2006; Singer and Frank, 2009). In this regard, McNamara et al. (2014) showed that optogenetic stimulation of dopaminergic neurons in rat ventral tegmental area (VTA) during spatial learning improves offline hippocampal replay strength by ~40%, in terms of correlation between wake and sleep firing patterns of hippocampal cells coding novel spatial memories. Such neuromodulation makes it possible for statistically infrequent but task-relevant events to become preferentially consolidated in the cortex.

Previous memory consolidation models

Several other computational models of memory consolidation have been published. Alvarez and Squire (1994) simulated memory transfer from short-term to long-term storage using a cortical-hippocampal neural network model with inter-areal projections but without intra-areal projections. By lesioning the hippocampal layer after training, they produced temporally graded retrograde amnesia in which memories are impaired only to the extent that they had not been consolidated before the lesion. McClelland and O'Reilly (1995) simulated the phenomenon of memory consolidation using a short-term hippocampal storage and long-term cortical storage, which were implemented as variables of memory strength. As a result of memory encoding, the hippocampal memory strength is instantiated at a higher level than the cortical memory strength. Further, the memory strength in the hippocampus decays at a faster rate compared to that in the cortex. Cortical memory strength is boosted during offline periods by the phenomenon of reactivation in proportion to the corresponding memory strength in the hippocampus. Memory recall accuracy is modeled as a function of memory strengths in both the hippocampus and the cortex. Káli and Dayan (2004) simulated a multi-modal hierarchical associative network with reciprocal projections in the slow-learning cortex and a hippocampal buffer of novel patterns from the conjunctive layer in the cortex. During offline periods, the hippocampus reactivates its memories randomly in the cortical conjunctive layer to facilitate more learning and thereby consolidation in the cortex. Note that these models use only static patterns or unitary variables to represent memories, do not consider the temporal aspects of sequential episodes, do not model the emergence of time-compressed replays that are coordinated between the hippocampus and the cortex during offline periods, and have not investigated the role of hippocampal learning and the effect of limited replay opportunities during offline periods.

Do hippocampal memory traces survive long term?

Figures 8, 9 show a hippocampal representation that survives through the course of the experiment (4 days) with no sustained decay. However, in those experiments there is no competition for limited replay opportunities during SWS periods from other sequences. And although the hippocampal memory traces decay rapidly each day (see Equation 5), they are refreshed by replays each night due to hippocampal learning that is enabled at all times. However, in the experiment where the hippocampal learning is disabled during SWS, the hippocampal representation does not survive for long (see Figure 11) because the weights are passively decaying at a high rate without any updates from the replays. Our model supports a systems-level consolidation theory in which a hippocampal representation is effectively transferred to the cortex (Buzsáki, 1989; Zola-Morgan and Squire, 1990; Marr et al., 1991; Alvarez and Squire, 1994; McClelland and O'Reilly, 1995; Squire and Alvarez, 1995; Dudai, 2004; Gais and Born, 2004; Káli and Dayan, 2004; Marshall and Born, 2007; Born and Wilhelm, 2012). A related suggestion (Alvarez and Squire, 1994; Paller and Voss, 2004) is that offline memory consolidation serves to bind together multi-modal aspects of an episode by developing cortical links between “cross-cortical storage” fragments so that a distinct episodic memory can be recalled.

It is generally accepted that cortical representations are more semantic, more generalized (Jai et al., 2018). So it is not clear how certain detailed remote episodic memories can be recalled for many years if the hippocampal representation decays completely. Multiple trace theory (MTT; Nadel et al., 2000) suggests that a hippocampal representation survives as long as the memory can be recalled, and is the source of detailed episodic memories that cannot be provided by the generalized cortical representations. The Kali and Dayan model (Káli and Dayan, 2004) supports that theory by demonstrating that if a hippocampal representation does persist, it is possible for it to stay linked to the corresponding cortical representation as it evolves over time. They speculate that the purpose of replays is to maintain the hippocampal indices by which the cortically stored memories can be retrieved. Nadel et al. (2000) also present fMRI evidence in support of MTT, which shows high hippocampal activation during recall for not only recent memories (which are dependent on the hippocampus) but also remote memories (which are presumably not dependent on the hippocampus). In our model, cueing the recall of a remote memory in the cortex would elicit the reactivation of the corresponding items of the sequence in the hippocampus due to projections from the cortex to the hippocampus even though the underlying weights have decayed significantly. In other words, just as hippocampal replays of unconsolidated, recent memories drive the replays in the cortex, the cortical recall of consolidated, remote memories drive the reactivations in the hippocampus. We suggest that the phenomenon of reconsolidation (Nader et al., 2000; Dudai, 2006), wherein so-called consolidated memories become temporarily labile following a recall and subject to modification, is facilitated by the hippocampal re-encoding of the recalled sequential experience in the cortex. If a remote memory is recalled while other items become active, causal links will be formed between all active items (see Figure 3) in the hippocampus. Hippocampal replays during subsequent offline periods can then alter the cortical representation via the mechanisms described in this paper and thereby mediate reconsolidation.

Testable predictions

Based on the experiments reported in the Results section, we can make several predictions: (1) If hippocampal projections to cortex are disabled, no cortical replays of recent memories are possible (see Figure 9). (2) The speed of replay of a particular episode increases from one night to the next, even in the absence of additional training, under certain conditions (see Figure 7); namely, the presence of hippocampal learning during SWS and the absence of other experiences during waking. With the rapid passive decay of hippocampal weights, the hippocampal replays tend to become slower with time. If learning is enabled in the hippocampus during SWS, then any replay of the episode would boost the corresponding hippocampal weights. With enough replays (due to the absence of competition from other experiences), the hippocampal weights might become stronger than when the episode was experienced during waking. And as the episode is consolidated in the cortex, the cortex can generate the replays by itself and thereby further boost the strength of the hippocampal replays and vice versa (owing to the bi-directional projections between the hippocampus and the cortex). (3) If there are no other experiences after the exposure to an episode, the decay of the hippocampal memory traces will be significantly slowed as they will be refreshed each night by replays. (4) Recent, salient memories can dominate offline consolidation via replays at the expense of older, less salient memories. Figure 10 shows how an unconsolidated sequence loses the benefit of overnight consolidation due to competition for replays from a more recent experience, causing the hippocampus representation to decay before the cortical representation can develop. Figure 7 illustrates how the more recent sequence comes to dominate the number of replays during sleep.

Conclusion

In summary, we present a first-of-its-kind computational model of the critical role of the bi-directional interactions between the hippocampus and cortex during wake and sleep for the systems-level consolidation of sequential experiences. This model simulates time-compressed replays of sequences during offline periods that are coordinated between the hippocampus and cortex, and emerge dynamically in response to the recency, frequency, and other saliency factors of the constituent items in the various experiences. Simulation experiments reveal that hippocampal learning during SWS may not be critical for cortical memory consolidation, and provide insights into how salience-driven competition for limited replay opportunities during SWS can underlie the interference among various experiences including those that do not overlap in their content. Further, the model provides an explanation in the context of systems-level consolidation for the neurophysiological evidence that has been argued in favor of the alternate multiple trace theory, as well as for the phenomenon of reconsolidation of remote memories.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

MH and PP designed the research, analyzed the data, and wrote the manuscript. MH, SS, and PP performed the research. All authors contributed to the article and approved the submitted version.

Funding

This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) via the Army Research Office (ARO) under Contract No. W911NF-16-C-0018 [Restoring Active Memory (RAM) Replay] and via Air Force Research Laboratory (AFRL) under Contract No. FA8750-18-C-0103 (Lifelong Learning Machines: L2M).

Acknowledgments

We would like to thank Dr. Nicholas Ketz and Dr. Itamar Lerner for their helpful feedback on an earlier version of this manuscript.

Conflict of interest

MH, SS, and PP were employed by HRL Laboratories, LLC.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author disclaimer

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of DARPA, ARO, AFRL, or the US Government.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnsys.2022.972235/full#supplementary-material

References

Alvarez, P., and Squire, L. R. (1994). Memory consolidation and the medial temporal lobe: a simple network model. Proc. Natl. Acad. Sci. U.S.A. 91, 7041–7045. doi: 10.1073/pnas.91.15.7041

PubMed Abstract | CrossRef Full Text | Google Scholar

Antony, J. W., Gobel, E. W., O'Hare, J. K., Reber, P. J., and Paller, K. A. (2012). Cued memory reactivation during sleep influences skill learning. Nat. Neurosci. 15, 1114–1116. doi: 10.1038/nn.3152

PubMed Abstract | CrossRef Full Text | Google Scholar

Atherton, L. A., Dupret, D., and Mellor, J. R. (2015). Memory trace replay: the shaping of memory consolidation by neuromodulation. Trends Neurosci. 38, 560–570. doi: 10.1016/j.tins.2015.07.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Barnes, D. C., and Wilson, D. A. (2014). Slow-wave sleep-imposed replay modulates both strength and precision of memory. J. Neurosci. 34, 5134–5142. doi: 10.1523/JNEUROSCI.5274-13.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

Bi, G., and Poo, M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464–10472. doi: 10.1523/JNEUROSCI.18-24-10464.1998

PubMed Abstract | CrossRef Full Text | Google Scholar

Born, J., and Wilhelm, I. (2012). System consolidation of memory during sleep. Psychol. Res. 76, 192–203. doi: 10.1007/s00426-011-0335-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Bramham, C. R., and Srebro, B. (1989). Synaptic plasticity in the hippocampus is modulated by behavioral state. Brain Res. 493, 74–86. doi: 10.1016/0006-8993(89)91001-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Buzsáki, G. (1989). Two-stage model of memory trace formation: a role for “noisy” brain states. Neuroscience 31, 551–570. doi: 10.1016/0306-4522(89)90423-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Buzsáki, G. (1998). Memory consolidation during sleep: a neurophysiological perspective. J. Sleep Res. 7, 17–23. doi: 10.1046/j.1365-2869.7.s1.3.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Buzsáki, G. (2005). Theta rhythm of navigation: Link between path integration and landmark navigation, episodic and semantic memory. Hippocampus 15, 827–840. doi: 10.1002/hipo.20113

PubMed Abstract | CrossRef Full Text | Google Scholar

Cai, D. J., Aharoni, D., Shuman, T., Shobe, J., Biane, J., Song, W., et al. (2016). A shared neural ensemble links distinct contextual memories encoded close in time. Nature 534, 115–118. doi: 10.1038/nature17955

PubMed Abstract | CrossRef Full Text | Google Scholar

Caporale, N., and Dan, Y. (2008). Spike timing–dependent plasticity: a Hebbian learning rule. Annu. Rev. Neurosci. 31, 25–46. doi: 10.1146/annurev.neuro.31.060407.125639

PubMed Abstract | CrossRef Full Text | Google Scholar

Carr, M. F., Jadhav, S. P., and Frank, L. M. (2011). Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval. Nat. Neurosci. 14, 147–153. doi: 10.1038/nn.2732

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheng, S., and Frank, L. M. (2008). New experiences enhance coordinated neural activity in the hippocampus. Neuron 57, 303–313. doi: 10.1016/j.neuron.2007.11.035

PubMed Abstract | CrossRef Full Text | Google Scholar

Chrobak, J., and Buzsáki, G. (1994). Selective activation of deep layer (V-VI) retrohippocampal cortical neurons during hippocampal sharp waves in the behaving rat. J. Neurosci. 14, 6160–6170. doi: 10.1523/JNEUROSCI.14-10-06160.1994

PubMed Abstract | CrossRef Full Text | Google Scholar

Cox, R., Korjoukov, I., de Boer, M., and Talamini, L. M. (2014). Sound asleep: processing and retention of slow oscillation phase-targeted stimuli. PLoS ONE 9:e101567. doi: 10.1371/journal.pone.0101567

PubMed Abstract | CrossRef Full Text | Google Scholar

Diba, K., and Buzsáki, G. (2007). Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci. 10, 1241–1242. doi: 10.1038/nn1961

PubMed Abstract | CrossRef Full Text | Google Scholar

Dudai, Y. (2004). The neurobiology of consolidations, or, how stable is the engram? Annu. Rev. Psychol. 55, 51–86. doi: 10.1146/annurev.psych.55.090902.142050

PubMed Abstract | CrossRef Full Text | Google Scholar

Dudai, Y. (2006). Reconsolidation: the advantage of being refocused. Curr. Opin. Neurobiol. 16, 174–178. doi: 10.1016/j.conb.2006.03.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Euston, D. R., Tatsuno, M., and McNaughton, B. L. (2007). Fast-forward playback of recent memory sequences in prefrontal cortex during sleep. Science 318, 1147–1150. doi: 10.1126/science.1148979

PubMed Abstract | CrossRef Full Text | Google Scholar

Fuchs, R. A., Feltenstein, M. W., and See, R. E. (2006). The role of the basolateral amygdala in stimulus–reward memory and extinction memory consolidation and in subsequent conditioned cued reinstatement of cocaine seeking. Eur. J. Neurosci. 23, 2809–2813. doi: 10.1111/j.1460-9568.2006.04806.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Gais, S., and Born, J. (2004). Declarative memory consolidation: mechanisms acting during human sleep. Learn. Mem. 11, 679–685. doi: 10.1101/lm.80504

PubMed Abstract | CrossRef Full Text | Google Scholar

Grossberg, S. (1976). Adaptive pattern classification and universal recoding, i: Parallel development and coding of neural feature detectors. Biol. Cybern. 23, 121–134. doi: 10.1007/BF00344744

PubMed Abstract | CrossRef Full Text | Google Scholar

Grossberg, S. (1980). How does a brain build a cognitive code? Psychol. Rev. 87, 1–51. doi: 10.1007/978-94-009-7758-7_1

PubMed Abstract | CrossRef Full Text | Google Scholar

Hahn, T., Sakmann, B., and Mehta, M. (2006). Phase-locking of hippocampal interneurons' membrane potential to neocortical up-down states. Nat. Neurosci. 9, 1359–1361. doi: 10.1038/nn1788

PubMed Abstract | CrossRef Full Text | Google Scholar

Izquierdo, I., and Medina, J. H. (1997). Memory formation: the sequence of biochemical events in the hippocampus and its connection to activity in other brain structures. Neurobiol. Learn. Mem. 68, 285–316. doi: 10.1006/nlme.1997.3799

PubMed Abstract | CrossRef Full Text | Google Scholar

Jai, Y. Y., Liu, D. F., Loback, A., Grossrubatscher, I., and Frank, L. M. (2018). Specific hippocampal representations are linked to generalized cortical representations in memory. Nat. Commun. 9:2209. doi: 10.1038/s41467-018-04498-w

PubMed Abstract | CrossRef Full Text | Google Scholar

Ji, D., and Wilson, M. A. (2007). Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat. Neurosci. 10, 100–107. doi: 10.1038/nn1825

PubMed Abstract | CrossRef Full Text | Google Scholar

Káli, S., and Dayan, P. (2004). Off-line replay maintains declarative memories in a model of hippocampal-neocortical interactions. Nat. Neurosci. 7, 286–294. doi: 10.1038/nn1202

PubMed Abstract | CrossRef Full Text | Google Scholar

Ketz, N., Jones, A. P., Bryant, N. B., Clark, V. P., and Pilly, P. K. (2018). Closed-loop slow-wave tACS improves sleep-dependent long-term memory generalization by modulating endogenous oscillations. J. Neurosci. 38, 7314–7326. doi: 10.1523/JNEUROSCI.0273-18.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

Koch, C. (2004). Biophysics of Computation: Information Processing in Single Neurons. New York, NY: Oxford University Press.

Google Scholar

Lee, A. K., and Wilson, M. A. (2002). Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 36, 1183–1194. doi: 10.1016/S0896-6273(02)01096-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Leonard, B. J., McNaughton, B., and Barnes, C. (1987). Suppression of hippocampal synaptic plasticity during slow-wave sleep. Brain Res. 425, 174–177. doi: 10.1016/0006-8993(87)90496-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Lerner, I., Ketz, N. A., Jones, A. P., Bryant, N. B., Robert, B., Skorheim, S. W., et al. (2019). Transcranial current stimulation during sleep facilitates insight into temporal rules, but does not consolidate memories of individual sequential experiences. Sci. Rep. 9:1516. doi: 10.1038/s41598-018-36107-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Lerner, I., and Gluck, M. (2019). Sleep and the extraction of hidden regularities: a systematic review and the importance of temporal rules. Sleep Med. Rev. 47, 39–50. doi: 10.1016/j.smrv.2019.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, X., Ramirez, S., Pang, P. T., Puryear, C. B., Govindarajan, A., Deisseroth, K., et al. (2012). Optogenetic stimulation of a hippocampal engram activates fear memory recall. Nature 484, 381–385. doi: 10.1038/nature11028

PubMed Abstract | CrossRef Full Text | Google Scholar

Malenka, R. C. (1994). Synaptic plasticity in the hippocampus: LTP and LTD. Cell 78, 535–538. doi: 10.1016/0092-8674(94)90517-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Marr, D., Willshaw, D., and McNaughton, B. (1991). “Simple memory: a theory for archicortex,” in From the Retina to the Neocortex, ed. L. Vaina (Boston, MA: Birkhäuser). doi: 10.1007/978-1-4684-6775-8_5

CrossRef Full Text | Google Scholar

Marshall, L., Helgadóttir, H., Mölle, M., and Born, J. (2006). Boosting slow oscillations during sleep potentiates memory. Nature 444, 610–613. doi: 10.1038/nature05278

PubMed Abstract | CrossRef Full Text | Google Scholar

Marshall, L., and Born, J. (2007). The contribution of sleep to hippocampus-dependent memory consolidation. Trends Cogn. Sci. 11, 442–450. doi: 10.1016/j.tics.2007.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar

McClelland, J. L., McNaughton, B. L., and O'Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102, 419–457. doi: 10.1037/0033-295X.102.3.419

PubMed Abstract | CrossRef Full Text | Google Scholar

McCloskey, M., and Cohen, N. J. (1989). Catastrophic interference in connectionist networks: the sequential learning problem. Psychol. Learn. Motiv. 24, 109–165. doi: 10.1016/S0079-7421(08)60536-8

CrossRef Full Text | Google Scholar

McNamara, C. G., Tejero-Cantero, Á., Trouche, S., Campo-Urriza, N., and Dupret, D. (2014). Dopaminergic neurons promote hippocampal reactivation and spatial memory persistence. Nat. Neurosci. 17, 1658–1660. doi: 10.1038/nn.3843

PubMed Abstract | CrossRef Full Text | Google Scholar

Michelmann, S., Staresina, B. P., Bowman, H., and Hanslmayr, S. (2019). Speed of time-compressed forward replay flexibly changes in human episodic memory. Nat. Hum. Behav. 3, 143–154. doi: 10.1038/s41562-018-0491-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Mickus, T., Jung, H.-Y., and Spruston, N. (1999). Properties of slow, cumulative sodium channel inactivation in rat hippocampal CA1 pyramidal neurons. Biophys. J. 76, 846–860. doi: 10.1016/S0006-3495(99)77248-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Nádasdy, Z., Hirase, H., Czurkó, A., Csicsvari, J., and Buzsáki, G. (1999). Replay and time compression of recurring spike sequences in the hippocampus. J. Neurosci. 19, 9497–9507. doi: 10.1523/JNEUROSCI.19-21-09497.1999

PubMed Abstract | CrossRef Full Text | Google Scholar

Nadel, L., Samsonovich, A., Ryan, L., and Moscovitch, M. (2000). Multiple trace theory of human memory: computational, neuroimaging, and neuropsychological results. Hippocampus 10, 352–368. doi: 10.1002/1098-1063(2000)10:4<352::AID-HIPO2>3.0.CO;2-D

PubMed Abstract | CrossRef Full Text | Google Scholar

Nader, K., Schafe, G. E., and Le Doux, J. E. (2000). Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval. Nature 406, 722–726. doi: 10.1038/35021052

PubMed Abstract | CrossRef Full Text | Google Scholar

Paller, K. A., and Voss, J. L. (2004). Memory reactivation and consolidation during sleep. Learn. Mem. 11, 664–670. doi: 10.1101/lm.75704

PubMed Abstract | CrossRef Full Text | Google Scholar

Pilly, P. K., Skorheim, S. W., Hubbard, R. J., Ketz, N. A., Roach, S. M., Lerner, I., et al. (2019). One-shot tagging during wake and cueing during sleep with spatiotemporal patterns of transcranial electrical stimulation can boost long-term metamemory of individual episodes in humans. bioRxiv 10.1101:672378. doi: 10.1101/672378

PubMed Abstract | CrossRef Full Text | Google Scholar

Ramirez, S., Liu, X., Lin, P. A., Suh, J., Pignatelli, M., Redondo, R. L., et al. (2013). Creating a false memory in the hippocampus. Science 341, 387–391. doi: 10.1126/science.1239073

PubMed Abstract | CrossRef Full Text | Google Scholar

Rasch, B., Büchel, C., Gais, S., and Born, J. (2007). Odor cues during slow-wave sleep prompt declarative memory consolidation. Science 315, 1426–1429. doi: 10.1126/science.1138581

PubMed Abstract | CrossRef Full Text | Google Scholar

Rasch, B., and Born, J. (2013). About sleep's role in memory. Physiol. Rev. 93, 681–766. doi: 10.1152/physrev.00032.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Ratcliff, R. (1990). Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychol. Rev. 97, 285–308. doi: 10.1037/0033-295x.97.2.285

PubMed Abstract | CrossRef Full Text | Google Scholar

Robertson, E. M. (2012). New insights in human memory interference and consolidation. Curr. Biol. 22, R66–R71. doi: 10.1016/j.cub.2011.11.051

PubMed Abstract | CrossRef Full Text | Google Scholar

Rudoy, J. D., Voss, J. L., Westerberg, C. E., and Paller, K. A. (2009). Strengthening individual memories by reactivating them during sleep. Science 326:1079. doi: 10.1126/science.1179013

PubMed Abstract | CrossRef Full Text | Google Scholar

Sanchez-Vives, M. V. (2020). Origin and dynamics of cortical slow oscillations. Curr. Opin. Physiol. 15, 217–223. doi: 10.1016/j.cophys.2020.04.005

CrossRef Full Text | Google Scholar

Scoville, W. B., and Milner, B. (1957). Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11–21. doi: 10.1136/jnnp.20.1.11 .

PubMed Abstract | CrossRef Full Text | Google Scholar

Singer, A. C., and Frank, L. M. (2009). Rewarded outcomes enhance reactivation of experience in the hippocampus. Neuron 64, 910–921. doi: 10.1016/j.neuron.2009.11.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Skaggs, W. E., and McNaughton, B. L. (1996). Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science 271, 1870–1873. doi: 10.1126/science.271.5257.1870

PubMed Abstract | CrossRef Full Text | Google Scholar

Squire, L. R. (1992). Declarative and nondeclarative memory: multiple brain systems supporting learning and memory. J. Cogn. Neurosci. 4, 232–243. doi: 10.1162/jocn.1992.4.3.232

PubMed Abstract | CrossRef Full Text | Google Scholar

Squire, L. R., and Alvarez, P. (1995). Retrograde amnesia and memory consolidation: a neurobiological perspective. Curr. Opin. Neurobiol. 5, 169–177. doi: 10.1016/0959-4388(95)80023-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Tulving, E. (1972). Episodic and semantic memory. Organ. Mem. 1, 381–403.

Google Scholar

Tulving, E., and Markowitsch, H. J. (1998). Episodic and declarative memory: role of the hippocampus. Hippocampus 8, 198–204. doi: 10.1002/(SICI)1098-1063(1998)8:3<198::AID-HIPO2>3.0.CO

PubMed Abstract | CrossRef Full Text | Google Scholar

Wikenheiser, A. M., and Redish, A. D. (2013). The balance of forward and backward hippocampal sequences shifts across behavioral states. Hippocampus 23, 22–29. doi: 10.1002/hipo.22049

PubMed Abstract | CrossRef Full Text | Google Scholar

Wilson, M. A., and McNaughton, B. L. (1994). Reactivation of hippocampal ensemble memories during sleep. Science 265, 676–679. doi: 10.1126/science.8036517

PubMed Abstract | CrossRef Full Text | Google Scholar

Yokose, J., Okubo-Suzuki, R., Nomoto, M., Ohkawa, N., Nishizono, H., Suzuki, A., et al. (2017). Overlapping memory trace indispensable for linking, but not recalling, individual memories. Science 355, 398–403. doi: 10.1126/science.aal2690

PubMed Abstract | CrossRef Full Text | Google Scholar

Yonelinas, A. P., and Ritchey, M. (2015). The slow forgetting of emotional episodic memories: an emotional binding account. Trends Cogn. Sci. 19, 259–267. doi: 10.1016/j.tics.2015.02.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Zola-Morgan, S., and Squire, L. R. (1990). The primate hippocampal formation: evidence for a time-limited role in memory storage. Science 250, 288–290. doi: 10.1126/science.2218534

PubMed Abstract | CrossRef Full Text | Google Scholar

Zucca, S., D'Urso, G., Pasquale, V., Vecchia, D., Pica, G., Bovetti, S., et al. (2017). An inhibitory gate for state transition in cortex. eLife 6:e26177. doi: 10.7554/eLife.26177

PubMed Abstract | CrossRef Full Text | Google Scholar