How Can Transcranial Magnetic Stimulation Be Used to Modulate Episodic Memory?: A Systematic Review and Meta-Analysis

A systematic review and meta-analysis were conducted to synthesize the existing literature on how transcranial magnetic stimulation (TMS) has been used to modulate episodic memory. Given the numerous parameters of TMS protocols and experimental design characteristics that can be manipulated, a mechanistic understanding of how changes in the combination of parameters (e.g., frequency, timing, intensity, targeted brain region, memory task) modulate episodic memory is needed. To address this, we reviewed 59 studies and conducted a meta-analysis on 245 effect sizes from 37 articles on healthy younger adults (N = 1,061). Analyses revealed generally more beneficial effects of 1-Hz rTMS vs. other frequencies on episodic memory. Moderation analyses revealed complex interactions as online 20-Hz rTMS protocols led to negative effects, while offline 20-Hz rTMS led to enhancing effects. There was also an interaction between stimulation intensity and frequency as 20-Hz rTMS had more negative effects when applied below- vs. at-motor threshold. Conversely, 1-Hz rTMS had more beneficial effects than other frequencies when applied below- vs. at- or above-motor threshold. No reliable aggregate or hypothesized interactions were found when assessing stimulation site (frontal vs. parietal cortex, left vs. right hemisphere), stimulated memory process (during encoding vs. retrieval), the type of retrieval (associative/recollection vs. item/familiarity), or the type of control comparison (active vs. sham or no TMS) on episodic memory. However, there is insufficient data to make strong inference based on the lack of aggregate or two-way interactions between these factors, or to assess more complex (e.g., 3-way) interactions. We reviewed the effects on other populations (healthy older adults and clinical populations), but systematic comparison of parameters was also prevented due to insufficient data. A database of parameters and effects sizes is available as an open source repository so that data from studies can be continuously accumulated in order to facilitate future meta-analysis. In conclusion, modulating episodic memory relies on complex interactions among the numerous moderator variables that can be manipulated. Therefore, rigorous, systematic comparisons need to be further investigated as the body of literature grows in order to fully understand the combination of parameters that lead to enhancing, detrimental or null effects on episodic memory.

Over the past 30 years, transcranial magnetic stimulation (TMS) has enabled researchers to move beyond correlational research and address causal relations between brain and behavior in humans. While powerful, the mechanisms of TMS and the resulting outcomes on higher order cognitive functioning such as episodic memory are complex and mired by differences across studies. As the body of research on TMS continues to emerge, rigorous, and systematic comparisons need to be investigated to better understand the moderating effects of the various parameters of TMS that can be manipulated. To this aim, we first reviewed studies that have used repetitive TMS (rTMS) in attempt to modulate episodic memory. Next, we conducted a meta-analysis to bridge the gap in understanding of aspects of rTMS protocols that lead to memory enhancement or impairment. Lastly, we reviewed and compared the limited number of studies that have used rTMS to modulate episodic memory in healthy older adults and those who suffer from clinical disorders.

In a systematic review, we focus on examining the effects of rTMS on episodic memory, which entails vividly remembering past events along with their spatial, temporal, and/or source details (Tulving, 2002). A full understanding of the mechanisms underlying cognitive processes in episodic memory is critical for developing effective interventions to target memory deficits that occur in healthy aging and in those with clinical disorders. However, the neural substrates of episodic memory are difficult to directly observe and manipulate and the field has generally relied on neuroimaging techniques that have provided important, albeit correlational, findings linking neural activity in specific brain regions with episodic memory processes. The introduction of TMS techniques has made it possible to make more causal claims about the role of specific brain regions (and functionally coupled regions) in specific processes by temporarily and reversibly activating neuronal firing in targeted cortical regions. Such exogenous stimulation can induce the activation of specific neural circuits and/or perturb or disrupt endogenous neural activity in targeted and functionally connected areas, and then allows for investigation of the resulting effects on behavior, as well as on associated neural signals when TMS is coupled with simultaneous neuroimaging. Therefore, it is critical to both understand the ways in which varying rTMS parameters can reveal the role of neurocognitive processes in supporting episodic memory, and how rTMS may be used to enhance or impair episodic memory functioning (for early reviews, see Grafman and Wassermann, 1998; Manenti et al., 2012). Until now, there has been no systematic review or meta-analysis on the effects of rTMS on episodic memory.

Principles of rTMS and Episodic Memory That Are Important for Researchers to Understand

When conducted properly, TMS is a relatively safe and painless technique that utilizes electromagnetism to induce current in the brain that can cause neurons to fire (Rossi et al., 2009). During TMS, an electrical current passes through a coil of insulated wiring and produces brief magnetic “pulses” to induce neural firing. These transitory currents cause rapid depolarization (i.e., action potentials) of neurons beneath the coil and can modulate cortical excitability with a relatively high level of spatial and temporal precision (Wagner et al., 2009). Importantly, through polysynaptic connections, action potentials can propagate to distal brain regions that are functionally connected at the time of stimulation.

The application of TMS protocols can be roughly classified with respect to the amount of time between pulses. For example, single and paired pulse procedures vs. repetitive TMS reflect differences in the time between delivering pulses. rTMS has been implemented to modulate cortical excitability with the intent to either enhance or attenuate subsequent cognitive and behavioral outcomes (Pascual-Leone et al., 1998). Stimulation parameters such as frequency, intensity, and stimulation timing have been shown to be critical in achieving the desired response. Multimodal techniques (i.e., TMS-EEG, TMS-fMRI) and machine learning analyses (i.e., multivariate pattern analysis) have revealed the importance of understanding the relationship between the anatomical and functional brain regions being targeted and the cognitive representations and processes that are coded in the neural signals and may be manipulated by stimulation. In other words, stimulation parameters interact with ongoing patterns of neural activity to drive changes in cognition and behavior (Silvanto et al., 2008; Romei et al., 2016). TMS techniques have grown in popularity due the feasibility of non-invasively modulating brain activity to study the role of specific brain regions in cognitive processes and modify behavior in awake, behaving humans. The techniques have far reaching implications within the field of memory for the development of treatments for people suffering from memory deficits due to normal aging or neurodegenerative disorders. As reviewed below, findings are often quite mixed and variable between individuals; thus, increased understanding of the most proficient rTMS protocols for effectively enhancing episodic memory is critical in order to achieve translational applications in clinical settings.

Episodic Memory

Memory is not an isomorphic construct and, of the various forms of memory, episodic memory is the form that is absolutely critical for maintaining one's self-identity. According to Tulving (2002), episodic memory is the type of declarative memory that represents knowledge of particular events that one has experienced as well as their contextual acquisition (e.g., autobiographical memory), such as when and where an event occurred (e.g., who you met at a dinner party last week). It is measured by retrieval accuracy on declarative memory tests such as recall and recognition. Episodic memory requires and extends a related type of declarative memory, i.e., semantic memory or one's general knowledge of the world. Semantic memory involves retrieving knowledge (i.e., facts, concepts, meaning) without the accompanying contextual information (e.g., knowing how many planets are in the solar system, but without remembering in vivid detail when or where this was learned). The critical role of the hippocampus and surrounding structures of the medial temporal lobes (MTL) in the encoding, consolidation, and retrieval of episodic memories has been highlighted by striking cases of amnesia following either surgical resection (as in patient HM), disease (as in Clive Wearing), or brain trauma (as in patient KC), and have been elucidated by decades of animal/behavioral, neuropsychological, neuroimaging, and neurostimulation research (for reviews see, Moscovitch et al., 2016; Eichenbaum, 2017).

The hippocampus alone does not support episodic memory, however. In order for an event to be remembered, it must first be encoded through some combination of perceptual (e.g., visual, acoustic), affective, contextual, and semantic processes that integrate an event with its contextual associations to form episodic memories. Additionally, executive control processes are thought to facilitate the multi-modal binding of features and associations for episodic memory representations and are thought to be mediated by regions of the prefrontal cortex (PFC) and parietal cortex (PC; Buckner et al., 1999). Encoding and retrieval processes are inherently intertwined. Once encoded, successful retrieval demonstrates access to a memory, which also serves as a measure for successful encoding. For example, the encoding specificity principle (Tulving and Thomson, 1973) and related concept of “transfer-appropriate-processing” (Morris et al., 1977; Blaxton, 1989) emphasize the role of a reinstatement of the encoding context during retrieval for episodic memory. At the neural level, context reinstatement is operationalized by the degree of overlap in neural activity elicited during encoding and retrieval, and functional neuroimaging studies have revealed the involvement of regions in the prefrontal (Prince, 2005), medial temporal (Rugg and Vilberg, 2013), and parietal cortices (Wagner et al., 2005) (for review, see Craik and Rose, 2012).

The Role of Encoding and Retrieval

Although encoding and retrieval are intertwined, there are important differences in the neural correlates of successful episodic encoding and retrieval. For example, there is often a hemispheric asymmetry in left vs. right PFC activation during encoding vs. retrieval, depending on the task stimuli, familiarity, and the cognitive processes engaged (see below). With regards to rTMS, it is important to note that the neural substrates of episodic memory are impacted by the underlying state of activation in a stimulated brain region that is driven by the task demands and stimuli (e.g., words vs. pictures; Miniussi et al., 2013; Romei et al., 2016).

Verbal Stimuli

It is generally theorized that, in younger adults, the left PFC is critical for encoding, while the right PFC is to be preferentially involved in retrieval (Habib et al., 2003). This is known as the hemispheric encoding-retrieval asymmetry (HERA) model, which has been supported by some evidence from rTMS findings (Rossi et al., 2004). However, the role of the left and right PFC in episodic memory may also vary due to numerous factors, such as task stimuli. For example, in two studies, left DLPFC stimulation impaired memory for verbal information (Rami et al., 2003), while right DLPFC stimulation impaired memory for non-verbal stimuli (Epstein et al., 2002; Floel et al., 2004). However, differences between the parameters of stimulation such as frequency and intensity across studies complicate direct comparison.

Stimulus Familiarity

The effects of rTMS on episodic memory has also been shown to vary with an individual's familiarity with the type of information that is to be remembered. For example, online 20 Hz rTMS to the left DLPFC during retrieval impaired novel compared to familiar information (Sandrini et al., 2003). Analogously, rTMS has been shown to preferentially affect memory for contextual information of studied associations. For example, online 10 Hz rTMS to the left inferior frontal gyrus (IFG) during encoding of novel faces paired with a “context” stimulus (e.g., the word “lawyer”) impaired subsequent recognition of faces compared to faces studied without a context stimulus (i.e., faces studied with “no context”; Feurra et al., 2010). Thus, effects of rTMS on episodic memory likely depends not only on the specific rTMS parameters (e.g., frequency, intensity, stimulation timing, or target site), but also on the cognitive processes that are tapped by the type of information presented or the way in which it is to be remembered.

Cognitive Processes Engaged

In a similar vein, the effects of rTMS on episodic memory likely also depends on the cognitive strategies involved during encoding and/or retrieval (Manenti et al., 2010b, 2012; Hawco et al., 2013). For example, Innocenti et al. (2010) had individuals engage in “deep” (semantic) or “shallow” (perceptual) levels of processing while encoding words and applying online 10 Hz rTMS to either the left or right DLPFC. There were dissociable effects on recognition of words that were deeply or shallowly encoded, with left DLPFC stimulation abolishing the typical benefit of deep processing on memory (Innocenti et al., 2010). Relatedly, in a face-name pair encoding task, participants received fMRI-guided 10 Hz rTMS to either the left or right DLPFC during retrieval. After subdividing performance for those who reported using a retrieval strategy from those who reported using no retrieval strategy, it was concluded that right DLPFC stimulation disrupted retrieval for strategy users, while left DLPFC stimulation disrupted retrieval for non-strategy users (Manenti et al., 2010b).

Taken together, the effects of rTMS on episodic memory likely depend not only on the specific parameters of the protocol, but also on the brain state and cognitive processes engaged at the time of stimulation. This brain-state dependence is consistent with research that has used TMS in a wide variety of domains that has shown that the effects of TMS to a targeted region interacts with the ongoing, endogenous neural activity in that region and functionally coupled regions at the time of stimulation. Thus, while patterns are complicated across conditions, this principle enables researchers to investigate the causal role of the targeted and functionally coupled regions in specific neurocognitive mechanisms through the systematic manipulation of different task conditions (for review see, Romei et al., 2016).

The Role of Retrieval Processes

According to dual-process models of episodic memory, the cognitive processes engaged during retrieval can be supported by two separate processes (i.e., familiarity and recollection). A central theme of dual-process models is that previously studied information supports recognition by being generally more “familiar” than new/non-studied information. A distinct retrieval process, “recollection,” can further facilitate recognition of previously studied information through the retrieval of associated spatial, temporal, or contextual information (Yonelinas, 2002). The remember/know procedure is commonly implemented to examine differences in the nature or subjective experience of retrieval. Specifically, during a recognition task, participants indicate if the decision to respond that a presented item was previously seen (i.e., old) was based on recollection of the encoding details or based on a feeling of familiarity; they are instructed to respond with “remember” or “know,” respectively, (Tulving, 1985).

rTMS enables the ability to causally investigate the underlying neural substrates of familiarity and recollection processes. For example, Turriziani et al. (2008) applied online 20 Hz rTMS to the left or right DLPFC during either encoding or retrieval (i.e., remember/know paradigm) of faces. They found that recollection (remember responses) was impaired following right compared to left DLPFC stimulation during encoding. However, familiarity was impaired following left and right DLPFC stimulation during encoding. These findings suggest that both the left and right DLPFC support familiarity and recollective retrieval processes, but the left vs. right DLPFC may contribute differentially depending on the type of retrieval process engaged.

The Role of Cortical Regions

The traditional view of episodic memory has focused on the hippocampus and its role in binding specific details of an experience (such as what events have occurred) with when (temporal), where (spatial), or how (source) the events occurred (Tulving, 1985, 2002). A large body of work on patient H.M., who had extensive surgical resection of the bilateral hippocampi and surrounding structures of the MTL in young adulthood, revealed the importance of the hippocampus in the formation of new episodic memories (Squire, 2009). However, inferences from brain lesion studies are often confounded by compensatory reorganization mechanisms (e.g., plasticity) that occur following brain injury and/or damage that is not isolated to specific brain regions. Alternatively, TMS can be used to causally modulate neural activity in healthy brain regions and networks at the precise time that a hypothesized cognitive operation is thought to occur; it affords a direct test of how specific brain regions contribute to episodic memory. However, the limited spatial ability to target deep regions in the MTL has led to substantial focus on targeting neocortical regions. The combination of TMS with neuroimaging (e.g., fMRI, EEG) and functional connectivity analyses provides a promising route to understanding brain network interactions, and has enabled further understanding of how TMS related changes in local neural activity propagate through polysynaptic connections to distal regions that are functionally connected at the time of stimulation (Shafi et al., 2012). For example, several studies have shown effects on episodic memory from targeting nodes in neocortical networks in prefrontal cortex or parietal cortex that are functionally connected with the hippocampus (e.g., Wang et al., 2014).

Prefrontal Cortex

The PFC in particular has received substantial attention as a stimulation site in rTMS protocols. For example, a fMRI-localized rTMS study targeted the left inferior prefrontal cortex (LIPFC) online during a word-pair encoding task with 7 Hz rTMS. In comparison to two active control conditions (i.e., RIPFC, left superior partial cortex; LPC), stimulation of the LIPFC enhanced subsequent memory on an immediate word recognition test. The authors suggested that this causal link between LIPFC and episodic memory may have been driven by LIPFC stimulation triggering elaborative processing of word pairs that led to more distinctive memory cues (Köhler et al., 2004). One important design aspect of this study that we will briefly return to later is the fMRI-localization targeting procedure that accounts for individual differences in the structure and function of stimulated brain regions for more accurate targeting of regions of interest in a specific task.

Parietal Cortex

While PFC-MTL interactions are a cornerstone of neurocognitive models of episodic memory, it is also clear that there is extensive, often task-dependent recruitment of distributed networks of areas that process sensory, semantic, or emotional information to support various aspects of episodic memory (Ciaramelli et al., 2008; Cabeza and Moscovitch, 2013). Thus, research has examined the posterior parietal cortex due to its hypothesized role in attending to memory representations. It has been suggested that regions of parietal cortex are critical for episodic retrieval because of their role in either the formation of memory representations or shifts in attention for decision making processes, or both (Dobbins et al., 2003).

For example, Manenti et al. (2010a) implemented an rTMS-fMRI design to investigate the causal functioning of both the parietal cortex and DLPFC during encoding and retrieval. The authors applied online 10 Hz rTMS to either the left or right DLPFC or individualized regions of the left or right parietal cortex (either supramarginal or angular gyrus) during either encoding or retrieval. The authors demonstrated word recognition decrements (i.e., slower reaction times) following rTMS to the left parietal cortex (compared to sham). Furthermore, those with larger rTMS-related memory decrements were those who demonstrated a larger degree of right parietal activation during retrieval in the initial fMRI-localization task (Manenti et al., 2010a). One implication from these findings is that different aspects of episodic memory performance (e.g., accuracy, reaction times, subjective confidence) may be supported differentially by the PFC vs. PC.

Additionally, the null memory results reported (Manenti et al., 2010a) contrast with previously reported findings of Innocenti et al. (2010), despite the use of many of the same parameters (i.e., 10 Hz, left and right DLPFC, encoding, and retrieval of words). One possible explanation may be attributable to differences in other parameters, such as stimulation intensity (i.e., 100 vs. 90%). Similarly, the findings of Manenti et al. (2010a) also differed from Köhler et al. (2004) despite implementing many of the same parameters (i.e., online stimulation at the same intensity to the PFC and PC during word encoding). However, differences in the stimulation frequency (i.e., 7 vs. 10 Hz) might account for the different results.

It should be noted that studies often select a stimulation site to have a selective (or, more likely, stronger) effect on one cognitive domain, but it must be acknowledged that stimulated brain regions may be implicated in numerous related cognitive processes. For example, a meta-analysis revealed that rTMS to the DLPFC improved working memory performance (e.g., accuracy, reaction times; Brunoni and Vanderhasselt, 2014). The studies included in the Brunoni and Vanderhasselt (2014) meta-analysis targeted the DLPFC because of its role in short-term (STM)/working memory (WM). Thus, subsequent assessments of the rTMS effects on long-term episodic memory should be interpreted based on possible modulation of STM/WM processes on encoding and maintenance processes that may subsequently result in modulation of long-term memory. Indeed, a number of studies have examined the effects of rTMS to the DLPFC during STM/WM and has assessed the effects of rTMS on subsequent long-term memory (Nilakantan et al., 2017; Marin et al., 2018). For example, Marin et al. (2018) found that continuous theta burst (cTBS) to the right DLPFC prior to the encoding of objects and their spatial location was impaired for subsequent long-term memory tasks (i.e., object recognition and spatial location recall), but short-term memory performance (i.e., short-term spatial location recall) was left unaffected compared to controls.

Taken together, the findings reviewed thus far build upon previous correlational neuroimaging findings that the PFC, PC, and MTL are important for episodic memory, but systematic examination of encoding, consolidation, and retrieval processes, as well as potential interactions among neocortical regions, is needed. For example, the functional roles of these regions in recollection and familiarity (Yonelinas, 2005) remains to be fully understood.

The Role of Stimulation Frequency, Intensity, and Timing

How rTMS can enhance or impair memory (or leave it unaffected) depends on numerous factors regarding the ways in which the rTMS protocols are implemented. A detailed review of all technical considerations regarding rTMS is beyond the scope of the current paper (for details, see Kammer et al., 2001; Rossi et al., 2009; Sauvé and Crowther, 2014). The main stimulation characteristics considered here are stimulation frequency, intensity, and timing. A common approach in selecting a stimulation frequency generally involves applying a protocol hypothesized to either enhance cortical excitability, such as a high frequency (>5 Hz) or intermittent theta burst (iTBS) protocol, or suppress cortical excitability, such as a low frequency (<1 Hz) or cTBS protocol (Huang et al., 2005; Hallett, 2007). However, these hypotheses stem largely from physiological findings that have been investigated in the motor cortex with the assumption that this remains true for other cortical regions.

Another way in which different stimulation frequencies may modulate neural activity is through altering neural oscillatory activity through the resetting of natural oscillations, and in some cases, driving neural entrainment that may influence communication between brain regions and cognitive performance (Luber and Lisanby, 2014). Importantly, this modulation of neural activity is assumed to be specific to functionally relevant frequencies. Because relevant oscillations may vary in a continuous fashion between tasks, across phases of a task, and/or between individuals, considerable research is needed to fully understand how rTMS frequency can modulate episodic memory processes. A major aim of the current meta-analysis is to help elucidate the aggregate and complex interactions associated with the effects of different rTMS frequencies on episodic memory.

One of the many factors that will influence the effectiveness of rTMS is stimulation intensity. Generally, researchers will adjust stimulation intensity based upon an individual's resting or active motor threshold (RMT, AMT; Ngomo et al., 2012), which is defined as the minimal stimulator intensity needed to induce a muscle evoked potential in 5 out of 10 occurrences in either a relaxed (RMT) or actively engaged muscle (AMT). Prior work has found evidence that low (below) and high (above) stimulation intensities mirror facilitatory vs. inhibitory neural activity that is also associated with enhancements vs. decrements in behavioral outcomes (Silvanto et al., 2018). In addition, recent findings have shown that the effects of stimulation intensity on cortical excitability also may interact with specific frequencies, which in turn can lead to divergent effects on cognitive performance (Chung et al., 2018). Furthermore, the effects of varying stimulation intensity may covary with numerous factors such as the “brain state” or the cortical excitability of targeted and functionally coupled regions at the time of stimulation (Romei et al., 2016). Thus, a systematic investigation of the effects of stimulation intensity and how it interacts with frequency is needed. Elucidating the effects of varying stimulation intensity on episodic memory was another aim of the current study.

When stimulation is applied (i.e., stimulation timing parameter) is also an important factor that enables researchers to investigate the time course of episodic memory processes. The timing of the stimulation protocol reflects differences in whether stimulation occurs simultaneously with a cognitive process (online; e.g., rTMS during picture encoding) or prior to or following a cognitive process (offline; e.g., rTMS prior to picture encoding). One reason that online and offline stimulation may affect episodic memory differently is that they may affect different mechanisms, with online stimulation having an immediate influence on brain activity while offline stimulation often results in after-effects up to an hour after stimulation (Bergmann et al., 2016). In other words, online approaches can interfere or enhance neural activity through immediate depolarization of targeted neurons, while offline approaches can modulate long-term potentiation- or long-term depression-like plasticity mechanisms to either enhance or inhibit episodic memory processes. In a previous meta-analysis of different forms of non-invasive brain stimulation on various cognitive measures, differences in stimulation timing were found, with larger enhancing effects occurring for online stimulation (Hsu et al., 2015). Importantly, stimulation timing may interact with stimulation intensity, which could lead to facilitating or impairing effects on cognition (Silvanto and Cattaneo, 2017).

The Importance of Proper Control Conditions

The implementation of a proper control conditions is important to note given the variety of different control comparisons that are available. A proper control comparison is critical in order to assess whether the observed effects on performance are due specifically to the effects of applying TMS to the targeted region as opposed to some other variable, such as non-specific effects associated with the sensations of stimulation (Sandrini et al., 2011). Active stimulation control conditions that involve stimulating some other brain region that is not hypothesized to be as involved in the specific process of interest as the experimentally targeted region at the same intensity and frequency in both conditions are ideal because they most closely replicate the same aspects of the experimental/treatment TMS procedures. Sham stimulation controls (e.g., sham coil or tilting the coil away from the scalp) are implemented to mimic the non-specific effects of TMS (i.e., the discharging sound and scalp contact of the coil). “No TMS” conditions provide the least amount of control for non-specific effects of rTMS. In order to test if the type of control condition moderated the size of rTMS effects on episodic memory we examined whether there were systematic differences between effect sizes associated with comparisons to active, active vertex, sham, or no TMS control conditions.

The Present Study

A clear understanding of when episodic memory is enhanced, impaired, or left unaffected should depend on a wide variety of factors regarding the ways in which the rTMS protocols are implemented, and a subset of these factors can be roughly classified into specific rTMS parameters (e.g., frequency, intensity, timing, stimulation site, and control comparisons) and experimental design characteristics (e.g., memory processes: encoding vs. consolidation vs. retrieval, and retrieval type: familiarity vs. recollection). Contrasting the effects of different parameters allows researchers to test hypotheses about the role of specific neurocognitive processes that support different aspects of episodic memory. For example, rTMS could be used to influence many different processes associated with the acquisition of an event (encoding), its storage and preservation over time (consolidation processes), and/or the ability to remember the event (retrieval processes). Collectively, these different stages may engage both, similar, or distinct brain regions and, thus, the targeted brain regions and the nature of stimulation should result in variable effects. With this in mind, we conducted our survey of the literature to examine how rTMS has been shown to modulate episodic memory with a specific focus on how, when, and where rTMS was applied, as well as how episodic memory was measured.

A significant hurdle with understanding how rTMS impacts episodic memory is that most studies differed in a number of ways, such as in stimulation parameters (e.g., frequency, intensity, timing, and stimulation site), or with respect to how episodic memory was measured. Such differences make it difficult to pinpoint the specific protocols that will enhance or impair episodic memory. In the current study, we conducted a comprehensive systematic review of the literature with the aim of identifying the aggregate effects of rTMS on episodic memory and interactions among various factors. The focus of the meta-analysis was on healthy younger adults due to the small number of older adult rTMS studies. To the extent of our knowledge, this is the first meta-analysis to investigate this relationship. Such a meta-analysis is important for drawing consistent conclusions regarding the circumstances that lead to rTMS impacting episodic memory.

In order to carry this out, our primary aim was to investigate the effects of potential moderators of effect sizes associated with specific rTMS parameters or experimental design characteristics. Specific rTMS parameters included: stimulation frequency (1, 5, 10, 20, iTBS, cTBS), intensity (above, at, and below 100% MT), timing (online vs. offline), targeted hemisphere (left vs. right), targeted cortical region (frontal cortex vs. parietal cortex), and control conditions (active, active vertex, sham, and no TMS). Experimental design moderators included: memory processes stimulated (encoding vs. retrieval) and retrieval type (recollection vs. familiarity). Relatedly, we attempted to elucidate the complex relationship between rTMS and episodic memory by examining two-way interactions between factors that are hypothesized to interact. This allowed us to begin to bridge the gap in the understanding of the circumstances in which rTMS will lead to memory enhancement, impairment, or leave memory unaffected. Although it is not yet possible to conduct a similar meta-analysis of the effects of rTMS on episodic memory in older adults and some clinical conditions, we provide a brief narrative review of the available studies for comparison.

Method

We pursued all possible pertinent articles that reported data (e.g., group means, mean differences, tables, and bar graphs with error information) regarding the relationship between rTMS and episodic memory performance, and coded relevant parameters for potential moderator variables. First, we detail how the literature search was conducted for the systematic review, including the inclusion criteria that were implemented and selection protocol. Next, we discuss the subset used for the meta-analysis, which will focus on rTMS and episodic memory in healthy younger adults given that they account for a majority of the findings reported in the literature. Details for the coding system and how effect sizes were calculated and implemented in a three-level random effects meta-analysis are reported below.

Literature Search

An exhaustive PubMed search was conducted to select studies for the review and meta-analysis with a combination of the following search terms: “transcranial magnetic stimulation,” “TMS,” “rTMS,” “Theta burst stimulation,” “TBS,” “memory,” “episodic memory,” “long term memory,” “associative memory,” “recollection,” “mild cognitive impairment.” The search for studies ended in July 2018. Our literature search yielded 484 studies from PubMed, and references from these relevant articles were examined for possible inclusion in our analysis. In line with previous protocols, when the title and abstract were insufficient to identify the study for inclusion, the full-text underwent further examination. In cases of uncertainty, articles were examined by the authors and research assistants and removed based on mutual agreement. The most frequent reasons for exclusion were that articles (a) did not apply TMS or (b) did not assess episodic memory, see Figure 1.

FIGURE 1

Figure 1. A total of 59 articles were included in the systematic review. The meta-analysis focused on healthy younger adults and rTMS in episodic memory resulting in 37 articles with 245 effect sizes.

Inclusion Criteria and Selected Studies

For inclusion in the systematic review, the studies were limited to (1) single pulse TMS, paired pulse TMS, rTMS, or TBS protocols that measured (2) episodic memory with a free-recall, cued-recall, or recognition test (3) in healthy younger adults, (4) healthy older adults, or (5) older adults with a clinical condition, (6) and those that were published in English.

A total of 59 studies were incorporated into the systematic review based on the search terms and inclusion criteria. However, inclusion in the meta-analysis was focused on rTMS studies of healthy younger adults. Seven of the remaining studies were excluded from the meta-analysis because there was insufficient reporting of data necessary to calculate effect sizes (e.g., means, SDs, t-, or F-values). Thus, 37 articles examining the effects of rTMS on episodic memory in healthy younger adults were included in the meta-analysis. See Figure 1 for a flow diagram illustrating this process. A total number of 245 effect sizes were extracted from the selected studies. For experiments that pooled participants' data across multiple studies, only participants who were uniquely recruited for each study were analyzed. Details about the calculation of effect sizes are described below in the data extraction section.

Quality Assessment

In order to assess the methodological quality of the selected studies, we were guided by the assessment criteria recommended by Higgins et al. (2008), which is to categorize study characteristics as low risk, unknown risk, or high risk based on whether there was random allocation of participants to conditions, blinding of participants, blinding of experimenter, blinding of experimenter assessing the outcome measure(s), selective reporting of outcome measures, and participant attrition. Characteristics were coded as unknown when insufficient details were provided; studies were categorized as high risk when it was clear that the conditions were not met, such as when it was implausible for the participants and/or experimenters to have been blind to the administration of the experimental vs. control conditions. We also extended these criteria to include additional assessments pertinent to TMS studies, namely the targeting procedure and type of control condition. For targeting procedure, if a study used structural or functional MRI neuronavigation, international 10–20 EEG positioning system, or distance measurements, they were classified as low risk, unknown risk, and high risk, respectively. A variety of control conditions were implemented, such as active, sham, and no TMS conditions, that were classified, as low, unknown, and high risk, respectively.

Coding of the Studies

Moderators that were hypothesized to modulate the size of effects of rTMS on episodic memory were coded for each study. In order to code for moderators, the full-text articles were examined for explicitly stated information regarding the moderators of interest. In situations where it was not clearly stated, the moderators were inferred from the study protocol (e.g., the study did not state if stimulation occurred “online” or “offline,” but, if the task figures showed that stimulation occurred prior to a cognitive task, then we coded stimulation timing as offline). Studies were independently coded by both the first author and research assistants to establish reliable coding and any discrepancies were resolved by the authors.

Coding of rTMS Parameter Moderators

Frequency was coded as a categorical variable with 1, 5, 10, and 20 Hz being coded as “one,” “five,” “ten,” and “twenty,” respectively. Theta burst frequencies were separated and coded as intermittent theta burst (iTBS) or continuous theta burst (cTBS). One study used a modified iTBS protocol with short iTBS (2 s) trains interleaved with stimulus presentation and was classified as online iTBS. None of the obtained studies implemented an intermediate theta burst (imTBS) procedure. A remaining 15 effect sizes across 3 studies implemented stimulation frequencies (i.e., 6.8, 7, 10.7, 17.5, and 18.7 Hz) that did not fall in these categories and, thus, were not used in analyses examining the effects of frequency.

Motor threshold intensity (MTI) can be calculated numerous ways, such as based upon individual RMT or AMT. Although procedures that estimate motor thresholds using AMT vs. RMT or MEP vs. visually-evoked responses result in slightly lower levels of stimulator intensity (Rossini et al., 2015), there were not enough studies that clearly implemented either RMT or AMT with either MEP or visually-determined thresholds to allow for systematic investigation of the role of different thresholding procedures. Therefore, MTI was treated as a categorical variable. Studies that reported using stimulation parameters at, above, or below 100% of motor threshold were coded as “at,” “above,” or “below,” respectively.

Studies were coded as “online” protocols when stimulation occurred simultaneously with the study's experimental task, such as when rTMS co-occurred with stimulus (e.g., words or pictures) presentation during either encoding or retrieval. Although studies differed with regards to the precise timing of the delivery of TMS pulses, for example, at stimulus onset or varying intervals from stimulus onset, there were not enough studies to examine possible differences in the effects of timing differences in the administration of online rTMS protocols at this time. Therefore, we collapsed across different online timing protocols. Studies where stimulation occurred in isolation without any co-occurring task, such as before or after an encoding or retrieval task, were coded as “offline” protocols.

The hemisphere that was stimulated was coded as “left” or “right.” Studies that targeted cortical regions in frontal cortex or parietal cortex were coded as “FC” and “PC,” respectively. There were too few studies that targeted regions beyond the FC and PC (e.g., occipital cortex) for valid analysis and inference of aggregated effect sizes at this time. Therefore, a meta-analysis investigating the effects of stimulation that targeted either consolidation processes or regions outside the PC and FC was not possible at this time. Again, these gaps in the literature highlight areas where more research is needed.

Lastly, we coded studies based on the type of control condition used for comparison to the primary “treatment” condition of interest in order to assess the potential differences between different types of control conditions. Classification of an “active” control occurred when experimental stimulation was compared to any active rTMS control site except for the vertex. A “vertex” control comparison was coded separately. When studies implemented either a sham coil or applied sham stimulation by tilting the coil away from the participant's head, they were coded as “sham” controls. In situations where an experimental stimulation group was compared to a condition in which no rTMS was applied, they were coded as “no TMS.”

Coding of Experimental Design Moderators

The “stimulated memory processes” (encoding or retrieval) were coded as the main memory processes that were most likely to be primarily affected by stimulation based on when stimulation was applied. A study was coded as targeting “encoding” if the stimulation parameters occurred within seconds to minutes prior to, during, or immediately after (e.g., within 400 ms of stimulus offset) encoding each item. For example, conditions with a train of cTBS administered within minutes prior to encoding all items was coded as primarily targeting “encoding.” Similarly, conditions with brief trains of 1 Hz rTMS during picture presentation were coded as targeting “encoding.” Situations in which stimulation occurred shortly after picture presentation offset (i.e., within milliseconds), were coded as targeting encoding. While such protocols may affect both encoding and early consolidation processes, studies were categorized using this range in order to maximize the number of effect sizes for reliable analysis. The critical distinction for this analysis was to compare the aggregate effect size of rTMS protocols that clearly impacted retrieval processes vs. encoding/early-consolidation processes. Unfortunately, there was only one study that applied stimulation during consolidation, which prevented its inclusion in the meta-analysis. If stimulation occurred immediately before or during a retrieval task (e.g., cTBS prior to attempting to retrieve all previously encoded words or 1 Hz rTMS during presentation of each picture in a recognition test), the study was coded as primarily targeting “retrieval.”

To test for a difference in the effects of rTMS on episodic retrieval processes that are hypothesized to be distinct by dual-process theories, effect sizes were extracted for conditions hypothesized to primarily measure either recollection or familiarity. Conditions were aggregated and coded as assessing “recollection” if they assessed free recall, cued recall, “remember” responses in remember/know recognition memory paradigms, and paradigms that reported associative- or source-recognition memory performance. Associative- or source-recognition estimates (i.e., associative- or source-hits, corrected recognition scores, or d′) were extracted from recognition memory tasks that required the retrieval of qualitative information about the study of a specific piece of information (e.g., if participants studied a soccer ball object and were asked to recall its spatial location), as in source memory paradigms. Conversely, conditions were aggregated and coded as assessing “familiarity” if they assessed and reported recognition memory performance (e.g., item memory hit rates, corrected scores, d′) on tests that did not require additional qualitative information to be retrieved. This coding scheme included “know” responses for remember/know paradigms.

Data Extraction

Means (M), standard deviations (SD), and sample sizes (n) were obtained for relevant episodic memory outcome measures for each study. When standard errors (SE) were reported, they were converted to SD. In cases where there was insufficient statistical information provided in the text or tables, they were collected from graphical representations using GetData Graph Digitizer data extraction software (downloaded at http://www.getdata-graph-digitizer.com). In the event that insufficient data could be obtained, attempts were made to contact the corresponding authors for additional data. Descriptive statistics and effect size calculations were done independently by the first author and research assistants to cross-check the calculations and resolve any discrepancies. The table of experiments included in the meta-analysis, their moderator variables, and all of the effect sizes that were extracted from the experiments are presented in an open-access database here: https://osf.io/6kbqe/?view_only=226e600d6e1546748b52f6bbfc593140. For additional information or to contribute additional data, please contact the first author. Our intention for sharing the data is so that effect size calculations can be confirmed and so that additional studies can be continuously added as the literature grows, which will facilitate future meta-analysis and systematic investigation.

Effect sizes for between-subject comparisons were computed with the following formula:

$\begin{array}{l}d=\frac{\left(M_{Experimental}-M_{Control}\right)}{S{D}_{Pooled}}\end{array}$

When studies reported comparisons between gain scores (e.g., Post-experimental—Pre-experimental) the effect size was calculated with the following formula:

$\begin{array}{l}d=\frac{\left(\left(M_{Experimental,Post}-M_{Experimental,Pre}\right)-\left(M_{Control,Post}-M_{Control,Pre}\right)\right)}{S{D}_{Pooled,Post}}\end{array}$

Effect sizes for within-subject comparisons were calculated with the following formula:

$\begin{array}{l}d=\frac{\left(M_{Experimental}-M_{Control}\right)}{\left(\frac{\left(S{D}_{Experimental}+S{D}_{Control}\right)}{2}\right)}\end{array}$

This was done to obtain an accurate effect size estimate for within-subject designs (Cumming, 2012) using the commonly recommended approach for meta-analyses (Lakens, 2013). Alternative routes that address dependencies in within-subject designs, such as averaging effect sizes, can lead to a reduction in statistical power and the loss of information can result in less precise estimates and SE's (Hedges and Pigott, 2001; Cheung, 2014). In a similar vein, limitations also arise when calculating Cohen's d_rm, which attempts to account for the dependencies by correcting for the correlation in within-subject designs. As correlations between conditions are rarely reported, estimations may be too conservative, especially when correlations are high (Lakens, 2013).

Hedge's g and g_av effect sizes were calculated, which provides a less biased estimate of the true effect than Cohen's d, especially for studies with small sample sizes (Hedges and Olkin, 1985). In order to do this, we first calculated Cohen's d and then applied a Hedge's g correction (see Cumming, 2012) with the following formula:

$\begin{array}{l}g=d\times \left(1-\frac{3}{\left(4\left(n_{Experimental}+n_{Control}\right)-9\right)}\right)\end{array}$

With the exception of 5 studies, we were able to collect multiple effect sizes per study. Thus, we can no longer assume statistical independence, as within-study effect sizes may be more similar than effect sizes from other studies. Thus, dependent effect sizes may contain redundant information that becomes generally less informative when effect sizes are correlated. This form of dependency may occur when multiple effect sizes can be calculated from multiple outcome measures (Van den Noortgate et al., 2013). For example, in examining the effects of rTMS on episodic memory, a study can test a set of stimulation parameters (e.g., 10 Hz rTMS at 100% AMT to the DLPFC) on multiple memory outcomes, such as recall, recognition and source memory. A second dependency occurs when effect sizes are correlated in situations where an experimental condition is compared to multiple control groups or multiple experimental conditions are compared to one control group. For instance, many of the studies compared experimental stimulation to combinations of active control, sham, or no stimulation conditions, while other effect sizes were comparisons between multiple experimental stimulation conditions to a single control condition. This can be problematic in meta-analyses and increase type 1 errors if dependent effect sizes are assumed to contain unique information. Therefore, to address multiple dependency concerns we conducted a three-level random effects meta-analysis with Hedge's g and g_av corrections, which is detailed below (for additional information on dependencies and three-level models see, Van den Noortgate et al., 2013).

Three Level Modeling

We conducted a three-level random effects model using the Metafor package in R (Viechtbauer, 2010). The typical random effects approach allows for effect sizes to vary due to sampling variation and due to differences between studies. In order to account for studies that contain multiple effect sizes, we can extend this model to a three-level random effects model to take into account the correlation of within-studies effect sizes. This approach does not assume independence among effect sizes as is done with other meta-analysis techniques. Multilevel modeling allows for an alternative for dealing with the dependency of effect sizes when correlations between outcomes are not reported or there is insufficient data to estimate the dependency (for additional details, see Assink and Wibbelink, 2016). The hierarchical structure of the data allows effect sizes to vary between participants (level 1; sampling variance), outcomes (level 2; within studies), and studies (level 3; between studies). In this case, individual studies (denoted by the subscript k) can contain one or multiple outcomes (denoted by the subscript j). The studies (i.e., Study ID) and individual effect sizes (i.e., Effect Size ID) were entered into the model as random effects. The simplest model with no moderators and three residuals (v_k, u_jk, e_jk) that are assumed to be normally distributed with a mean of zero is:

$\begin{array}{l}{\text{ES}}_{\text{jk}}={\beta }_0+{\text{v}}_{\text{k}}+{\text{u}}_{\text{jk}}+{\text{e}}_{\text{jk}}\end{array}$

where an observed effect size for outcome j within study k is represented by ES_jk. The overall population effect size mean across all outcomes and studies is represented by β_0. The v_k element is the random mean deviation effect in study k from the overall effect. The random deviation of jth population effect in study k from the mean effect in study k is denoted by u_jk. Lastly, e_jk is the random residual error from the population effect due to sampling variation. Parameter estimates in the meta-analysis include the between-study variance (${\sigma }_{\text{v}}^2$) and within-study variance (${\sigma }_{\text{u}}^2$), and the sampling variance is not reported because it is calculated from reported data prior to the analysis. Therefore, observed significant between-study variance differences reflect that effect sizes systematically vary between studies to a greater degree than would be expected due to random differences in sampling variance (due to chance). Meanwhile significant within-study variance above zero signifies that the observed effects within a study vary across individuals to a greater degree than chance.

In order to implement a parsimonious model, we examined if our three-level model provided a better model fit over more simplistic models with multiple ANOVAs. There was a significant difference between a single and two-level model (within-study variance), likelihood ratio test (LRT) = 41.75, p < 0.01. There was also a significant difference between two and three-level models (between-study variance), LRT = 41.83, p < 0.01. Thus, the three-level modeling of the dependent effect sizes provided a better model fit than treating the effect sizes as independent.

Publication Bias

A prevalent issue in systematic reviews and meta-analyses is how to address missing data due to publication bias. This problem occurs when significant results are more likely to be published than non-significant findings. If left unaccounted for, publication bias may lead to spurious results. In this instance, publication bias may lead to the true effect of rTMS on episodic memory being inflated. We used two methods to assess the degree of possible publication bias in the data: funnel plots and an extension of Egger's regression method (Egger et al., 1997)

Funnel Plots

One approach to identify and estimate missing data is to examine funnels plots for asymmetry (Sterne et al., 2011). In order to do this, a scatter plot is created with the effect size on the x-axis and the sampling variance (or some variation taking into account the sample size) on the y-axis. Publication bias is less likely to be a concern when effect sizes are distributed symmetrically around the mean effect and are clustered around the mean effect size. Conversely, support for possible publication bias manifests visually in these plots with an asymmetrical distribution of effect sizes around the mean effect size, often with larger sampling variances producing larger effect sizes.

Contour-enhanced plots offer an extension of assessing publication bias by drawing a reference line at zero with contour color changes at different significance levels (0.01 and 0.05) as the effect sizes are plotted against precision (1/SE, inverse of the standard error). In this case, publication bias can be examined with missing effect sizes observed in non-significant regions of the plot. However, there are other possible causes of asymmetry besides publication bias. Asymmetry due to heterogeneity of potential moderators is another factor that can influence asymmetry and can be assessed with residual plots. Here, residuals are plotted against their standard errors at different levels of the moderators (e.g., online vs. offline protocols). Similar to the contour-enhanced plots, significant levels can be added, and asymmetry of the moderators can be examined in relation to the predicted effect sizes. In an attempt to address both publication bias and other potential forms of publication bias we examined contour enhanced and residuals plots.

Extension of Egger's Test

Additional traditional approaches for formally testing for publication bias, such as trim and fill and Egger's test, are complicated with three-level modeling. Although correcting (e.g., trim and fill) for publication bias is important (Copas, 1999), obtaining more precise estimates through correction methods may only occur if the bias is large (Hedges and Vevea, 1996). Therefore, we decided to implement an extension of Egger's test for more complex models by entering the standard error into the model as a moderator. Essentially, this provides an extension of Egger's regression method which involves estimating funnel plot asymmetry by measuring how much the regression line (effect size vs. precision, 1/SE) deviates from zero. However, future research is needed to determine how to properly assess missing data in three level meta-analyses, as the numerous available methods to measure and correct for publication bias each come with limitations and, to our knowledge, have not been evaluated in the three-level meta-analysis approach.

Results

The systematic review incorporated 1,532 participants across 59 articles, see Figure 1. This resulted in an average age of 33.12 (18–80) with 61% females and 39% males in the studies. Of the 59 articles, 44 contained rTMS procedures focusing on younger adults, while 12 articles included healthy older adults and those with memory impairments (i.e., aMCI, Alzheimer's disease) or other disorders (i.e., depression, fibromyalgia, alcohol dependency). The small number of studies focusing on older adults made a meta-analysis unfeasible for comparison with younger adults. Therefore, our approach was 2-fold. First, we conducted a three-level meta-analysis focusing on the effects of rTMS on episodic memory in younger adults. Second, we synthesized the results of studies using rTMS to modulate episodic memory in older adults in the form of a narrative review.

A total of 1,016 participants were included in the meta-analysis of the 37 articles that were identified. The average age of participants across the studies was 25.70 years old with 39% male and 61% female. The average sample size was 24.19 (range 10–69). For a breakdown of the study designs and protocols, see Table 1.

TABLE 1

Table 1. Parameters and results of experiments that applied TMS and assessed episodic memory in healthy younger adults, healthy older adults, and older adults with memory impairment.

Quality Assessment

The proportion of studies classified as low, medium, and high risk for the different categories of the quality assessment analysis are presented in Figure 2. Overall, many of the studies implemented appropriate methodological protocols for random allocation of participants to conditions, blinding of the experimenters assessing effects on the outcome measure(s), and non-selective reporting of outcome measures. However, several studies either did not implement proper blinding of participants and/or experimenters, or did not provide enough information to confirm proper blinding procedures. There was low concern for the degree of participant attrition that was reported. For our additional quality assessment measure of control type, we found that the majority of studies implemented either active or sham controls. With regards to the rTMS targeting procedure, we found that the majority of studies implemented either low risk neuronavigational procedures or used the 10–20 system for targeting cortical regions of interest.

FIGURE 2

Figure 2. Quality assessment findings for the 59 articles included in the systematic review. Blinding of outcome assessment reflects blinding of the experimenter in coding memory outcome measures (e.g., recall tasks).

Evaluation of Publication Bias

Several approaches were used to examine possible publication bias. First, we examined the enhanced-contour plot (i.e., effect size vs. 1/standard error). The reference line is set to g = −0.06 with the contour shading representing p-value significance. The enhanced-contour plot revealed some asymmetry, but no systematic patterns of missing non-significant effect sizes, see Figure 3. Given the numerous effect sizes collected, the larger effect sizes observed in the funnel plots are expected. Importantly, they appear to be symmetrically distributed. An examination of influential points revealed no Cook's d values above 0.5.

FIGURE 3

Figure 3. Contour-Enhanced funnel plot. Reference line is set to g = −0.06. Contour lines represent p-value significance (i.e., <0.05, < 0.01) with the white area representing p-values > 0.05.

Next, we examined the residual plots (residuals vs. standard error) for each moderator variable, see Figure 4. The reference line is set to average effect size with contour shading once again representing p-value significance. The residual plots revealed some asymmetry across the moderator variables.

FIGURE 4

Figure 4. Residual funnel plots for moderator variables. The residuals are on the x-axis with the standard error on the y-axis. Contour lines represent p-value significant (i.e., <0.05, < 0.01) with the white area representing p-values > 0.05. Reference line set to g = −0.06.

Interpreting funnel plots is inherently difficult (Terrin et al., 2005) and may be further complicated in three-level models (e.g., observed asymmetry may not be problematic). For example, clustered data points due to the three-level structure could potentially be misinterpreted as bias. Therefore, in a more formal test of asymmetry we entered standard error into the model. This revealed no significant moderating effect of standard error on effect size, F_{(1, 244)} = 1.76, p = 0.18, indicating that the funnel plot was not significantly asymmetrical. Collectively, this suggests that publication bias is unlikely to be problematic in this analysis. However, future research is needed to properly address how to handle detecting and correcting for missing data (e.g., publication bias) in three-level models.

Overall Relationship Between rTMS and Episodic Memory

We initially entered individual effect sizes (i.e., Effect Size ID) and studies (i.e., Study ID) as random effects. This model revealed the average effect size (g = −0.06, SE = 0.08) was not significantly different from zero, t₍₂₄₄₎ = −0.75, p = 0.45, see Figure 5. Thus, we do not see an overall effect of rTMS on episodic memory performance, which is unsurprising given that most studies selected and implemented protocol parameters to either enhance or impair episodic memory functioning.

FIGURE 5

Figure 5. Overview of effect sizes included in meta-analysis. Reference line set to the average effect size (g = −0.06). Legend represents study sample size ranges: 10–19, 20–29, 30–39, 40–49, 50–59, 60–69.

Heterogeneity of Effect Sizes

We tested the heterogeneity of effect sizes to examine if differences between effect sizes were systematic and not due to random sampling variance. If a substantial amount of between- or within-study effect size variance is found, potential moderators can be added into the model to explain this variability. There was significant within-study variance (level 2), estimate = 0.19, p < 0.01 and between-study variance (level 3), estimate = 0.12, p < 0.01, demonstrating that effect sizes varied both within and between studies. Consequently, 44.58% of the variance is accounted for by differences in effect sizes within studies, 27.25% by variance between studies, and 28.07% by random sampling variance. The significant variability across both levels highlights the substantial differences in effect sizes between studies and within the same study. Therefore, we next conducted a moderation analysis in order to assess whether the observed differences in effect sizes were moderated by other factors.

rTMS Parameter Moderator Analysis

A moderation test revealed a significant moderating effect of rTMS frequency on episodic memory effects, F_{(6, 224)} = 2.40, p = 0.03, see Table 2. Specifically, 1 Hz (β = 0.59, SE = 0.20) rTMS protocols led to significantly larger enhancing effects compared to 10 Hz, (β = −0.28, SE = 0.20), 20 Hz (β = −0.22, SE = 0.14), iTBS (β = −0.42, SE = 0.36), and cTBS (β = −0.05, SE = 0.15). The comparisons between other frequencies were not significant. The aggregate effects of stimulation intensity or timing, or the targeted hemisphere, cortical region, memory process, or retrieval type, or the type of control condition were not significant moderators of rTMS effect sizes, see Table 2.

TABLE 2

Table 2. Main effects of moderator variables (F-values) and beta and t values of each level in the meta-analysis.

Interactions

Effect of Frequency and Motor Threshold Intensity

Although we did not find support for MTI moderating episodic memory performance overall, we anticipated that MTI may interact with frequency. Recent findings have shown that varying MTI's during iTBS follow an inverted U-shaped pattern in modulating working memory performance (Chung et al., 2018). An investigation of MTI (above, at, or below) and frequency (1, 5, 10, 20, cTBS, iTBS) revealed an interaction, F_{(14, 202)} = 1.78, p = 0.05, see Tables 3, 4. In order to examine what was driving this interaction we considered how differences between MTI (above vs. at vs. below) at each frequency affected episodic memory. However, the number of comparisons were restricted due to the limited number of studies that included frequencies at different MTIs. Follow-up tests revealed that the advantage of 1 Hz rTMS compared to 10, 20, iTBS, and cTBS protocols in producing significantly stronger enhancing effects of experimental/treatment rTMS vs. control/sham rTMS on memory performance was specific to intensities below MTI. There was also a significant effect of 20 Hz rTMS “below” MTI (β = −0.38, SE = 0.17) in that it produced a larger negative effect of experimental/treatment rTMS vs. control/sham rTMS compared to 20 Hz “at” MTI (β = 0.40, SE = 0.35), F_{(1, 202)} = 3.98, p = 0.04. There were no other differences between levels of MTI for any other frequency, see Tables 3–5¹.

TABLE 3

Table 3. Descriptive statistics for motor threshold intensity by frequency interaction.

TABLE 4

Table 4. Comparisons for frequency by motor threshold intensity interaction.

TABLE 5

Table 5. Additional comparisons for frequency by motor threshold intensity interaction.

Effect of Frequency and Stimulation Timing

There was a significant interaction of stimulation timing (online vs. offline) and frequency (1, 5, 10, 20, iTBS, cTBS) on episodic memory effects, F_{(9, 221)} = 4.16, p < 0.01. We first examined differences between frequencies within online and offline stimulation protocols. Subsequently, we looked at differential outcomes between online vs. offline protocols for specific frequencies. However, the number of comparisons we could make was limited due to the small number of studies that included online or offline protocols at each frequency.

Follow-up tests revealed that during online stimulation protocols, 1 Hz rTMS (β = 0.60, SE = 0.45) compared to 20 Hz rTMS (β = −0.32, SE = 0.12) led to larger enhancing effects of experimental/treatment rTMS vs. control/sham rTMS on memory performance, see Tables 6, 7. No other interactions with online stimulation were significant.

TABLE 6

Table 6. Descriptive statistics for stimulation timing by frequency interaction.

TABLE 7

Table 7. Stimulation timing moderator comparisons.

Following offline stimulation protocols, 1 Hz rTMS (β = 0.58, SE = 0.21) led to significantly larger enhancing effects compared to both iTBS (β = −1.84, SE = 0.53) and cTBS (β = −0.06, SE = 0.13). In addition, offline 20 Hz rTMS (β = 0.94, SE = 0.45) led to significantly larger enhancing effects compared to both iTBS and cTBS. Interestingly, offline iTBS led to significantly larger negative effects compared to cTBS. No other comparisons were significant, although a full decomposition of the interaction was not possible because the number of studies with offline stimulation for each type of frequency were limited, see Table 7.

Additional follow-up tests revealed that stimulation timing significantly influenced the directionality of the effects of 20 Hz rTMS. Specifically, online 20 Hz rTMS led to larger negative effects, while offline 20 Hz rTMS led to larger enhancing effects of experimental/treatment rTMS vs. control/sham rTMS on memory performance, F_{(1, 221)} = 7.33, p < 0.01. There were no other significant differences.

Memory Processes and Hemispheric Stimulation

Findings of individual studies have reported that there may be different effects of rTMS when stimulating the left or right hemisphere during either encoding or retrieval. To assess potential aggregate effects across studies, we examined the interaction between stimulated hemisphere (left vs. right hemisphere) and memory process stimulated (encoding vs. retrieval processes). However, there was no significant interaction between the effect sizes of stimulating different memory processes and hemisphere, F_{(4, 218)} = 1.03, p = 0.39.

Retrieval Type and Cortical Regions

Although it is plausible that the effects of stimulating either frontal or parietal cortex may differ for familiarity- vs. recollection-based retrieval processes, we found no significant interaction between effect sizes of the stimulated cortical region (FC vs. PC) and retrieval type (familiarity vs. recollection), F_{(4, 233)} = 0.45, p = 0.77, although, again, inferences are currently limited by the relatively small numbers of studies that could be included in such a moderation-based meta- analysis.

Discussion

A systematic review of the literature spanning the past 30 years yielded 59 articles that focused on how TMS can modulate episodic memory in younger and older adults. To better understand the circumstances of when rTMS will enhance or impair memory, we conducted a three-level random effects meta-analysis of 245 effect sizes aggregated from 37 articles focused on young adults. Because the effects of some studies used parameters designed to impair memory to probe the necessity of a stimulated brain region in a particular process relevant for episodic memory, whereas others implemented designs to facilitate episodic memory, the mean effect size, collapsing across all parameters is relatively uninformative.

Enhancing Effects of 1 Hz rTMS

Moderation analyses revealed that stimulation frequency modulated the effect of rTMS on episodic memory in younger adults, with larger facilitatory effects of 1 Hz (g = 0.59) rTMS compared to 10 (g = −0.28), 20 (g = −0.22), iTBS (g = −0.42), and cTBS (g = −0.05) protocols. The facilitatory effect of 1 Hz rTMS may appear counter-intuitive because low frequency (e.g., <1 Hz) rTMS is typically thought to lead to inhibitory effects on neural processing of targeted regions, whereas, many high frequency protocols are generally thought to lead to enhancing cortical excitability (Walsh and Cowey, 2000; Rossi and Rossini, 2004). Relatedly, the numerically larger negative effects of iTBS compared to cTBS paradigms is also surprising given that cTBS is typically assumed to have long-term-depression-like inhibitory effects (LTD) that can suppress cortical excitability in targeted regions whereas iTBS is assumed to lead to long-term potentiation-like effects (LTP) that can enhance cortical excitability (Huang et al., 2005). However, these findings stem from physiological studies that modulated cortical activity in the motor cortex (Jelić et al., 2015), while the studies in the present meta-analysis generally targeted prefrontal and parietal regions. One possible explanation is that the effects of TMS may differ according to the structural and functional architecture of the targeted region (e.g., motor cortex vs. prefrontal/parietal cortex). However, strong inferences based on these findings are currently limited due to the numerous differences between studies in the specific rTMS parameters and experimental designs that were used, which may interact to affect episodic memory.

Interactions Between rTMS Frequency and Stimulation Intensity

Interaction analyses revealed facilitatory effects of 1 Hz rTMS for both online and offline stimulation. However, the effects of 1 Hz rTMS was driven by stimulating below MTI. While the underlying mechanisms of 1 Hz rTMS on episodic memory are poorly understood, one possible explanation comes from the principle of stochastic resonance, whereby neural communication is enhanced by low levels of noise that can push a neural signal past threshold (Schwarzkopf et al., 2011; Silvanto and Cattaneo, 2017). Thus, 1 Hz rTMS may enhance episodic memory by interacting with intensity to enhance the signal-to-noise ratio in the stimulated network of neurons. However, this process is affected by a variety of other factors that further complicate the process. For example, the effects of MTI are known to be sensitive to “brain states” and changes in brain activity can influence if below- or above-MTI leads to facilitatory or inhibitory effects (Romei et al., 2016).

Stimulation intensity also interacted with 20 Hz rTMS with attenuating effects of below-MTI compared to at-MTI. These findings are not consistent with stochastic resonance, where below MTI stimulation should lead to enhancing effects. However, the situations in which different levels of intensity will lead to either enhancing or inhibitory effects have also been known to depend on brain states. One reason for the observed effects may be due to differences in experimental designs, as task stimuli or cognitive processes engaged likely resulted in different activation states in the targeted brain regions. Relatedly, previous work has shown that cortical excitability varies with different MTI levels following iTBS (Chung et al., 2018) protocols. Thus, it is possible that the enhancing or inhibitory effects of rTMS may vary with MTI, which may also differ depending on the frequency of stimulation.

Interaction Between rTMS Frequency and Stimulation Timing

Additional analyses revealed that the effects of episodic memory performance in young adults depended on frequency and stimulation timing. Specifically, the enhancing effects of 1 Hz compared to 20 Hz was driven by online protocols. On the other hand, both 1 Hz and 20 Hz rTMS protocols had enhancing effects compared to either iTBS or cTBS paradigms when stimulation occurred offline. Interestingly, online 20 Hz rTMS protocols had inhibitory effects on episodic memory; however, offline 20 Hz rTMS had enhancing effects on episodic memory. These findings suggest that 20 Hz rTMS modulates episodic memory through different mechanisms when applied online vs. offline. For example, online paradigms may disrupt ongoing neural activity while offline paradigms may potentiate episodic memory processes through LTP-like effects.

Taken together, while the meta-analysis suggests novel perspectives on the effects of interacting factors on rTMS studies of episodic memory, the limited numbers of studies with combinations of factors spanning the entire parameter space may have hindered our ability to reliably estimate the role of many factors (e.g., targeted hemisphere, cortical region, memory process, and retrieval type) that are hypothesized to moderate the effects of rTMS on episodic memory. Moreover, the current meta-analysis did not find a moderating effect of the type of control condition (i.e., active, active vertex, sham, or no TMS) that was implemented. Importantly, these findings do not suggest that there are no differences between using active, vertex, sham, or no TMS control conditions. There are substantial gains in the quality of inference that can be obtained with comparison to proper control conditions. Inferences about the role of a particular brain region in a specific process are strengthened when effects are contrasted against conditions in which active stimulation at the same level of intensity and frequency is applied to either the targeted brain region at a different time point or to a different brain region that is not hypothesized to be involved in the process of interest at the time of stimulation. Therefore, researchers should still take careful consideration in the type of control condition that is most appropriate and feasible for specific experimental designs (for further information on control conditions see, Davis et al., 2013; Duecker and Sack, 2015).

In the next section we discuss how the meta-analysis on younger adults relates to the pattern of observed effects of rTMS effects on episodic memory functioning in healthy older adults and those with clinical disorders. This systematic review of the effects of rTMS on older adults revealed several similar findings that were observed in the meta-analysis on younger adults, thereby providing some sense of cross-validation of the findings in different populations.

Comparisons to rTMS Effects on Older Adults and Clinical Populations

Enhancing Effects of 1 Hz rTMS When Applied Below MTI

A similar pattern of effects of 1 Hz rTMS on episodic memory that was revealed in the meta-analysis on young adults was found in 1 Hz rTMS studies on older adults with below-MTI stimulation enhancing performance (Turriziani et al., 2012) while above-MTI had no effect (Davis et al., 2017). Turriziani et al. (2012) administered offline 1 Hz rTMS at 90% MT to the left or right DLPFC of patients with mild cognitive impairment following encoding of faces, buildings, and words and found enhanced recognition memory. In contrast, Davis et al. (2017) administered offline 1 or 5 Hz rTMS at 120% MT to the left DLPFC of healthy older adults prior to an encoding task and observed null effects on associative memory (although there were differential effects on brain network activity related to successful memory performance, for details see Davis et al., 2017). Although several parameters differed between these studies, the difference in outcomes of 1 Hz rTMS as a function of MTI is consistent with the findings of studies included in the meta-analysis irrespective of differences among those studies. However, there are not enough studies available to compare all combinations of parameters of offline 1 Hz rTMS below-, at-, or above-MTI stimulation.

Impairments From Online 20 Hz rTMS Below MTI

Twenty Hz rTMS findings in healthy older adults found a similar pattern that was revealed in the meta-analysis with impairments to episodic memory when 20 Hz rTMS was administered online or below MTI. Rossi et al. (2004) reported attenuating effects of online 20 Hz rTMS at 90% MT to the left DLPFC during picture retrieval. In addition, Manenti et al. (2011) found attenuating effects on episodic memory when online 20 Hz rTMS at 90% RMT was applied to left vs. right DLPFC during either encoding or retrieval. However, this effect was largest in low performing adults when stimulation was applied to left compared to right DLPFC during encoding. Thus, it is possible that the effects of 20 Hz rTMS on episodic memory may rely on more complex interactions that were not possible to examine in the current meta-analysis due to the limited number of studies that have investigated the combinations of parameters.

Enhancing Effects of Offline 20 Hz rTMS

Two studies on older adults with memory disorders found a similar pattern that was revealed in the meta-analysis with enhancing effects of multi-session administration of offline 20 Hz rTMS (Wang et al., 2014; Nilakantan et al., 2017). In prodromal Alzheimer's patients, Koch et al. (2018) reported that 10 sessions (5 sessions/week) with 2-week separation between cross over stimulation (active vs. sham) of offline 20 Hz rTMS at 100% RMT to the precuneus enhanced delayed recall (Koch et al., 2018). Similarly, in an individual with mild cognitive impairment, Cotelli et al. (2012) found enhanced associative memory compared to healthy controls following 10 sessions (5 sessions/week) of offline 20 Hz rTMS at 100% MT to the left parietal cortex (Cotelli et al., 2012).

Importantly, the enhancing effect of offline 20 Hz contrasts with an overall impairment of online 20 Hz rTMS on episodic memory (see Table 2). However, offline 20 Hz studies implemented multi-session protocols (e.g., 5 sessions/week for 2 weeks) while all nine online 20 Hz studies implemented single session protocols (see Table 1), so this confounds interpretation of the potential difference in the direction of effects of 20 Hz rTMS when applied offline vs. online. This also reveals a gap in the literature as few studies have implemented multi-session protocols with younger adults. Collectively, these findings are encouraging with regards to the potential for applying multi-session offline 20 Hz rTMS protocols to enhance episodic memory in older adults, even those with MCI or AD.

Enhancing Effects of Offline 5 Hz rTMS Below MTI

Three studies implemented 5 Hz rTMS and found enhanced behavioral outcomes (Solé-Padullés et al., 2006; Peña-Gomez et al., 2012) or modulation of neural activity (Davis et al., 2017). In low performing older adults, Solé-Padullés et al. (2006) reported enhancing effects when offline 5 Hz rTMS at 80% MT was applied prior to encoding face-name associations. In addition, in carriers of a major genetic risk factor for Alzheimer's disease (Apolipoprotein ε4 allele; APOE), enhancing effects were reported following offline 5 Hz rTMS at 80% MT prior to a face-name encoding task (Peña-Gomez et al., 2012). In healthy older adults, Davis et al. (2017) reported modulated neural activity, but no differences in memory performance following offline 5 Hz rTMS at 120% MT prior to an associative encoding task. There are no studies on young adults that administered offline 5 Hz rTMS. This prevents making comparisons and highlights the need for future work to address this gap.

Enhancing Effects of Offline 10 Hz rTMS

Five studies applied offline 10 Hz rTMS to older adults or clinical populations that included a wide age range and all but two found enhancing effects on episodic memory (see Table 1). The offline 10 Hz studies that found null effects used a different stimulation intensity than the others (Rektorova et al., 2005) or applied rTMS to motor cortex in fibromyalgia patients (Baudic et al., 2013). The studies that found enhancing effects of offline 10 Hz used stimulation intensities below- or above-MTI (Drumond Marra et al., 2015; Qiao et al., 2016; Kavanaugh et al., 2018). These findings diverged from the results of the meta-analysis that revealed a trend for impairments of online 10 Hz rTMS in younger adults. Differences in the stimulation timing between studies is one potential reason for the different effects; however, direct comparison is complicated because no studies to date have applied offline 10 Hz in young adults.

Limitations of Conducting a Meta-Analysis on Available rTMS research

Although the systematic review and meta-analysis revealed some promising findings, a considerable amount of research is needed to more fully understand the roles of both specific aspects of the rTMS parameters (frequency, intensity, timing, hemisphere, cortical region, and control conditions) and design considerations (targeted memory process and retrieval type) on the effects on episodic memory functioning, as well as how these numerous factors may interact. Because there are not published studies for every pairwise combination of the rTMS parameters and design aspects of interest, there were many missing cells in the meta-analysis. Relatedly, because of missing data cells and small numbers of observed effects sizes for each possible comparison, we weren't able to investigate higher-order interactions.

Effects on Objective vs. Subjective Aspects of Episodic Memory

It should also be noted that the current meta-analysis focused on accuracy as the outcome measure of episodic memory performance. We did not address the potential moderating effects of rTMS and study design parameters on other measures of episodic memory performance such as reaction times and confidence ratings. These are important issues to investigate in future research as the body of literature accumulates.

For example, while the meta-analysis on accuracy measures failed to reveal a main effect or interaction with the cortical region that was targeted or the memory process that was stimulated, based on the systematic review of the published studies, it seemed to be the case that several studies found that applying rTMS to regions of parietal cortex affected the subjective experience of remembering as evidenced by modulation of confidence ratings even in the absence of effects on objective measures of accuracy. For example, Ye et al. (2018) recorded participants playing an action-adventure video game and then applied offline 1 Hz rTMS at 110% AMT to the precuneus prior to snapshot images from the video game and then tested subjects' temporal order memory and confidence ratings. The authors found decreases in confidence ratings during retrieval following precuneus compared to vertex stimulation, despite no differences in memory performance (Ye et al., 2018).

In a similar fashion, Bonnì et al. (2015) applied offline cTBS at 100% MT to the precuneus, posterior parietal cortex, or vertex prior to associative (color-context) memory tests of studied object images (e.g., apple outlined in red). While there were no differences in hit rates among the conditions, there was a significant decrease in the number of source memory errors (e.g., correct judgment that the apple was outlined in red not green) that were made following precuneus stimulation (Bonnì et al., 2015). Relatedly, offline 20 Hz rTMS at 100% MT to the left angular gyrus prior to retrieval of previously encoded pictures led to decreased recognition relative to superior parietal lobe stimulation, and a decrease in recollective details (by altering response bias) compared to either superior parietal or vertex stimulation (Sestieri et al., 2013). Moreover, offline cTBS at 70% RMT to the left angular gyrus prior to encoding auditory stimuli reduced subjective confidence ratings for source memory while leaving free- and cued- recall of word pairs unaffected (Yazar et al., 2014). Taken together, it seems that rTMS to parietal regions tends to affect the subjective experience of remembering even when there are null effects on episodic memory accuracy. However, too few studies have been published to allow for meta-analysis of the effects of moderators of the size of effects on episodic memory. The same was true for other important measures of episodic memory performance such as reaction times. Future research should address these limitations.

Effects on Encoding, Consolidation, and/or Retrieval

Another limitation has to do with the complication of interpreting the effects of rTMS on episodic memory due to the design of many studies, particularly studies with multiple rTMS sessions or offline rTMS, because it is difficult to know if the effects of rTMS are modulating encoding, consolidation, and/or retrieval processes. Relatedly, to our knowledge, only one study implemented rTMS to definitively target consolidation-based processes. In a pair of studies, Sandrini et al. (2013) investigated the role of right DLPFC in the reconsolidation of memories for studied words of objects by administering offline 1 Hz rTMS at 100% MT. The first experiment spanned 3 days. On day one, participants learned a list of words with contextual reminders (e.g., specific room and colored bag containing the words). On day two, participants were assigned to receive either rTMS to right DLPFC during a pure consolidation phase without reactivating the memories or rTMS to right DLPFC or the vertex before reactivating the memories in a contextual reminder reconsolidation phase. Finally, on day three, participants underwent a free recall task in which participants recalled as many words as possible. The findings revealed that rTMS prior to memory reactivation led to memory enhancements compared to the no reactivation and vertex control conditions. To examine if rTMS effects were specific to reconsolidation and rather than either encoding or retrieval processes, a follow up experiment used a similar procedure with the exception that memory was assessed 1 h after stimulation instead of the next day. Given that reconsolidation processes take several hours, finding no memory differences across the conditions after 1 h supported the conclusion that the DLPFC was specifically involved in reconsolidation processes (Sandrini et al., 2013). The use of multiple active control conditions further helped the researchers to isolate the effects to reconsolidation processes (as opposed to initial encoding or retrieval). This study demonstrates the unique feasibility of using rTMS to systematically manipulate (re)consolidation processes. However, because there is so little research that has investigated the effects of rTMS on consolidation processes, the current meta-analysis was restricted to studies that have investigated effects on encoding and retrieval stages. This highlights an area where future TMS research is needed.

Variability in the Quality of Targeting Procedures

Another limitation of the current meta-analysis is the possibility that we may have underestimated the true effect of rTMS on episodic memory performance across conditions. The quality assessment revealed that a large number of studies targeted cortical regions using the international 10–20 electrode positioning system. This approach is imprecise and does not take into account inter-individual differences in the location of specific brain structures. Structurally or functionally guided TMS targeting using a neuronavigation system with each subject's structural and/or functional MRI can more precisely stimulate the targeted brain region of interest. Modeling work that has contrasted the variety of targeting methods has shown that larger TMS effects are obtained with individual fMRI-guided TMS neuronavigation when targeting parietal regions than targeting with the 10–20 EEG positioning system, which yielded smaller effect sizes (Sack et al., 2009). Power analyses revealed that a sample size of 47 was required to detect a significant effect when the 10–20 EEG positioning system is used for TMS targeting (Sack et al., 2009). Given that a large number of studies implemented the 10–20 targeting system and the selected studies had an average sample size of 24.19 in the meta-analysis, it is possible that the existing effects sizes available in the published literature underestimate the true effect of rTMS on episodic memory. Another important aspect to note is that only published effect sizes could be included in the meta-analysis and, therefore, excluding any unpublished null effects would obscure and overestimate the true effect of rTMS on episodic memory. Reassuringly, the steps we took to investigate a potential publication bias suggests that this was unlikely to be of major concern. However, it should be noted that absence of small-study biases does not prove the absence of publication bias, and further research is needed to properly address publication bias in multi-level models.

Considerations for Future Research

Personalized rTMS

Below we highlight and discuss potentially important factors that need to be systematically investigated over the next 30 years of TMS research and beyond. One pervasive issue in the field has been the highly variable findings across both behavioral and neural outcomes. Certain individuals may respond to a set of rTMS parameters, while other individuals may have null or negative effects to the same rTMS protocol. One possible solution to this heterogeneity in the response to rTMS is to implement more personalized rTMS parameters titrated to match each individual's physiological characteristics that likely underlie the variability between individuals. We have already noted the need to use structural and functional neuronavigated targeting procedures. Other characteristics that are likely to be important concern the titration of the frequency of stimulation to the precise frequency of endogenous oscillations of interest in targeted brain regions.

For example, in a pair of studies, Waldhauser et al. (2016) implemented a personalized rTMS-EEG design to examine if reactivation of sensory information is necessary for recognition of studied items and/or their studied context/source. In experiment one, participants intentionally encoded visual stimuli (i.e., objects) presented laterally (left and right) to a fixation cross; at test, participants made old/new judgments and source judgments about whether the item initially appeared on the left or right side of the screen while undergoing concurrent EEG recording. Identification of a time window (33.5–204.5 ms), frequency (17.5 Hz), and source (MNI coordinates ± 40, −78, 0) of oscillatory behavior for successful source memory judgments were obtained by examining the lateralization of alpha/beta activity during encoding and retrieval. In a second experiment using the same encoding/retrieval procedure they applied rTMS (17.5 Hz) to the occipital lobe (MNI coordinates ± 40, −78, 0) during retrieval from 33.5 to 204.5 ms after cue presentation (Waldhauser et al., 2016). Importantly, when rTMS was applied using the specific temporal, spatial, and frequency parameters related to the task demands for successful episodic retrieval, there was a decrease in source memory performance relative to control.

In a similar study, Hanslmayr et al. (2014) compared online rTMS-EEG at 90% RMT in the beta frequency range (18.7 Hz) to 10.7, 6.8 Hz, and sham stimulation to the left IFG during word encoding. On a subsequent free recall task, participants showed memory impairment following 18.7 Hz stimulation compared to sham. Critically, despite targeting the same task-dependent region, there were no differences in memory performance for 10.7 or 6.8 Hz stimulation compared to sham. At the neural level, the degree of beta entrainment following 18.7 Hz stimulation was correlated with memory impairment, with higher levels of beta entrainment leading to greater decrements in memory performance (Hanslmayr et al., 2014). The findings that artificial beta synchronization leads to memory impairment supports electrophysiology findings that alpha/beta desynchronization is associated with memory enhancement (Hanslmayr et al., 2016). Thus, the implementation of more nuanced rTMS frequencies affords the possibility to more effectively modulate episodic memory.

Aspects of Episodic Memory

In this review, many of the studies that assessed the effects of rTMS on episodic memory involved spatial locations/associations with stimuli (e.g., object-location associations). While spatial context information is often one important element of episodic memory, there are other important aspects of episodic memories (e.g., object, temporal, social, and emotional information) that may result in dissociable effects of rTMS to different brain regions. Furthermore, it is possible that the type of episodic information (e.g., visual, auditory, multi-modal) may interact with different aspects of episodic memory (e.g., object, temporal, and spatial). Future research should consider these potentially important moderating variables when investigating the effects of rTMS on various aspects of episodic memory.

Closed Loop rTMS

The majority of rTMS findings have focused on researching the brain as a “black box” in an open-loop manner, whereby a priori input stimulation parameters remain constant and one observes the corresponding output. Emerging evidence from multi-modal approaches has begun to clarify how stimulation parameters such as timing and frequency modulate brain-behavior relationships. This has also led to a shift in focus from mapping segregated brain regions to a more interactive-systems approach by attempting to understand how fast-changing patterns of neural activity interact with local and distal brain regions to give rise to human behavior. Understanding the frequency and temporally specific manner that different cortical regions interact has helped uncover how neural processing is linked with behavior. However, the optimal stimulation parameters that lead to episodic memory enhancement or detriment have not been clearly identified and the sparseness of available findings await replication and extension with systematic manipulations to map the parameter space. Moreover, there has been little consideration paid to how memory outcomes may be dependent on interactions between stimulation protocols and temporal patterns of initial brain state activity and ongoing neural activity during a task. The results of the current meta-analysis reveal that it is of paramount importance that future studies take this into consideration in order to determine optimal stimulation parameters for episodic memory (for additional information see, Bergmann et al., 2016)

While this “black box” approach has proven fruitful, the brain is a dynamic system capable of receiving, modulating, and generating both further inputs (modified representations, novel associations) as well as outputs (Berkman and Lieberman, 2011). A wave of technological advancements has made it possible to make stimulation adjustments based on real-time neural signals. This has given rise to an informed closed-loop paradigm where neural activity from EEG/MEG provide amplitude, phase, or power of oscillatory activity markers that allow for stimulation in a temporally specific fashion. This has also paved the way for fully adaptive closed-loops that aim to use stimulated neural markers to not only trigger stimulation but also to use neural activity to adaptively alter stimulation protocols in real time (Karabanov et al., 2016).

Multi-Modal rTMS

Given the wealth of knowledge that has been obtained using multimodal approaches, extending these typically two-way combinations (e.g., TMS-EEG and TMS-fMRI) to three-way TMS-EEG-fMRI tools may provide a more holistic view of brain functioning. Such a combination with TMS enables novel stimulation findings based on the temporal precision of EEG to be complemented with fMRI to examine changes in functionally related neural activity in cortical and subcortical structures with high spatial resolution. The feasibility of combining TMS-EEG-fMRI has already been established to be safe and provide reliable signal recordings (Peters et al., 2013).

Conclusion

The findings of the meta-analysis on younger adults revealed that frequency interacts with MTI as well as with stimulation timing. Specifically, both online and offline 1 Hz rTMS led to enhancing effects, which was driven by below-MTI stimulation. In addition, offline 20 Hz rTMS had enhancing effects whereas, online 20 Hz rTMS and 20 Hz rTMS at below-MTI led to impairing effects on episodic memory. A systematic review of the older adults and those with clinical disorders revealed a similar pattern of enhancing and attenuating effects of rTMS on episodic memory performance. However, important differences did arise with older adults and clinical populations implementing both offline 5 and 10 Hz rTMS protocols that had enhancing effects on episodic memory.

In sum, these findings highlight the importance of the contextual aspects of stimulation to reveal brain-behavior relationships at a more causal level than permitted by methods that are inherently correlational. Important contextual aspects to consider include the brain state of the targeted region and functionally connected regions at the time of stimulation. Furthermore, researchers must take into consideration the time-dependent processes engaged during specific encoding, consolidation, and retrieval phases that are coded by the brain region of interest, as well as variability both between and within individuals in the precise timing and magnitude of activation of such relevant brain areas. Multi-modal and closed-loop approaches offer promise for addressing these likely pernicious contributors to the variability that is seen across the literature, which may be necessary to parse out the role of exogenous stimulation of brain regions and functionally connected regions for modulating episodic memory. Although still in the initial stages of development, the future of closed-loop paradigms offers the ability to analyze stimulation induced brain activity to dynamically fine-tune stimulation parameters to facilitate rTMS effects on cognition. It is exciting to ponder what the next 30 years will elucidate regarding the use of non-invasive brain stimulation for causal modulation of brain-behavior relationships and, ideally, the enhancement of episodic memory in both healthy and clinical populations.

Author Contributions

NY conducted the review and meta-analysis and wrote the initial drafts. NR helped design the study, provide edits, conceptual and theoretical feedback, and write the final draft.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank Clare Hannon, Frank Calabrese, Katie McGuckin, Annie Birmingham, and Davis Gonsalves for their contributions in calculating effect sizes and coding moderator variables. This work was supported by the William P. and Hazel B. White Collegiate Chair fund awarded to NR.

Footnote

1. ^There was also a significant difference of 20 Hz rTMS “above” compared to both 20 Hz rTMS “at” and “below” MTI, although this is relatively uninformative given that only one study applied 20 Hz rTMS “above” MTI, see Tables 4 and 5.

References

Assink, M., and Wibbelink, C. J. M. (2016). Fitting three-level meta-analytic models in R: a step-by-step tutorial. Quant. Methods Psychol. 12, 154–174. doi: 10.20982/tqmp.12.3.p154

CrossRef Full Text | Google Scholar

Balconi, M., and Cobelli, C. (2015). rTMS on left prefrontal cortex contributes to memories for positive emotional cues: a comparison between pictures and words. Neuroscience 287, 93–103. doi: 10.1016/j.neuroscience.2014.12.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Balconi, M., and Ferrari, C. (2013). Left DLPFC rTMS stimulation reduced the anxiety bias effect or how to restore the positive memory processing in high-anxiety subjects. Psychiatry Res. 209, 554–559. doi: 10.1016/j.psychres.2013.03.032

PubMed Abstract | CrossRef Full Text | Google Scholar

Baudic, S., Attal, N., Mhalla, A., Ciampi de Andrade, D., Perrot, S., and Bouhassira, D. (2013). Unilateral repetitive transcranial magnetic stimulation of the motor cortex does not affect cognition in patients with fibromyalgia. J. Psychiatr. Res. 47, 72–77. doi: 10.1016/j.jpsychires.2012.09.003

CrossRef Full Text | Google Scholar

Bergmann, T. O., Karabanov, A., Hartwigsen, G., Thielscher, A., and Siebner, H. R. (2016). Combining non-invasive transcranial brain stimulation with neuroimaging and electrophysiology: current approaches and future perspectives. Neuroimage 140, 4–19. doi: 10.1016/j.neuroimage.2016.02.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Berkers, R. M., van der Linden, M., de Almeida, R. F., Müller, N. C., Bovy, L., Dresler, M., et al. (2017). Transient medial prefrontal perturbation reduces false memory formation. Cortex 88, 42–52. doi: 10.1016/j.cortex.2016.12.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Berkman, E. T., and Lieberman, M. D. (2011). What's outside the black box?: the status of behavioral outcomes in neuroscience research. Psychol. Inq. 22, 100–107. doi: 10.1080/1047840X.2011.550182

PubMed Abstract | CrossRef Full Text | Google Scholar

Blaxton, T. A. (1989). Investigating dissociations among memory measures: support for a transfer-appropriate processing framework. J. Exp. Psychol. 15:657.

Google Scholar

Blumenfeld, R. S., Lee, T. G., and D'Esposito, M. (2014). The effects of lateral prefrontal transcranial magnetic stimulation on item memory encoding. Neuropsychologia 53, 197–202. doi: 10.1016/j.neuropsychologia.2013.11.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Bonnì, S., Veniero, D., Mastropasqua, C., Ponzo, V., Caltagirone, C., and Bozzali, M. (2015). TMS evidence for a selective role of the precuneus in source memory retrieval. Behav. Brain Res. 282, 70–75. doi: 10.1016/j.bbr.2014.12.032

PubMed Abstract | CrossRef Full Text | Google Scholar

Bonnici, H. M., Cheke, L. G., Green, D. A. E., FitzGerald, T. H. M. B., and Simons, J. S. (2018). Specifying a causal role for angular gyrus in autobiographical memory. J. Neurosci. 38, 10438–10443. doi: 10.1523/JNEUROSCI.1239-18.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

Brunoni, A. R., and Vanderhasselt, M. A. (2014). Working memory improvement with non-invasive brain stimulation of the dorsolateral prefrontal cortex: a systematic review and meta-analysis. Brain Cogn. 86, 1–9. doi: 10.1016/j.bandc.2014.01.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Buckner, R. L., Kelley, W. M., and Petersen, S. E. (1999). Frontal cortex contributes to human memory formation. Nat. Neurosci. 4, 311–314. doi: 10.1038/7221

CrossRef Full Text | Google Scholar

Cabeza, R., and Moscovitch, M. (2013). Memory systems, processing modes, and components: functional neuroimaging evidence. Perspect. Psychol. Sci. 8, 49–55. doi: 10.1177/1745691612469033

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheung, M. W. (2014). Modeling dependent effect sizes with three-level meta-analyses: a structural equation modeling approach. Psychol. Methods 19, 211–229. doi: 10.1037/a0032968

PubMed Abstract | CrossRef Full Text | Google Scholar

Chung, S. W., Rogasch, N. C., Hoy, K. E., Sullivan, C. M., Cash, R. F. H., and Fitzgerald, P. B. (2018). Impact of different intensities of intermittent theta burst stimulation on the cortical properties during TMS-EEG and working memory performance. Hum. Brain Mapp. 39, 783–802. doi: 10.1002/hbm.23882

PubMed Abstract | CrossRef Full Text | Google Scholar

Ciaramelli, E., Grady, C. L., and Moscovitch, M. (2008). Top-down and bottom-up attention to memory: a hypothesis (AtoM) on the role of the posterior parietal cortex in memory retrieval. Neuropsychologia 46, 1828–1851. doi: 10.1016/j.neuropsychologia.2008.03.022

PubMed Abstract | CrossRef Full Text | Google Scholar

Copas, J. (1999). What works?: selectivity models and meta-analysis. J. R. Stat. Soc. 162, 95–109. doi: 10.1111/1467-985X.00123

CrossRef Full Text | Google Scholar

Cotelli, M., Calabria, M., Manenti, R., Rosini, S., Maioli, C., and Zanetti, O. (2012). Brain stimulation improves associative memory in an individual with amnestic mild cognitive impairment. Neurocase 18, 217–223. doi: 10.1080/13554794.2011.588176

PubMed Abstract | CrossRef Full Text | Google Scholar

Craik, F. I., and Rose, N. S. (2012). Memory encoding and aging: a neurocognitive perspective. Neurosci. Biobehav. Rev. 36, 1729–1739. doi: 10.1016/j.neubiorev.2011.11.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Cumming, G. (2012). Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis. New York, NY: Routledge.

Google Scholar

Davis, N. J., Gold, E., Pascual-leone, A., and Bracewell, R. M. (2013). Challenges of proper placebo control for non-invasive brain stimulation in clinical and experimental applications. Eur. J. Neurosci. 38, 2973–2977. doi: 10.1111/ejn.12307

PubMed Abstract | CrossRef Full Text | Google Scholar

Davis, S. W., Luber, B., Murphy, D. L. K., Lisanby, S. H., and Cabeza, R. (2017). Frequency-specific neuromodulation of local and distant connectivity in aging and episodic memory function. Hum. Brain Mapp. 38, 5987–6004. doi: 10.1002/hbm.23803

PubMed Abstract | CrossRef Full Text | Google Scholar

Demeter, E., Mirdamadi, J. L., Meehan, S. K., and Taylor, S. F. (2016). Short theta burst stimulation to left frontal cortex prior to encoding enhances subsequent recognition memory. Cogn. Affect. Behav. Neurosci. 16, 724–735. doi: 10.3758/s13415-016-0426-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Dobbins, I. G., Rice, H. J., Wagner, A. D., and Schacter, D. L. (2003). Memory orientation and success: separable neurocognitive components underlying episodic recognition. Neuropsychologia 41, 318–333. doi: 10.1016/S0028-3932(02)00164-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Drumond Marra, H. L., Myczkowski, M. L., Maia Memória, C., Arnaut, D., Leite Ribeiro, P., Sardinha Mansur, C. G., et al. (2015). Transcranial magnetic stimulation to address mild cognitive impairment in the elderly: a randomized controlled study. Behav. Neurol. 2015:287843. doi: 10.1155/2015/287843

PubMed Abstract | CrossRef Full Text | Google Scholar

Duecker, F., and Sack, A. T. (2015). Rethinking the role of sham TMS 6, 1–5. doi: 10.3389/fpsyg.2015.00210

PubMed Abstract | CrossRef Full Text | Google Scholar

Egger, M., Davey Smith, G., Schneider, M., and Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ 315, 629–634. doi: 10.1136/bmj.315.7109.629

PubMed Abstract | CrossRef Full Text | Google Scholar

Eichenbaum, H. (2017). Prefrontal–hippocampal interactions in episodic memory. Nat. Rev. Neurosci. 18, 547–558. doi: 10.1038/nrn.2017.74

PubMed Abstract | CrossRef Full Text | Google Scholar

Epstein, C. M., Sekino, M., Yamaguchi, K., Kamiya, S., and Ueno, S. (2002). Asymmetries of prefrontal cortex in human episodic memory: effects of transcranial magnetic stimulation on learning abstract patterns. Neurosci. Lett. 320, 5–8. doi: 10.1016/S0304-3940(01)02573-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Feurra, M., Fuggetta, G., Rossi, S., and Walsh, V. (2010). The role of the left inferior frontal gyrus in episodic encoding of faces: an interference study by repetitive transcranial magnetic stimulation. Cogn. Neurosci. 1, 118–125. doi: 10.1080/17588921003660736

PubMed Abstract | CrossRef Full Text | Google Scholar

Floel, A., Poeppel, D., Buffalo, E. A., Braun, A., Wu, C. W. H., Seo, H. J., et al. (2004). Prefrontal cortex asymmetry for memory encoding of words and abstract shapes. Cereb. Cortex 14, 404–409. doi: 10.1093/cercor/bhh002

PubMed Abstract | CrossRef Full Text | Google Scholar

Gagnon, G., Blanchet, S., Grondin, S., and Schneider, C. (2010). Paired-pulse transcranial magnetic stimulation over the dorsolateral prefrontal cortex interferes with episodic encoding and retrieval for both verbal and non-verbal materials. Brain Res. 1344, 148–158. doi: 10.1016/j.brainres.2010.04.041

PubMed Abstract | CrossRef Full Text | Google Scholar

Galli, G., Feurra, M., Pavone, E. F., Sirota, M., and Rossi, S. (2017). Dynamic changes in prefrontal cortex involvement during verbal episodic memory formation. Biol. Psychol. 125, 36–44. doi: 10.1016/j.biopsycho.2017.02.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Grafman, J., and Wassermann, E. (1998). Transcranial magnetic stimulation can measure and modulate learning and memory. Neuropsychologia 37, 159–167.

PubMed Abstract | Google Scholar

Habib, R., Nyberg, L., and Tulving, E. (2003). Hemispheric asymmetries of memory: the HERA model revisited. Trends Cogn. Sci. 7, 241–245. doi: 10.1016/S1364-6613(03)00110-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Hallett, M. (2007). Primer transcranial magnetic stimulation : a primer. Neuron 55, 187–199. doi: 10.1016/j.neuron.2007.06.026

PubMed Abstract | CrossRef Full Text | Google Scholar

Hanslmayr, S., Matuschek, J., and Fellner, M. C. (2014). Entrainment of prefrontal beta oscillations induces an endogenous echo and impairs memory formation. Curr. Biol. 24, 904–909. doi: 10.1016/j.cub.2014.03.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Hanslmayr, S., Staresina, B. P., and Bowman, H. (2016). Oscillations and episodic memory: addressing the synchronization/desynchronization conundrum. Trends Neurosci. 39, 16–25. doi: 10.1016/j.tins.2015.11.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Hawco, C., Armony, J. L., Daskalakis, Z. J., Berlim, M. T., Chakravarty, M. M., and Pike, G. B. (2017). Differing time of onset of concurrent TMS-fMRI during associative memory encoding: a measure of dynamic connectivity. Front. Hum. Neurosci. 11:404. doi: 10.3389/fnhum.2017.00404

PubMed Abstract | CrossRef Full Text | Google Scholar

Hawco, C., Berlim, M. T., and Lepage, M. (2013). The dorsolateral prefrontal cortex plays a role in self-initiated elaborative cognitive processing during episodic memory encoding: RTMS evidence. PLoS ONE 8:e73789. doi: 10.1371/journal.pone.0073789

PubMed Abstract | CrossRef Full Text | Google Scholar

Hedges, L. V., and Olkin, I. (1985). Statistical methods in meta-analysis. J. Educ. Stat. 20, 128–131. doi: 10.2307/1164953

CrossRef Full Text | Google Scholar

Hedges, L. V., and Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychol. Methods 6, 203–217. doi: 10.1037/1082-989X.6.3.203

PubMed Abstract | CrossRef Full Text | Google Scholar

Hedges, L. V., and Vevea, J. L. (1996). Estimating effect size under publication bias: small sample properties and robustness of a random effects selection model. J. Educ. Behav. Stat. 21, 299–332. doi: 10.2307/1165338

CrossRef Full Text | Google Scholar

Higgins, J., Altman, D., and Green, S. (2008). Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 (updated September 2008) Oxford: Cochrane Collaborations.

Google Scholar

Hsu, W.-Y., Ku, Y., Zanto, T. P., and Gazzaley, A. (2015). Effects of noninvasive brain stimulation on cognitive function in healthy aging and Alzheimer's disease: a systematic review and meta-analysis. Neurobiol. Aging 36, 2348–2359. doi: 10.1016/j.neurobiolaging.2015.04.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, Y.Z., Edwards, M. J., Rounis, E., Bhatia, K. P., and Rothwell, J. C. (2005). Theta burst stimulation of the human motor cortex. Neuron 45, 201–206. doi: 10.1016/j.neuron.2004.12.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Innocenti, I., Cappa, S. F., Feurra, M., Giovannelli, F., Santarnecchi, E., Bianco, G., et al. (2013). TMS interference with primacy and recency mechanisms reveals bimodal episodic encoding in the human brain. J. Cogn. Neurosci. 25, 109–116. doi: 10.1162/jocn_a_00304

PubMed Abstract | CrossRef Full Text | Google Scholar

Innocenti, I., Giovannelli, F., Cincotta, M., Feurra, M., Polizzotto, N. R., Bianco, G., et al. (2010). Event-related rTMS at encoding affects differently deep and shallow memory traces. Neuroimage 53, 325–330. doi: 10.1016/j.neuroimage.2010.06.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Jelić, M. B., Milanović, S. D., and Filipović, S. R. (2015). Differential effects of facilitatory and inhibitory theta burst stimulation of the primary motor cortex on motor learning. Clin. Neurophys. 126, 1016–1023. doi: 10.1016/j.clinph.2014.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Julian, J. B., Ryan, J., Hamilton, R. H., and Epstein, R. A. (2016). The occipital place area is causally involved in representing environmental boundaries during navigation. Curr. Biol. 26, 1104–1109. doi: 10.1016/j.cub.2016.02.066

PubMed Abstract | CrossRef Full Text | Google Scholar

Kahn, I. (2005). Transient disruption of ventrolateral prefrontal cortex during verbal encoding affects subsequent memory performance. J. Neurophysiol. 94, 688–698. doi: 10.1152/jn.01335.2004

PubMed Abstract | CrossRef Full Text | Google Scholar

Kammer, T., Beck, S., Thielscher, A., Laubis-Hermann, U., and Topka, H. (2001). Motor threshold in humans: a transcranial magnetic stimulation study comparing different pulse waveforms, current directions and stimulator types. Clin. Neurophys. 112, 250–258. doi: 10.1016/S1388-2457(00)00513-7

CrossRef Full Text | Google Scholar

Karabanov, A., Thielscher, A., and Siebner, H. R. (2016). Transcranial brain stimulation: closing the loop between brain and stimulation. Curr. Opin. Neurol. 29, 397–404. doi: 10.1097/WCO.0000000000000342

PubMed Abstract | CrossRef Full Text | Google Scholar

Kavanaugh, B. C., Aaronson, S. T., Clarke, G. N., Holtzheimer, P. E., Johnson, C. W., McDonald, W. M., et al. (2018). Neurocognitive effects of repetitive transcranial magnetic stimulation with a 2-coil device in treatment-resistant major depressive disorder. J. ECT 34, 258–265. doi: 10.1097/YCT.0000000000000494

PubMed Abstract | CrossRef Full Text | Google Scholar

Koch, G., Bonnì, S., Pellicciari, M. C., Casula, E. P., Mancini, M., Esposito, R., et al. (2018). Transcranial magnetic stimulation of the precuneus enhances memory and neural activity in prodromal Alzheimer's disease. Neuroimage 169, 302–311. doi: 10.1016/j.neuroimage.2017.12.048

PubMed Abstract | CrossRef Full Text | Google Scholar

Koen, J. D., Thakral, P. P., and Rugg, M. D. (2018). Transcranial magnetic stimulation of the left angular gyrus during encoding does not impair associative memory performance. Cogn. Neurosci. 9, 127–138. doi: 10.1080/17588928.2018.1484723

PubMed Abstract | CrossRef Full Text | Google Scholar

Köhler, S., Paus, T., Buckner, R. L., and Milner, B. (2004). Effects of left inferior prefrontal stimulation on episodic memory formation: a two-stage fMRI—rTMS study. J. Cogn. Neurosci. 16, 178–188. doi: 10.1162/089892904322984490

PubMed Abstract | CrossRef Full Text | Google Scholar

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front. Psychol. 4:863. doi: 10.3389/fpsyg.2013.00863

PubMed Abstract | CrossRef Full Text | Google Scholar

Luber, B., and Lisanby, S. (2014). Enhancement of human cognitive performance using transcranial magnetic stimulation (TMS). Neuroimage 85, 961–970. doi: 10.1016/j.neuroimage.2013.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Machizawa, M. G., Kalla, R., Walsh, V., and Otten, L. J. (2010). The time course of ventrolateral prefrontal cortex involvement in memory formation. J. Neurophys. 103, 1569–1579. doi: 10.1152/jn.90937.2008

PubMed Abstract | CrossRef Full Text | Google Scholar

Manenti, R., Cotelli, M., Calabria, M., Maioli, C., and Miniussi, C. (2010a). The role of the dorsolateral prefrontal cortex in retrieval from long-term memory depends on strategies: a repetitive transcranial magnetic stimulation study. Neuroscience 166, 501–507. doi: 10.1016/j.neuroscience.2009.12.037

PubMed Abstract | CrossRef Full Text | Google Scholar

Manenti, R., Cotelli, M., and Miniussi, C. (2011). Successful physiological aging and episodic memory: a brain stimulation study. Behav. Brain Res. 216, 153–158. doi: 10.1016/j.bbr.2010.07.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Manenti, R., Cotelli, M., Robertson, I. H., and Miniussi, C. (2012). Transcranial brain stimulation studies of episodic memory in young adults, elderly adults and individuals with memory dysfunction: a review. Brain Stimul. 5, 103–109. doi: 10.1016/j.brs.2012.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Manenti, R., Tettamanti, M., Cotelli, M., Miniussi, C., and Cappa, S. F. (2010b). The neural bases of word encoding and retrieval: A fMRI-guided transcranial magnetic stimulation study. Brain Topogr. 22, 318–332. doi: 10.1007/s10548-009-0126-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Marin, B. M., VanHaerents, S. A., Voss, J. L., and Bridge, D. J. (2018). Prefrontal θ-burst stimulation disrupts the organizing influence of active short-term retrieval on episodic memory. Eneuro 5:ENEURO.0347-17.2018. doi: 10.1523/ENEURO.0347-17.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

Martin-Trias, P., Lanteaume, L., Solana, E., Cassé-Perrot, C., Fernández-Cabello, S., Babiloni, C., et al. (2018). Adaptability and reproducibility of a memory disruption rTMS protocol in the PharmaCog IMI European project. Sci. Rep. 8:9371. doi: 10.1038/s41598-018-27502-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Miniussi, C., Harris, J. A., and Ruzzoli, M. (2013). Neuroscience and biobehavioral reviews modelling non-invasive brain stimulation in cognitive neuroscience? Neurosci. Biobehav. Rev. 37, 1702–1712. doi: 10.1016/j.neubiorev.2013.06.014

CrossRef Full Text | Google Scholar

Morris, C. D., Bransford, J. D., and Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. J. Verbal Learn. Verbal Behav. 16, 519–533. doi: 10.1016/S0022-5371(77)80016-9

CrossRef Full Text | Google Scholar

Moscovitch, M., Cabeza, R., Winocur, G., and Nadel, L. (2016). Episodic memory and beyond: the hippocampus and neocortex in transformation. Annu. Rev. Psychol. 67, 105–134. doi: 10.1146/annurev-psych-113011-143733

PubMed Abstract | CrossRef Full Text | Google Scholar

Ngomo, S., Leonard, G., Moffet, H., and Mercier, C. (2012). Comparison of transcranial magnetic stimulation measures obtained at rest and under active conditions and their reliability. J. Neurosci. Methods 205, 65–71. doi: 10.1016/j.jneumeth.2011.12.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Nilakantan, A. S., Bridge, D. J., Gagnon, E. P., VanHaerents, S. A., and Voss, J. L. (2017). Stimulation of the posterior cortical-hippocampal network enhances precision of memory recollection. Curr. Biol. 27, 465–470. doi: 10.1016/j.cub.2016.12.042

PubMed Abstract | CrossRef Full Text | Google Scholar

Pascual-Leone, A., Tormos, J. M., Keenan, J., Tarazona, F., Cañete, C., and Catalá, M. D. (1998). Study and modulation of human cortical excitability with transcranial magnetic stimulation. J. Clin. Neurophys. 15, 333–343.

PubMed Abstract | Google Scholar

Peña-Gomez, C., Solé-Padullés, C., Clemente, I. C., Junqué, C., Bargalló, N., Bosch, B., et al. (2012). APOE status modulates the changes in network connectivity induced by brain stimulation in non-demented elders. PLoS ONE 7:e51833. doi: 10.1371/journal.pone.0051833

PubMed Abstract | CrossRef Full Text | Google Scholar

Peters, J. C., Reithler, J., Schuhmann, T., de Graaf, T., Uludag, K., and Goebel, R. (2013). On the feasibility of concurrent human TMS-EEG-fMRI measurements. J. Neurophys. 109, 1214–1227. doi: 10.1152/jn.00071.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Prince, S. E. (2005). Neural correlates of relational memory: successful encoding and retrieval of semantic and perceptual associations. J. Neurosci. 25, 1203–1210. doi: 10.1523/JNEUROSCI.2540-04.2005

PubMed Abstract | CrossRef Full Text | Google Scholar

Qiao, J., Jin, G., Lei, L., Wang, L., Du, Y., and Wang, X. (2016). The positive effects of high-frequency right dorsolateral prefrontal cortex repetitive transcranial magnetic stimulation on memory, correlated with increases in brain metabolites detected by proton magnetic resonance spectroscopy in recently detoxified al. Neuropsychiatr. Dis. Treat. 12, 2273–2278. doi: 10.2147/NDT.S106266

CrossRef Full Text | Google Scholar

Rami, L., Gironell, A., Kulisevsky, J., García-Sánchez, C., Berthier, M., and Estévez-González, A. (2003). Effects of repetitive transcranial magnetic stimulation on memory subtypes: a controlled study. Neuropsychologia 41, 1877–1883. doi: 10.1016/S0028-3932(03)00131-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Rektorova, I., Megova, S., Bares, M., and Rektor, I. (2005). Cognitive functioning after repetitive transcranial magnetic stimulation in patients with cerebrovascular disease without dementia: a pilot study of seven patients. J. Neurol. Sci. 229–230, 157–161. doi: 10.1016/j.jns.2004.11.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Romei, V., Thut, G., and Silvanto, J. (2016). Information-based approaches of noninvasive transcranial brain stimulation. Trends Neurosci. 39, 782–795. doi: 10.1016/j.tins.2016.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossi, S., Cappa, S. F., Babiloni, C., Pasqualetti, P., Miniussi, C., Carducci, F., et al. (2001). Prefrontal [correction of Prefontal] cortex in long-term memory: an “interference” approach using magnetic stimulation. Nat. Neurosci. 4, 948–952. doi: 10.1038/nn0901-948

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossi, S., Hallett, M., Rossini, P. M., and Pascual-Leone, A. (2009). Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research. Clin. Neurophysiol. 120, 2008–2039. doi: 10.1016/j.clinph.2009.08.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossi, S., Innocenti, I., Polizzotto, N. R., Feurra, M., De Capua, A., Ulivelli, M., et al. (2011). Temporal dynamics of memory trace formation in the human prefrontal cortex. Cereb. Cortex 21, 368–373. doi: 10.1093/cercor/bhq103

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossi, S., Miniussi, C., Pasqualetti, P., Babiloni, C., Rossini, P. M., and Cappa, S. F. (2004). Age-related functional changes of prefrontal cortex in long-term memory: a repetitive transcranial magnetic stimulation study. J. Neurosci. 24, 7939–7944. doi: 10.1523/JNEUROSCI.0703-04.2004

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossi, S., Pasqualetti, P., Zito, G., Vecchio, F., Cappa, S. F., Miniussi, C., et al. (2006). Prefrontal and parietal cortex in human episodic memory: an interference study by repetitive transcranial magnetic stimulation. Euro. J. Neurosci. 23, 793–800. doi: 10.1111/j.1460-9568.2006.04600.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossi, S., and Rossini, P. M. (2004). TMS in cognitive plasticity and the potential for rehabilitation. Trends Cogn. Sci. 8, 273–279. doi: 10.1016/j.tics.2004.04.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossini, P. M., Burke, D., Chen, R., Cohen, L. G., Daskalakis, Z., Di Iorio, R., et al. (2015). Non-invasive electrical and magnetic stimulation of the brain, spinal cord, roots and peripheral nerves: basic principles and procedures for routine clinical and research application. An updated report from an IFCN Committee. Clin. Neurophysiol. 126, 1071–1107. doi: 10.1016/j.clinph.2015.02.001

CrossRef Full Text | Google Scholar

Rugg, M. D., and Vilberg, K. L. (2013). Brain networks underlying episodic memory retrieval. Curr. Opin. Neurobiol. 23, 255–260. doi: 10.1016/j.conb.2012.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Ryals, A. J., Rogers, L. M., Gross, E. Z., Polnaszek, K. L., and Voss, J. L. (2016). Associative recognition memory awareness improved by theta-burst stimulation of frontopolar cortex. Cereb. Cortex 26, 1200–1210. doi: 10.1093/cercor/bhu311

PubMed Abstract | CrossRef Full Text | Google Scholar

Sack, A. T., Cohen Kadosh, R., Schuhmann, T., Moerel, M., Walsh, V., and Goebel, R. (2009). Optimizing functional accuracy of TMS in cognitive studies: a comparison of methods. J. Cogn. Neurosci. 21, 207–221. doi: 10.1162/jocn.2009.21126

PubMed Abstract | CrossRef Full Text | Google Scholar

Sandrini, M., Cappa, S. F., Rossi, S., Rossini, P. M., and Miniussi, C. (2003). The role of prefrontal cortex in verbal episodic memory: rTMS evidence. J. Cogn. Neurosci. 15, 855–861. doi: 10.1162/089892903322370771

PubMed Abstract | CrossRef Full Text | Google Scholar

Sandrini, M., Censor, N., Mishoe, J., and Cohen, L. G. (2013). Causal role of prefrontal cortex in strengthening of episodic memories through reconsolidation. Curr. Biol. 23, 2181–2184. doi: 10.1016/j.cub.2013.08.045

PubMed Abstract | CrossRef Full Text | Google Scholar

Sandrini, M., Umiltà, C., and Rusconi, E. (2011). The use of transcranial magnetic stimulation in cognitive neuroscience: a new synthesis of methodological issues. Neurosci. Biobehav. Rev. 35, 516–536. doi: 10.1016/j.neubiorev.2010.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Sauvé, W. M., and Crowther, L. J. (2014). The science of transcranial magnetic stimulation. Psychiatr. Ann. 44, 279–283. doi: 10.3928/00485713-20140609-05

CrossRef Full Text | Google Scholar

Schwarzkopf, D. S., Silvanto, J., and Rees, G. (2011). Stochastic resonance effects reveal the neural mechanisms of transcranial magnetic stimulation. J. Neurosci. 31, 3143–3147. doi: 10.1523/JNEUROSCI.4863-10.2011.

PubMed Abstract | CrossRef Full Text | Google Scholar

Sestieri, C., Capotosto, P., Tosoni, A., Luca Romani, G., and Corbetta, M. (2013). Interference with episodic memory retrieval following transcranial stimulation of the inferior but not the superior parietal lobule. Neuropsychologia 51, 900–906. doi: 10.1016/j.neuropsychologia.2013.01.023

CrossRef Full Text | Google Scholar

Shafi, M. M., Westover, M. B., Fox, M. D., and Pascual-Leone, A. (2012). Exploration and modulation of brain network interactions with noninvasive brain stimulation in combination with neuroimaging. Euro. J. Neurosci. 35, 805–825. doi: 10.1111/j.1460-9568.2012.08035.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Silvanto, J., Bona, S., Marelli, M., and Cattaneo, Z. (2018). On the mechanisms of transcranial magnetic stimulation (TMS): how brain state and baseline performance level determine behavioral effects of TMS. Front. Psychol. 9:741. doi: 10.3389/fpsyg.2018.00741

PubMed Abstract | CrossRef Full Text | Google Scholar

Silvanto, J., and Cattaneo, Z. (2017). Common framework for “virtual lesion” and state-dependent TMS: the facilitatory/suppressive range model of online TMS effects on behavior. Brain Cogn. 119, 32–38. doi: 10.1016/j.bandc.2017.09.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Silvanto, J., Muggleton, N., and Walsh, V. (2008). State-dependency in brain stimulation studies of perception and cognition. Trends Cogn. Sci. 12, 447–454. doi: 10.1016/j.tics.2008.09.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Škrdlantová, L., Horáček, J., Dockery, C., Lukavský, J., Kopeček, M., Preiss, M., et al. (2005). The influence of low-frequency left prefrontal repetitive transcranial magnetic stimulation on memory for words but not for faces. Physiol. Res. 54, 123–128. doi: 10.1002/14651858.CD012700

CrossRef Full Text | Google Scholar

Solé-Padullés, C., Bartrés-Faz, D., Junqué, C., Clemente, I. C., Molinuevo, J. L., Bargalló, N., et al. (2006). Repetitive transcranial magnetic stimulation effects on brain function and cognition among elders with memory dysfunction. a randomized sham-controlled study. Cereb. Cortex 16, 1487–1493. doi: 10.1093/cercor/bhj083

PubMed Abstract | CrossRef Full Text | Google Scholar

Squire, L. R. (2009). The legacy of patient H.M. for neuroscience. Neuron 61, 6–9. doi: 10.1016/j.neuron.2008.12.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., and Higgins, J. P. T. (2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomized controlled trials. BMJ 343:d4002. doi: 10.1136/bmj.d4002

CrossRef Full Text | Google Scholar

Tambini, A., Nee, D. E., and D'Esposito, M. (2018). Hippocampal-targeted theta-burst stimulation enhances associative memory formation. J. Cogn. Neurosci. 30, 1452–1472. doi: 10.1162/jocn_a_01300

PubMed Abstract | CrossRef Full Text | Google Scholar

Terrin, N., Schmid, C. H., and Lau, J. (2005). In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. J. Clin. Epidemiol. 58, 894–901. doi: 10.1016/j.jclinepi.2005.01.006

CrossRef Full Text | Google Scholar

Thakral, P. P., Madore, K. P., and Schacter, D. L. (2017). A role for the left angular gyrus in episodic simulation and memory. J. Neurosci. 37, 1319–1317. doi: 10.1523/JNEUROSCI.1319-17.2017

PubMed Abstract | CrossRef Full Text | Google Scholar

Tulving, E. (1985). Memory and consciousness. Can. Psychol. 26, 1–12. doi: 10.1037/h0080017

CrossRef Full Text | Google Scholar

Tulving, E. (2002). EPISODIC MEMORY: from mind to brain. Ann. Rev. Psychol. 53, 1–25. doi: 10.1146/annurev.psych.53.100901.135114

PubMed Abstract | CrossRef Full Text | Google Scholar

Tulving, E., and Thomson, D. M. (1973). Encoding specificity and retrieval processes in episodic memory. Psychol. Rev. 80:352.

Google Scholar

Turriziani, P., Oliveri, M., Salerno, S., Costanzo, F., Koch, G., Caltagirone, C., et al. (2008). Recognition memory and prefrontal cortex: dissociating recollection and familiarity processes using rTMS. Behav. Neurol. 19, 23–27. doi: 10.1155/2008/568057

PubMed Abstract | CrossRef Full Text | Google Scholar

Turriziani, P., Smirni, D., Zappalà, G., Mangano, G. R., Oliveri, M., and Cipolotti, L. (2012). Enhancing memory performance with rTMS in healthy subjects and individuals with mild cognitive impairment: the role of the right dorsolateral prefrontal cortex. Front. Hum. Neurosci. 6:62. doi: 10.3389/fnhum.2012.00062

PubMed Abstract | CrossRef Full Text | Google Scholar

Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., and Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behav. Res. Methods 45, 576–594. doi: 10.3758/s13428-012-0261-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Vidal-Piñeiro, D., Martin-Trias, P., Arenaza-Urquijo, E. M., Sala-Llonch, R., Clemente, I. C., Mena-Sánchez, I., et al. (2014). Task-dependent activity and connectivity predict episodic memory network-based responses to brain stimulation in healthy aging. Brain Stimul. 7, 287–296. doi: 10.1016/j.brs.2013.12.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48. doi: 10.18637/jss.v036.i03

CrossRef Full Text | Google Scholar

Wagner, A. D., Shannon, B. J., Kahn, I., and Buckner, R. L. (2005). Parietal lobe contributions to episodic memory retrieval. Trends Cogn. Sci. 9, 445–453. doi: 10.1016/j.tics.2005.07.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Wagner, T., Rushmore, J., Eden, U., and Valero-Cabre, A. (2009). Biophysical foundations underlying TMS: setting the stage for an effective use of neurostimulation in the cognitive neurosciences. Cortex 45, 1025–1034. doi: 10.1016/j.cortex.2008.10.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Wais, P. E., Kim, O. Y., and Gazzaley, A. (2012). Distractibility during episodic retrieval is exacerbated by perturbation of left ventrolateral prefrontal cortex. Cereb. Cortex 22, 717–724. doi: 10.1093/cercor/bhr160

PubMed Abstract | CrossRef Full Text | Google Scholar

Waldhauser, G. T., Braun, V., and Hanslmayr, S. (2016). Episodic memory retrieval functionally relies on very rapid reactivation of sensory information. J. Neurosci. 36, 251–260. doi: 10.1523/JNEUROSCI.2101-15.2016

PubMed Abstract | CrossRef Full Text | Google Scholar

Walsh, V., and Cowey, A. (2000). Transcranial magnetic stimulation and cognitive neuroscience. Nat. Rev. Neurosci. 1, 73–79. doi: 10.1038/35036239

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, J. X., Rogers, L. M., Gross, E. Z., Ryals, A. J., Dokucu, M. E., Brandstatt, K. L., et al. (2014). Memory enhancement: targeted enhancement of cortical-hippocampal brain networks and associative memory. Science 345, 1054–1057. doi: 10.1126/science.1252900

PubMed Abstract | CrossRef Full Text | Google Scholar

Yazar, Y., Bergström, Z. M., and Simons, J. S. (2014). Continuous theta burst stimulation of angular gyrus reduces subjective recollection. PLoS ONE 9:e110414. doi: 10.1371/journal.pone.0110414

PubMed Abstract | CrossRef Full Text | Google Scholar

Yazar, Y., Bergström, Z. M., and Simons, J. S. (2017). Reduced multimodal integration of memory features following continuous theta burst stimulation of angular gyrus. Brain Stimul. 10, 624–629. doi: 10.1016/j.brs.2017.02.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Ye, Q., Zou, F., Lau, H., Hu, Y., and Kwok, S. C. (2018). Causal evidence for mnemonic metacognition in human precuneus. J. Nurosci. 38, 6379–6387. doi: 10.1101/280750

PubMed Abstract | CrossRef Full Text | Google Scholar

Yonelinas, A. P. (2002). The nature of recollection and familiarity : a review of 30 years of research. J. Mem. Lang. 517, 441–517. doi: 10.1006/jmla.2002.2864

CrossRef Full Text | Google Scholar

Yonelinas, A. P. (2005). Separating the brain regions involved in recollection and familiarity in recognition memory. J. Neurosci. 25, 3002–3008. doi: 10.1523/JNEUROSCI.5295-04.2005

PubMed Abstract | CrossRef Full Text | Google Scholar