Efficacy of twice-daily high-frequency repetitive transcranial magnetic stimulation on associative memory

Objectives: Several studies have examined the effects of repetitive transcranial magnetic stimulation (rTMS) on associative memory (AM) but findings were inconsistent. Here, we aimed to test whether twice-daily rTMS could significantly improve AM.

Methods: In this single-blind, sham-controlled experiment, 40 participants were randomized to receive twice-daily sham or real rTMS sessions for five consecutive days (a total of 16,000 pulses). The stimulation target in left inferior parietal lobule (IPL) exhibiting peak functional connectivity to the left hippocampus was individually defined for each participant. Participants completed both a picture-cued word association task and Stroop test at baseline and 1 day after the final real or sham rTMS session. Effects of twice-daily rTMS on AM and Stroop test performance were compared using two-way repeated measures analysis of variance with main factors Group (real vs. sham) and Time (baseline vs. post-rTMS).

Results: There was a significant Group × Time interaction effect. AM score was significantly enhanced in the twice-daily real group after rTMS, but this difference could not survive the post hoc analysis after multiple comparison correction. Further, AM improvement in the twice-daily real group was not superior to a previously reported once-daily rTMS group receiving 8,000 pulses. Then, we combined the twice- and once-daily real groups, and found a significant Group × Time interaction effect. Post hoc analysis indicated that the AM score was significantly enhanced in the real group after multiple comparisons correction.

Conclusion: Our prospective experiment did not show significant rTMS effect on AM, but this effect may become significant if more participants could be recruited as revealed by our retrospective analysis.

Introduction

Transcranial magnetic stimulation (TMS) is a non-invasive technique for modulating brain network connectivity with demonstrated therapeutic efficacy against neurological and neuropsychiatric illnesses (Fox et al., 2012). Wang et al. (2014) reported that once-daily repetitive TMS (rTMS) to the inferior parietal lobule (IPL), a region strongly connected to the hippocampus, significantly improved associative memory (AM) in healthy participants, suggesting possible utility for treatment of disorders characterized by AM deficits, such as stroke, age-related cognitive decline, neurotrauma, and various neuropsychiatric and neurodegenerative conditions.

However, the physiological response to non-invasive brain stimulation is known to be highly variable among individuals (López-Alonso et al., 2014), and several subsequent TMS studies found no significant AM improvement. For instance, a survey of research groups found that approximately 50% were not able to reproduce rTMS effects from original publications and a recent investigation by Héroux et al. (2015) reported significant changes in functional connectivity (FC) following multi-day rTMS of the parietal cortex but no AM enhancement (Hendrikse et al., 2020). Similarly, we found no significant difference in AM following once-daily rTMS sessions for 5 days compared to a sham group (Gao et al., 2021). Collectively, these inconsistencies suggest that rTMS efficacy for improving AM is highly dependent on stimulus protocol (e.g., stimulus intensity, duration), target, study design, and (or) treatment group characteristics.

Increasing the stimulation dose (total number of impulses) is one potential method to achieve a more robust effect on AM (Nettekoven et al., 2014). A recent study found that a high-dose rTMS protocol is safe and produces more reliable remission from depression (Cole et al., 2020). In addition to stimulation dose, the experimental design may influence outcome. The seminal study by Wang et al. (2014) and most subsequent studies reproducing AM improvement (Freedberg et al., 2019; Hermiller et al., 2019; Hendrikse et al., 2020; Gao et al., 2021) used a within-subject crossover design with an approximately 2-week delay (washout period) between real and control (sham) stimulation. However, the AM improvement from baseline was still significant 2-weeks after real rTMS, suggesting that a longer interval is needed for crossover studies (Wang and Voss, 2015; Gao et al., 2021). Nonetheless, no study has specifically examined the optimal interval for comparison of control (sham) stimulation to real stimulation.

In the present study, we aimed to investigated whether a higher rTMS dose could produce significantly greater AM improvement, beyond the sham rTMS. To this end, we modified the paradigm of our previous study (Gao et al., 2021) in two points: (1) using a parallel rather than crossover design; (2) doubling the 20-Hz rTMS dose (twice-daily sessions for 5 days, total 16,000 pulses).

Materials and methods

Participants

Forty healthy subjects (24 females and 16 males) with no history of rTMS, transcranial electric stimulation, neuropsychological disorders, or psychoactive drug use were recruited for this study. All participants met the safety criteria for MRI and rTMS (Rossi et al., 2009) and provided written informed consent. Each was remunerated for their participation after study completion. Experiments were conducted in accordance with the Declaration of Helsinki (2008 revised edition) and were approved by the local ethics committee.

Experimental design

This was a randomized, single-blind, sham-controlled, parallel design study consisting of two arms, real 20-Hz rTMS over the IPL (experimental) and sham rTMS (control). Forty subjects were included based on our previous work using single daily rTMS sessions. Subjects were assigned to the real or sham group according to random number selection while ensuring 20 per group. All subjects were unaware of the stimulation protocols until the end of the study (single-blind).

Each participant received twice-daily rTMS sessions over five consecutive days, for a total of ten sessions and 16,000 pulses. A face-cued word recall task was using to test AM and the Stroop test to assess non-associative memory cognitive processing. Tasks were performed both 1 day prior to the first real or sham stimulation session (baseline assessment) and 1 day after the final session (post-rTMS assessment). Structural MRI and rs-fMRI were conducted on each subject prior to baseline testing to identify the IPL target site (Figure 1A). After each session, subjects self-reported TMS adverse events on a numeric rating scale from 0 (no side effects) to 5 (unbearable side effects).

FIGURE 1

Figure 1. Experimental design. (A) Each participant received ten real or sham rTMS sessions delivered two per day for five consecutive days. A face-cued recall (associative memory) task and non-associative Stroop task were completed 1 day prior to the first stimulation session (baseline) and 1 day after the final session. (B) The individual stimulation sites in the left IPL were identified by rs-fMRI maps of hippocampal functional connectivity. The stimulation sites for all subjects in real and sham groups are demarcated by green and yellow spheres, respectively, in standard MNI space.

Magnetic resonance images acquisition

Magnetic resonance images (MRI) were collected at the University of Science and Technology of China (Hefei, Anhui Province) using a 3.0 T scanner (Discovery 750; GE Healthcare, Milwaukee, WI, United States). Functional and structural images were acquired using the same parameters as in our previous studies (Ji et al., 2019a,b, 2021). Briefly, high spatial resolution T1-weighted anatomic images were acquired in the sagittal orientation using a three-dimensional brain-volume sequence (repetition/echo time, 8.16/3.18 ms; flip angle, 12°; field of view, 256 × 256 mm²; 256 × 256 matrix; section thickness, 1 mm, without intersection gap; voxel size, 1 × 1 × 1 mm³; 188 sections). Following structural MRI scanning, functional images (217 volumes) were acquired using a single-shot gradient-recalled echo planar imaging sequence (repetition/echo time, 2,400/30 ms; flip angle, 90°). Images of 46 transverse sections (field of view, 192 mm × 192 mm; 64 × 64 in-plane matrix; section thickness without intersection gap, 3 mm) were acquired parallel to the anteroposterior commissure line. Foam fillers and earplugs were used to minimize head motion and scanner noise during image acquisition. All participants were asked to keep their eyes close and rest without falling asleep during scanning. Scanning was performed prior to AM and Stroop testing to exclude potential carry-over effects.

Identification of stimulation locations

Cortical stimulation locations over the IPL were identified from individual resting-state functional connectivity maps with the left hippocampus as the seed region (Wang et al., 2014; Hendrikse et al., 2020; Gao et al., 2021). The hippocampal seed was set at [−24, −18, −18] in Montreal Neurological Institute (MNI) coordinates according to Wang et al. (2014). The stimulation target for each participant was defined as the strongest connectivity site within the spherical mask of the left IPL (MNI coordinates [−47, −68, 36], radius 15 mm).

Preprocessing consisted of seven steps: (1) deleting the first five functional volumes; (2) slice timing correction and realignment; (3) co-registration of structural and functional images; (4) normalization of functional images by the matrix computed in structural segmentation and normalization; (5) smoothing of functional images using a 4-mm full-width at half-maximum isotropic Gaussian kernel; (6) temporal band-pass filtering (0.01–0.1 Hz); (7) regressing out 27 nuisance signals (average white matter, cerebrospinal fluid, and whole-brain signals as well as 24 head motion parameters). All processing steps were performed using Statistical Parametric Mapping (SPM) 12 and in-house software TMStarget.^1,2

Transcranial magnetic stimulation parameters

Transcranial magnetic stimulation was delivered to the IPC using a Magstim Rapid² stimulator (Magstim Company, Whitland, United Kingdom) through a 70-mm air-cooled figure-of-eight coil under the guidance of a frameless stereotactic optical tracking neuronavigation system (Brainsight; Rogue Research, Montreal, QC, Canada) (Gao et al., 2021). Individual stimulus intensity was set according to the resting motor threshold (RMT), defined as the minimum stimulator output necessary to evoke a potential with peak-to-peak amplitude ≥50 mV from the right first dorsal interosseous (FDI) muscle in at least 5 out of 10 consecutive trials (Ji et al., 2017). For the real stimulation group, rTMS was applied with 100% of RMT at 20-Hz (2 s on, 28 s off) for 20 min (1,600 pulses/session) over the individual IPL target. A total of 10 rTMS sessions were performed over a 5-day period with 2 sessions per day (16,000 pulses in total) separated by at least 1 h.

For the sham stimulation condition, participants received the same rTMS protocol using a sham coil (Magstim Company, Whitland, United Kingdom) with the same appearance as the real coil to avoid participants identifying rTMS group allocation. This sham coil generated only sound and sensations on the scalp similar to the real coil but no current (Chen et al., 2019).

Associative memory test

We employed the computerized Chinese face-cued word recall task described by Gao et al. (2021) to assess AM. Each subject studied 15 photographs of Chinese faces presented individually on a computer screen while a common word was read aloud in standard Mandarin. Each face corresponded to a unique word and was shown for 4 s. There were four alternative versions of the test, each using a different set of faces and words, and each participant was randomly assigned two, one to complete at baseline and the other following sham or real rTMS. All the face photos were presented at 4,800 × 6,000 pixels per inch in greyscale and all words were nouns of two Chinese characters taken from the Chinese Corpus Word list, with written frequency between 500 and 3,000 (Chinese Language and Writing Network).³ Subjects were instructed to pay attention and try to remember the face-word associations. After the learning phase, subjects were given a rest of approximately 1 min, followed by face ire-presentation in a different and random order. Subjects were instructed to recall the word that accompanied each face during the learning phase, and each word response was scored as correct or incorrect (with no errors relating to pronunciation). Participants received no prompts or feedback on the correctness of their answers. The number of correctly recalled face–word pairs was recorded as the AM score. To account for inter-subject differences in baseline AM performance, the individual improvement in AM following rTMS was expressed as a percentage change relative to baseline [AM score percentage change = (post correct – baseline correct)/baseline correct × 100%] (Wang et al., 2014).

Stroop test

To prevent the participants from easily guessing the purpose of the study and to assess general non-associative cognitive processing capacity, a Stroop Color Word Test (Victoria version) adapted to local Chinese was also conducted (Lee and Chan, 2000; Lee et al., 2002; Yu and Lee, 2018). The test stimuli included images of colored dots (Part A), words unrelated to color presented in colored font (Part B), and color names presented in font colors different from the word (Part C) (e.g., the word “red” in green font). Each image consisted of 24 items in red, green, blue, or yellow presented in a 4-by-6 matrix. Each color was used six times per image, and the four colors were arranged once per row in a pseudo-random order. The participants were asked to name the colors of stimuli (font) from left to right and from top to bottom while ignoring semantic content (i.e., the correct answer for the example above is “green”). For each condition, the naming completion time (response time) and number of errors were recorded. The interference value was defined as the response time for Part B minus Part A (low interference condition), and Part C minus Part A (high interference condition).

Methodological similarities and differences from Gao et al. (2021)

Gao et al. (2021) used a within-subjects design to examine the effect of rTMS on AM. Participants received real rTMS on IPL and sham rTMS on pre-SMA targets, separated by at least 2 weeks. For each condition, a total of 5 rTMS sessions were performed over 5 consecutive days with one session per day. The rTMS involving 1,600 total pulses in one session was delivered at 100% of RMT at 20 Hz (2 s followed by 28 s of vacancy). The experimental design of the current study was modified according to Gao et al. (2021). The same rTMS parameters and face-cued word recall task were used in both studies. In contrast to Gao et al. (2021), the current study added a second session to double the rTMS dose (for a total of 16,000 pulses rather than 8,000). Additionally, we used a between-subjects design and sham stimulation was performed at 100% RMT over IPL with a sham coil in the current study.

Statistical analysis

Continuous baseline variables were compared between groups by independent samples t-test and categorical baseline variables by χ² test. Differences in AM and Stroop test performance were compared by two-way repeated measures analysis of variance (RT-ANOVA) with main factors Group (sham vs. real rTMS) and Time (baseline vs. post-rTMS), followed by post hoc Sidak’s multiple comparison tests. AM score percentage change was compared between groups using the independent samples t-test. Outliers were identified by non-linear regression using GraphPad Prism and removed from subsequent analysis.

Three separate analyses were performed using two-way ANOVA. The primary analysis included only the current data from participants receiving sham rTMS and participants receiving real rTMS (termed the Twice-daily dataset including Twice-daily(R) and Twice-daily(S) groups). The second analysis tested if the higher rTMS dose produced more prominent effects on AM by comparing the Twice-daily(R) group to the Once-daily(R) group. This Once-daily(R) group included 16 subjects from Gao et al. (2021). In the third analysis, we combined Twice-daily(R) and Once-daily(R) groups to produce a Combined(R) group and investigated the effects on AM compared to the Twice-daily(S) group.

Results

Primary analysis of the current experimental cohort

Characteristics of participants

Forty subjects were initially recruited to receive twice-daily sham or real rTMS (Twice-daily dataset) but one participant randomized to the sham group [Twice-daily(S)] dropped out for personal reasons. Thus, data from 39 subjects (23 females/16 males, mean age = 21.69, SEM = 0.37, range 18–29 years) were included in the primary analyzes. Twice-daily(R) and Twice-daily(S) groups did not differ significantly in age, gender ratio, RMT, test delay (time between the final stimulation session and post-rTMS test), baseline AM score, or baseline Stroop test score (Table 1). The averaged side effect scores are presented in Table 1, and no significant difference (p = 0.12) between the Twice-daily(R) and Twice-daily(S) groups. In general, both sham and real rTMS were well tolerated, with only slight discomfort reported by some participants. But this effect disappeared after the stimulation ended. The average (± SEM) MNI coordinate of IPL stimulation was x = −45.4 (0.92), y = −72.0 (0.83), z = 33.1 (1.01) (Figure 1B).

TABLE 1

Table 1. Characteristics of participants from twice-daily data.

Associative memory performance

The AM scores for the Twice-daily(R) and Twice-daily(S) groups are presented in Table 2. Two-way RT-ANOVA revealed a significant Group [Twice-daily(R) vs. Twice-daily(S)] × Time (baseline vs. post-rTMS) interaction (F₁,₃₇ = 5.99, p = 0.019), but no main effect of Time (F₁,₃₇ = 0.51, p = 0.48) or Group (F₁,₃₇ = 0.16, p = 0.70). Post hoc analyses using Sidak’s multiple comparison test indicated that AM score was not increased significantly after real stimulation (t = 2.26, p = 0.06; Figure 2A) or sham stimulation (t = 1.21, p = 0.41; Figure 2A) compared to baseline. Further, AM score change normalized to baseline (percentage change) did not differ between Twice-daily(R) and Twice-daily(S) groups [34.7% (SEM = 15.2) vs. 3.0% (SEM = 14.8); P = 0.16].

TABLE 2

Table 2. Associative memory performance.

FIGURE 2

Figure 2. Effects of rTMS on associative memory performance. (A) Mean percentage change in AM score from baseline for Twice-daily(R) and Twice-daily(S) groups. The interaction effect was significant, but post hoc analyses indicated no significant increase following real or sham stimulation. (B) Mean percentage change in AM score from baseline for Twice-daily(R) and Once-daily(R) groups. There was no significant Group × Time interaction. (C) Mean percentage change in AM score for Combined(R) and Twice-daily(S) groups. A significant Group × Time interaction was found, and post hoc analyses indicated that real rTMS but not sham rTMS significantly improved AM score compared to baseline. AM score Percentage-change = (post correct – baseline correct)/baseline correct × 100%. Error bars indicate SEM, ^**P < 0.01.

Stroop test performance

Two measures of Stroop test performance, response time and error rate, are presented in Supplementary Table 1. ANOVA indicated no significant main effects of Group [Twice-daily(R) vs. Twice-daily(S)] and Time, and no significant Group [Twice-daily(R) vs. Twice-daily(S)] × Time (baseline vs. post-rTMS) interaction (see details in the Supplementary material).

Twice-daily versus once-daily repetitive transcranial magnetic stimulation

Characteristics of participants

There were no significant differences in age, gender ratio, RMT, AM scores, and Stroop tests between Twice-daily(R) and Once-daily(R) groups, while test delay was slightly but significantly lower in the Twice-daily(R) group (Table 3).

TABLE 3

Table 3. Characteristics of participants from Twice-daily(R) and Once-daily(R).

Associative memory performance

Analysis of variance revealed no significant Group [Twice-daily(R) and Once-daily(R)] × Time (baseline and post-rTMS) interaction (F₁,₃₄ = 0.02, p = 0.89) (Table 2) suggesting no difference in effect stability between Twice-daily(R) and Once-daily(R) rTMS. There was a significant main effect of Time (F₁,₃₄ = 11.96, p = 0.002) but not Group (F₁,₃₄ = 0.05, p = 0.82). Average percentage change in AM score did not differ significantly between Twice-daily(R) and Once-daily(R) groups [34.7% (SEM = 15.2) vs. 28.9% (SEM = 10.2); p = 0.80] (Figure 2B).

Stroop test performance

The response times and error rates for the Stroop tests from Once-daily(R) and Twice-daily(R) groups are presented in Supplementary Table 1. ANOVA revealed no significant interaction of Group [Twice-daily(R) vs. Once-daily(R)] × Time (baseline vs. post-rTMS). See details in the Supplementary material.

Combined analysis of once-daily and twice-daily repetitive transcranial magnetic stimulation

Characteristics of participants

There were no significant differences in any of the aforementioned baseline characteristics between Combined(R) and Twice-daily(S) groups (Table 4).

TABLE 4

Table 4. Characteristics of participants from Combined(R) and Twice-daily(S).

Associative memory performance

Associative memory scores for the Combined(R) and Twice-daily(S) groups are presented in Table 2. ANOVA revealed a significant Group [Combined(R) vs. Twice-daily(S)] × Time (baseline vs. post-rTMS) interaction effect on AM change (F₁,₅₃ = 9.08, p = 0.004) but no main effect of Time (F₁,₅₃ = 0.86, p = 0.36) or Group (F₁,₅₃ = 0.29, p = 0.59). Further, post hoc analyses using Sidak’s multiple comparison tests indicated that real stimulation significantly enhanced AM score compared to baseline (t = 3.35, p = 0.003) (Figure 2C) while sham stimulation did not (t = 1.29, p = 0.37) (Figure 2C). In addition, the average percentage change in AM score was not significantly higher in the Combined(R) group than the Twice-daily(S) group [32.11% (SEM = 9.49) vs. 3.0% (SEM = 14.8); p = 0.09] (Figure 2C).

Stroop test performance

The Stroop test response times and error rates for Combined(R) and Twice-daily(S) groups are presented in Supplementary Table 1. ANOVA indicated no significant Group [Combined(R) vs. Twice-daily(S)] × Time (baseline vs. post-rTMS) interaction. See details in the Supplementary material.

Discussion

In the present study, we found that the twice-daily rTMS enhanced AM significantly better than the placebo effect. However, the improvement in the twice-daily real stimulation group was not significant after multiple comparisons correction. Furthermore, the twice-daily protocol was not superior to the once-daily sessions previously reported by Gao et al. (2021). When combining the twice- and once-daily rTMS data to enlarge sample size, the group by time interaction effect showed better AM improvement in the real than sham group, and the post hoc analysis in real group survived the multiple comparisons correction.

Wang et al. (2014) first reported that once-daily rTMS session targeting the hippocampal-cortical network can significantly improve AM, but several subsequent studies yielded inconsistent findings. We speculated that doubling the number of stimuli by delivery twice-daily rTMS sessions could induce significant AM improvement as has been shown for other clinical effects of rTMS, including relief of depressive symptoms (Cole et al., 2020). Twice-daily rTMS sessions did enhance AM task performance to a greater extent than sham rTMS (34.7 vs. 3.0%). But the difference from baseline was not significant by post hoc multiple comparison tests for the twice-daily real stimulation group. As small sample size reduces statistical power (Button et al., 2013), we combined the Twice-daily dataset and Once-daily dataset from our previous experiment (Gao et al., 2021) to enlarge the sample size (N = 36), and post hoc multiple comparison test analyses showed significant AM improvement. An insufficient sample size could contribute to variability and affect the reliability of non-invasive brain stimulation studies (Guerra et al., 2020a,b). Thus, the required sample size should be calculated with great care in future TMS studies. We recommend that future studies examining multi-day rTMS-induced effects on memory include more participants in the real rTMS group.

Increasing the number of rTMS sessions per day and number of pulses per day has been reported to have superior antidepressant and altering cortical excitability efficacy (Nettekoven et al., 2014; Cole et al., 2020). Thus, increasing the session number was expected to enhance AM improvement from rTMS on the IPL. However, improvements twice-daily and once-daily rTMS were roughly equal (34.7 vs. 28.9%). An alternative strategy to increase rTMS dose is to use multiple targets, as several studies have demonstrated that other cortical nodes in the hippocampal-cortical network, such as dorsolateral prefrontal cortex (DLPFC), contribute to AM (Blumenfeld and Ranganath, 2006; Bilek et al., 2013). Future studies should consider combined DLPFC- and IPL-targeted rTMS.

This study has several limitations, most notably the small sample size. Although combining datasets yielded significant AM improvement, the two studies differed in several respects (e.g., experimental design, stimulus doses). Future large-sample prospective studies are warranted to document the reliability and duration of this AM-enhancing effect. Second, the RMT was measured only once rather than prior to each session, and a previous study reported that RMT varied significantly across days among subjects receiving rTMS (Cotovio et al., 2021).

Conclusion

Possibly due to the small sample size, our prospective experiment did not find significant rTMS effect on memory. But this effect may become significant if more participants could be recruited as revealed by our retrospective analysis.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Anhui Medical University. The patients/participants provided their written informed consent to participate in this study.

Author contributions

QH, G-JJ, and KW: conceptualization. QH, YZ, QL, XG, RD, YW, QZ, TZ, LZ, and JS: data acquisition. QH and G-JJ: methodology and formal analysis. QH: writing – original draft. G-JJ and KW: writing – review and editing and funding acquisition. All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Funding

This study was funded by the National Natural Science Foundation of China, Grant/Award Numbers: 31970979, 82090034, 81971689, 32071054, and 82001429; the Scientific Research Fund of Anhui Medical University: 2019xkj199; and the Collaborative Innovation Center of Neuropsychiatric Disorders and Mental Health of Anhui Province: 2020xkjT057.

Acknowledgments

We thank the participants for taking part in this study. We also thank the Information Science Laboratory Center of USTC for measurement services.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnhum.2022.973298/full#supplementary-material

Footnotes

1. ^ https://www.fil.ion.ucl.ac.uk/spm/
2. ^ https://github.com/jigongjun/Neuroimaging-and-Neuromodulation
3. ^ http://www.china-language.edu.cn/

References

Bilek, E., Schäfer, A., Ochs, E., Esslinger, C., Zangl, M., Plichta, M. M., et al. (2013). Application of high-frequency repetitive transcranial magnetic stimulation to the DLPFC alters human prefrontal-hippocampal functional interaction. J. Neurosci. 33, 7050–7056. doi: 10.1523/JNEUROSCI.3081-12.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Blumenfeld, R. S., and Ranganath, C. (2006). Dorsolateral prefrontal cortex promotes long-term memory formation through its role in working memory organization. J. Neurosci. 26, 916–925. doi: 10.1523/JNEUROSCI.2353-05.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., et al. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365–376. doi: 10.1038/nrn3475

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, X., Ji, G. J., Zhu, C., Bai, X., Wang, L., He, K., et al. (2019). Neural correlates of auditory verbal hallucinations in schizophrenia and the therapeutic response to theta-burst transcranial magnetic stimulation. Schizophr. Bull. 45, 474–483. doi: 10.1093/schbul/sby054

PubMed Abstract | CrossRef Full Text | Google Scholar

Cole, E. J., Stimpson, K. H., Bentzley, B. S., Gulser, M., Cherian, K., Tischler, C., et al. (2020). Stanford accelerated intelligent neuromodulation therapy for treatment-resistant depression. Am. J. Psychiatry 177, 716–726. doi: 10.1176/appi.ajp.2019.19070720

PubMed Abstract | CrossRef Full Text | Google Scholar

Cotovio, G., Oliveira-Maia, A. J., Paul, C., Faro Viana, F., Rodrigues, da Silva, D., et al. (2021). Day-to-day variability in motor threshold during rTMS treatment for depression: Clinical implications. Brain Stimul. 14, 1118–1125. doi: 10.1016/j.brs.2021.07.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Fox, M. D., Halko, M. A., Eldaief, M. C., and Pascual-Leone, A. (2012). Measuring and manipulating brain connectivity with resting state functional connectivity magnetic resonance imaging (fcMRI) and transcranial magnetic stimulation (TMS). Neuroimage 62, 2232–2243. doi: 10.1016/j.neuroimage.2012.03.035

PubMed Abstract | CrossRef Full Text | Google Scholar

Freedberg, M., Reeves, J. A., Toader, A. C., Hermiller, M. S., Voss, J. L., and Wassermann, E. M. (2019). Persistent enhancement of hippocampal network connectivity by parietal rTMS is reproducible. eNeuro 6. doi: 10.1523/ENEURO.0129-19.2019

PubMed Abstract | CrossRef Full Text | Google Scholar

Gao, X., Hua, Q., Du, R., Sun, J., Hu, T., Yang, J., et al. (2021). Associative memory improvement after 5 days of magnetic stimulation: A replication experiment with active controls. Brain Res. 1765:147510. doi: 10.1016/j.brainres.2021.147510

PubMed Abstract | CrossRef Full Text | Google Scholar

Guerra, A., López-Alonso, V., Cheeran, B., and Suppa, A. (2020a). Solutions for managing variability in non-invasive brain stimulation studies. Neurosci. Lett. 719:133332. doi: 10.1016/j.neulet.2017.12.060

PubMed Abstract | CrossRef Full Text | Google Scholar

Guerra, A., López-Alonso, V., Cheeran, B., and Suppa, A. (2020b). Variability in non-invasive brain stimulation studies: Reasons and results. Neurosci. Lett. 719:133330. doi: 10.1016/j.neulet.2017.12.058

PubMed Abstract | CrossRef Full Text | Google Scholar

Hendrikse, J., Coxon, J. P., Thompson, S., Suo, C., Fornito, A., Yücel, M., et al. (2020). Multi-day rTMS exerts site-specific effects on functional connectivity but does not influence associative memory performance. Cortex 132, 423–440. doi: 10.1016/j.cortex.2020.08.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Hermiller, M. S., Karp, E., Nilakantan, A. S., and Voss, J. L. (2019). Episodic memory improvements due to noninvasive stimulation targeting the cortical-hippocampal network: A replication and extension experiment. Brain Behav. 9:e01393. doi: 10.1002/brb3.1393

PubMed Abstract | CrossRef Full Text | Google Scholar

Héroux, M. E., Taylor, J. L., and Gandevia, S. C. (2015). The use and abuse of transcranial magnetic stimulation to modulate corticospinal excitability in humans. PLoS One 10:e0144151. doi: 10.1371/journal.pone.0144151

PubMed Abstract | CrossRef Full Text | Google Scholar

Ji, G. J., Liu, T., Li, Y., Liu, P., Sun, J., Chen, X., et al. (2021). Structural correlates underlying accelerated magnetic stimulation in Parkinson’s disease. Hum. Brain Mapp. 42, 1670–1681. doi: 10.1002/hbm.25319

PubMed Abstract | CrossRef Full Text | Google Scholar

Ji, G. J., Ren, C., Li, Y., Sun, J., Liu, T., Gao, Y., et al. (2019a). Regional and network properties of white matter function in Parkinson’s disease. Hum. Brain Mapp. 40, 1253–1263. doi: 10.1002/hbm.24444

PubMed Abstract | CrossRef Full Text | Google Scholar

Ji, G. J., Wei, J. J., Liu, T., Li, D., Zhu, C., Yu, F., et al. (2019b). Aftereffect and reproducibility of three excitatory repetitive tms protocols for a response inhibition task. Front. Neurosci. 13:1155. doi: 10.3389/fnins.2019.01155

PubMed Abstract | CrossRef Full Text | Google Scholar

Ji, G. J., Yu, F., Liao, W., and Wang, K. (2017). Dynamic aftereffects in supplementary motor network following inhibitory transcranial magnetic stimulation protocols. Neuroimage 149, 285–294. doi: 10.1016/j.neuroimage.2017.01.035

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, T., Yuen, K., and Chan, C. (2002). Normative data for neuropsychological measures of fluency, attention, and memory measures for Hong Kong Chinese. J. Clin. Exp. Neuropsychol. 24, 615–632. doi: 10.1076/jcen.24.5.615.1001

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, T. M., and Chan, C. C. (2000). Stroop interference in Chinese and English. J. Clin. Exp. Neuropsychol. 22, 465–471. doi: 10.1076/1380-3395(200008)22:4;1-0;FT465

CrossRef Full Text | Google Scholar

López-Alonso, V., Cheeran, B., Río-Rodríguez, D., and Fernández-Del-Olmo, M. (2014). Inter-individual variability in response to non-invasive brain stimulation paradigms. Brain Stimul. 7, 372–380. doi: 10.1016/j.brs.2014.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Nettekoven, C., Volz, L. J., Kutscha, M., Pool, E. M., Rehme, A. K., Eickhoff, S. B., et al. (2014). Dose-dependent effects of theta burst rTMS on cortical excitability and resting-state connectivity of the human motor system. J. Neurosci. 34, 6849–6859. doi: 10.1523/JNEUROSCI.4993-13.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossi, S., Hallett, M., Rossini, P. M., and Pascual-Leone, A. (2009). Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research. Clin. Neurophysiol. 120, 2008–2039. doi: 10.1016/j.clinph.2009.08.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, J. X., Rogers, L. M., Gross, E. Z., Ryals, A. J., Dokucu, M. E., Brandstatt, K. L., et al. (2014). Targeted enhancement of cortical-hippocampal brain networks and associative memory. Science 345, 1054–1057. doi: 10.1126/science.1252900

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, J. X., and Voss, J. L. (2015). Long-lasting enhancements of memory and hippocampal-cortical functional connectivity following multiple-day targeted noninvasive stimulation. Hippocampus 25, 877–883. doi: 10.1002/hipo.22416

PubMed Abstract | CrossRef Full Text | Google Scholar

Yu, J., and Lee, T. M. C. (2018). Profiles of cognitive impairments in an older age community sample: A latent class analysis. Neuropsychology 32, 102–109. doi: 10.1037/neu0000391

PubMed Abstract | CrossRef Full Text | Google Scholar