Test–Retest Reliability of the Effects of Continuous Theta-Burst Stimulation

Objectives: The utility of continuous theta-burst stimulation (cTBS) as index of cortical plasticity is limited by inadequate characterization of its test–retest reliability. We thus evaluated the reliability of cTBS aftereffects, and explored the roles of age and common single-nucleotide polymorphisms in the brain-derived neurotrophic factor (BDNF) and apolipoprotein E (APOE) genes.

Methods: Twenty-eight healthy adults (age range 21–65) underwent two identical cTBS sessions (median interval = 9.5 days) targeting the motor cortex. Intraclass correlation coefficients (ICCs) of the log-transformed, baseline-corrected amplitude of motor evoked potentials (ΔMEP) at 5–60 min post-cTBS (T5–T60) were calculated. Adjusted effect sizes for cTBS aftereffects were then calculated by taking into account the reliability of each cTBS measure.

Results: ΔMEP at T50 was the most-reliable cTBS measure in the whole sample (ICC = 0.53). Area under-the-curve (AUC) of ΔMEPs was most reliable when calculated over the full 60 min post-cTBS (ICC = 0.40). cTBS measures were substantially more reliable in younger participants (< 35 years) and in those with BDNF Val66Val and APOE ε4– genotypes.

Conclusion: cTBS aftereffects are most reliable when assessed 50 min post-cTBS, or when cumulative ΔMEP measures are calculated over 30–60 min post-cTBS. Reliability of cTBS aftereffects is influenced by age, and BDNF and APOE polymorphisms. Reliability coefficients are used to adjust effect-size calculations for interpretation and planning of cTBS studies.

Introduction

Transcranial magnetic stimulation (TMS) is a method for focal non-invasive stimulation of the brain through electromagnetic induction (Barker et al., 1985). Application of TMS within the recommended guidelines (Rossi et al., 2009; Rossini et al., 2015) is a safe means of triggering or modulating neural activity in a given brain region or network (Pascual-Leone et al., 2011; Fox et al., 2012; Valero-Cabré et al., 2017). A form of repetitive TMS (rTMS) known as continuous theta-burst stimulation (cTBS) consists of 50 Hz bursts of three TMS pulses repeated at 5 Hz for a total of 600 pulses over 40 s (Huang et al., 2005). The average amplitude of motor evoked potentials (MEPs) induced by single TMS pulses is reduced by approximately 25% for up to 50 min following cTBS of the primary motor cortex (M1) (Wischnewski and Schutter, 2015). This neuromodulatory effect is thought to involve mechanisms similar to long-term depression (LTD) (Pascual-Leone et al., 1994; Huang et al., 2005; Hallett, 2007). Therefore, the pattern of cTBS-induced changes in MEPs provides a neurophysiologic index of the mechanism of cortical plasticity (Pascual-Leone et al., 2005, 2011; Oberman et al., 2010, 2012, 2014, 2016; Suppa et al., 2016).

The neuromodulatory effect of cTBS applied to M1 or other brain regions has been investigated for its potential as a neurophysiological biomarker and a therapeutic intervention in several neurological and psychiatric disorders (Koch et al., 2009, 2012; Eberle et al., 2010; McClintock et al., 2011; Cazzoli et al., 2012; Oberman et al., 2012, 2014; Di Lazzaro et al., 2013, 2016; Mori et al., 2013; Cantone et al., 2014; Chuang et al., 2014; Forogh et al., 2014; Li et al., 2014; Suppa et al., 2014; Carrette et al., 2016). Despite its growing popularity, however, cTBS responses show large inter-individual (Hamada et al., 2013; Goldsworthy et al., 2014; López-Alonso et al., 2014; Vallence et al., 2015; Guerra et al., 2017; Heidegger et al., 2017; Hordacre et al., 2017; Jannati et al., 2017) and intra-individual variability (Vernet et al., 2014; Vallence et al., 2015) that can limit the utility of cTBS for assessing brain plasticity in clinical populations.

Only two published studies have assessed the reproducibility of cTBS aftereffects (Vernet et al., 2014; Vallence et al., 2015). The first (Vernet et al., 2014) used a relatively small sample size (n = 10) and did not report reliability coefficients of cTBS aftereffects, which can be compared with the reliability coefficients of other TMS measures (Carroll et al., 2001; Kimiskidis et al., 2004; Christie et al., 2007; Livingston and Ingersoll, 2008; Bastani and Jaberzadeh, 2012; Ngomo et al., 2012; Hinder et al., 2014; Liu and Au-Yeung, 2014; Sankarasubramanian et al., 2015; Schambra et al., 2015; Hermsen et al., 2016; Fried et al., 2017). The second study (Vallence et al., 2015) used an input–output curve approach that allowed assessment of cTBS aftereffects elicited over a range of stimulation intensities, but at the cost of fewer time-points. Specifically, assessments were only performed at 0, 15, and 30 min post-cTBS. This excluded the earliest time points, i.e., 5 and 10 min post-cTBS, which typically exhibit the maximal cTBS effects (Wischnewski and Schutter, 2015) and later time-points, i.e., 40–60 min post-cTBS, which capture the longer-lasting TBS effects and have been found to be useful in differentiating clinical populations such as individuals with Alzheimer’s disease (Freitas et al., 2011), autism spectrum disorder (Oberman et al., 2012), diabetes (Fried et al., 2016), and schizophrenia (McClintock et al., 2011) from healthy individuals. For comparison, at least three studies have assessed the reliability of intermittent theta-burst stimulation (iTBS) aftereffects (Hinder et al., 2014; Fried et al., 2017; Schilberg et al., 2017).

Full characterization of the test–retest reliability of cTBS aftereffects is essential to properly interpret results and plan for future studies. We thus aimed to address this need by systematically assessing the test–retest reliability of cTBS aftereffects in 5- or 10-min intervals for 60 min post-cTBS in a sizeable sample of healthy adults. We also calculated adjusted effect sizes for cTBS aftereffects by taking into account the reliability (or lack thereof) of each cTBS measure (Friedman, 1968; Wright, 2014; Fried et al., 2017). In addition, we explored the effects of age group on the reproducibility of cTBS aftereffects, as well as of single-nucleotide polymorphisms (SNPs) in brain-derived neurotrophic factor (BDNF) and apolipoprotein E (APOE) genes, which have been found to influence neuroplasticity (White et al., 2001; Cheeran et al., 2008; Nichol et al., 2009; Antal et al., 2010; Peña-Gomez et al., 2012; Lee et al., 2013; Chang et al., 2014; Di Lazzaro et al., 2015; Jannati et al., 2017). Our results can improve the utility of cTBS as a neurophysiologic index of cortical plasticity in neurological and psychiatric disorders, help elucidate the sources of intra-individual variability in cTBS responses, and ensure adequate sample size and power in future cTBS studies in clinical populations.

Materials and Methods

Participants

Twenty-eight healthy adults (25 males, age range: 21–65) participated in the study, which was approved by the local Institutional Review Board in accordance with the Declaration of Helsinki. All participants provided written informed consent prior to enrollment and received monetary compensation upon completion. None of the participants had any TMS contraindication (Rossi et al., 2009), and all had normal physical and neurological examinations. Individual and group-level demographics are presented in Tables 1 and 2, respectively.

TABLE 1

Table 1. Participants’ demographics, neuropsychological measures, single-nucleotide polymorphisms, and inter-visit measures.

TABLE 2

Table 2. Participants’ demographics, single-nucleotide polymorphisms, neuropsychological results, and neurophysiological measures for the total sample and for the age and genetic subgroups.

Neuropsychological Testing

Mini-Mental State Examination (Folstein et al., 1975; Crum et al., 1993) and the Abbreviated Battery of Stanford–Binet IV intelligence scale (Thorndike et al., 1986), including Verbal Knowledge and Non-Verbal Fluid Reasoning subscores, were completed.

Genetic Analyses

Saliva samples from 22 participants were assessed for BDNF Val66Met polymorphism and the presence of APOE ε4 allele, as reported previously (Jannati et al., 2017). Aliquot (700 μL) extraction of genomic DNA was performed on saliva samples collected using the Oragene Discover OGR-250 Kit (DNA Genotek Inc., Ottawa, ON, Canada). DNA was extracted from samples using standard methodology and the prepIT L2P reagent (DNA Genotek Inc., 2015). The rs6265 SNP of the BDNF gene, and the rs429358 and the rs7412 SNPs of the APOE gene were analyzed using a TaqMan single-tube genotyping assay, which uses polymerase chain reaction (PCR) amplification and a pair of fluorescent dye detectors that target the SNP. During PCR, the polymerase released the fluorescent probe into solution where it was detected using endpoint analysis in an 7900HT Real-Time instrument from Applied Biosystems, Inc. (Foster City, CA, United States).

Transcranial Magnetic Stimulation

Two identical TMS visits (7–33 days apart; median interval = 9.5 days) were conducted. The starting times of the two visits were 0–254 min apart (interquartile range = 1–103 min; median = 12 min). The inter-visit intervals and starting-time differences for individual subjects are presented in Table 1.

All TMS procedures followed the recommended guidelines endorsed by the International Federation of Clinical Neurophysiology (Rossi et al., 2009; Rossini et al., 2015). Participants were seated in a comfortable chair with the right arm and hand in a natural pronated ∼90° angle on a pillow in front of them. They were instructed to keep their right hand as still and relaxed as possible throughout the experiment. They were also monitored for drowsiness and were asked to keep their eyes open during the TMS application. Single TMS pulses and cTBS were applied to the left primary motor cortex (M1) at 120% of individual resting motor threshold (RMT) and 80% of active motor threshold (AMT), respectively, as biphasic pulses with an antero-posterior–postero-anterior (AP-PA) induced current direction using a MagPro X100 stimulator and a MC-B70 Butterfly Coil (outer diameter: 97 mm; MagPro, MagVenture A/S, Farum, Denmark). The coil was held tangentially to the participant’s head surface, with the handle pointing occipitally and positioned at 45° relative to the mid-sagittal axis of the participant’s head. The optimal spot for the maximal responses of the right first dorsal interosseous (FDI) muscle (“motor hotspot”) was localized. A Polaris infrared-optical tracking system (Northern Digital Inc., Waterloo, ON, Canada) and a Brainsight TMS neuronavigation system (Rogue Research Inc., Montreal, QC, Canada) with a brain MRI template (for 21 participants) or the participant’s brain MRI (for the remaining 7 participants) was used to ensure consistent targeting throughout the experiment.

Surface electromyogram (EMG) was recorded from the right FDI with a PowerLab 4/25 data-acquisition device and LabChart 8 software (AD Instruments, Colorado Springs, CO, United States). Electrodes were placed over the FDI belly (negative) and the first interphalangeal joint of the second finger (positive). The ground electrode was placed over the ipsilateral ulnar styloid process. The TMS system delivered triggered pulses that synchronized the TMS and EMG systems. EMG signal was digitized at 1 kHz for 500 ms following each stimulus trigger and 100 ms pre-trigger, amplified with a range of ±10 mV (band-pass filter 0.3–1000 Hz).

Each TMS session began by localizing the motor hotspot for FDI and assessment of the RMT, defined as the lowest intensity of stimulation that elicited MEPs ≥ 50 μV in at least five of ten pulses in the relaxed right FDI. To assess pre-cTBS cortico-motor reactivity, three blocks of 30 single TMS pulses were applied to M1, with a 5–10 min inter-block interval and at a random 4–6 s inter-pulse interval. In each block, individual MEPs > 2.5 SD from the mean were excluded. Baseline MEP amplitude was calculated as the average of the peak-to-peak amplitude of MEPs in the three blocks. The AMT was then assessed as the lowest intensity that elicited MEPs ≥ 200 μV in at least five of ten pulses with the FDI slightly contracted. After a 5-min break, during which participants were instructed to maintain hand relaxation to control the effects of voluntary hand movements on cTBS responses (Iezzi et al., 2008), cTBS was applied as 200 bursts of three pulses at 50 Hz, repeated at 200-ms intervals for 40 s (for a total of 600 pulses). Cortico-motor reactivity was reassessed at 5, 10, 15, 20, 30, 40, 50, and 60 min post-cTBS (T5–T60).

Statistical Analyses

Stata software version 13.1 (StataCorp, College Station, TX, United States) and MATLAB and Statistics and Machine Learning Toolbox R2016b (The MathWorks, Natick, MA, United States) were used for data analysis. Data from each TMS visit included: (a) RMT and AMT, expressed as percentage of maximum stimulator output; (b) baseline MEP amplitude, calculated as the average of baseline MEP amplitude in 3 blocks of 30 single TMS pulses; and (c) percent change in the average amplitude of 30 MEPs at T5–T60 relative to baseline (%Δ) for each participant.

The Shapiro–Wilk test found significant deviations in MEP values from normal distribution; thus, natural log-transformed, baseline-corrected MEP amplitude at each post-cTBS time point (ΔMEP) was averaged over all participants separately for each visit. The following measures were also calculated: absolute MEP modulation at T5–T60 (|ΔMEP|), maximum suppression and maximum modulation of MEPs during 60 min post-cTBS, area under-the-curve (AUC) and the absolute AUC value (|AUC|) of ΔMEPs over T0–T10, …, and T0–T60 intervals. Cumulative AUC and |AUC| measures up to each time-point were calculated as the summed products of the average ΔMEP and the average |ΔMEP|, respectively, across each two consecutive time-points and the time in minutes between them.

Grand-average values for all cTBS measures were calculated separately for both visits and were compared against zero using one-sample t-tests. Visit-B minus visit-A difference (Δ_B-A) and |Δ_B-A| were calculated for each neurophysiological measure (Table 3). All analyses were two-tailed, and the α level was set to 0.05. When explicitly noted, false discovery rate (FDR) was used to adjust p-values for multiple testing (Benjamini and Hochberg, 1995; Benjamini and Yekutieli, 2001).

TABLE 3

Table 3. Neurophysiological measures (mean ± SD) and their test-retest reliability for the whole sample (N = 28).

ΔMEPs at T10 and T40 were previously found to be the best predictors of inter-individual variability in cTBS aftereffects in visit-A (Jannati et al., 2017). Thus, to assess the effect of potential covariates on the intra-individual variability of cTBS aftereffects at T10 and T40, we conducted linear mixed-effects (LME) regression analyses with ΔMEPs at T10 or T40 as dependent variable, Visit (visit-A vs. visit-B) as a within-subject factor, and potential covariates including RMT, AMT, baseline MEP amplitude, number of days between the two visits (Inter-visit Interval), and the absolute inter-visit difference in starting time (in minutes) (Time Difference) as between-subject factors. Based on previous studies that found in many situations a regression model is likely to be reliable when the number of candidate predictors is smaller than one-tenth of the number of subjects (Harrell, 2015) (p. 72), up to three between-subjects predictors were considered for simultaneous inclusion in any regression model.

To assess test–retest reliability, intraclass correlation coefficients (ICCs) (Portney and Watkins, 2009) were calculated in the form of absolute agreement between the two visits for all neurophysiological measures. ICCs were calculated using a two-way mixed-effects model, with fixed column (C) effects and random row (R) effects (McGraw and Wong, 1996):

$$\mathrm{ICC}(\mathrm{A},\mathrm{\hspace{0.17em}}1)=\frac{M{S}_{\mathrm{R}}-M{S}_E}{M{S}_{\mathrm{R}}+(k-1)M{S}_{\mathrm{E}}+\frac{k}{n}(M{S}_{\mathrm{C}}-M{S}_{\mathrm{E}})}$$

where ICC(A,1) represents the degree of absolute agreement of measurements made under the two fixed levels of the column factor. k = the number of raters/measurements per subject; MS_R = mean square for rows (representing the individual subjects); MS_E = mean square error; MS_C = mean square for columns (representing the two visits); n = the number of subjects.

Using this formula, ICC = 1 indicates maximum reliability and ICC ≤ 0 indicates no reliability [in the case that the within-group variance is equal to or higher than the between-groups variance (Kenny et al., 2002)]. ICC values were interpreted as follows (Portney and Watkins, 2009): (i) ICC < 0.25: very low to no reliability; (ii) 0.25 ≤ ICC < 0.50: low reliability; (iii) 0.50 ≤ ICC < 0.75: moderate reliability; and (iv) ICC ≥ 0.75: high reliability. ICC values were statistically compared using two-way mixed-effects F statistics (McGraw and Wong, 1996, Table 8). The effects on the ICCs of covariates that had a significant effect on ΔMEPs were assessed by including the covariate in the corresponding mixed-effects regression model and re-calculating the residual intraclass correlation.

Lack of reliability of a measure of interest attenuates the observed effect size compared to the population parameter (Hunter and Schmidt, 1994). Following previously applied methodology (Friedman, 1968; Wright, 2014; Fried et al., 2017), we assessed how test–retest reliability (or lack thereof) of TMS measures would attenuate small, medium, and large effect sizes, i.e., Cohen’s d values of 0.2, 0.5, and 0.8 (Cohen, 1992), respectively, which assume perfect reproducibility. First, each idealized Cohen’s d is converted to an r (Cohen, 1988) (p. 23):

$$r_{\mathrm{IDEALIZED}}=\frac{d_{\mathrm{IDEALIZED}}}{\sqrt{d_{\mathrm{IDEALIZED}}^2+4}}$$

This idealized r is then adjusted for unreliability using the ICC (Wright, 2014):

$$r_{\mathrm{ADJUSTED}}^2=r_{\mathrm{IDEALIZED}}^2\ast \sqrt{ICC}$$

Finally, the adjusted r is converted back to an adjusted d (Friedman, 1968) (p. 246):

$$d_{\mathrm{ADJUSTED}}=\frac{2\ast r_{\mathrm{ADJUSTED}}}{\sqrt{1-r_{\mathrm{ADJUSTED}}^2}}$$

Exploratory Analyses

Though unintended, the age of our participants conformed to a bimodal distribution. Thus, to explore the impact of age on our reliability measures, the total sample was subdivided into two distinct age groups with a 10-year gap and a ∼27-year difference in mean age: a Younger group with age < 35 (n = 16; range: 21–34; mean ± SD, 25.3 ± 4.3) and an Older group with age ≥ 45 (n = 12; range: 45–65; mean ± SD, 52.1 ± 6.5). To explore the effect of age on the test–retest reliability of cTBS aftereffects: (i) the ICC values of TMS measures were calculated separately for the two age groups; (ii) separate repeated-measures analyses of variance (Rm-ANOVAs) were conducted with ΔMEP at T10 or T40 as dependent variable, Age Group as a between-subjects factor, Visit as a within-subject factor, and Age Group × Visit interaction. Because the proportion of Hispanic participants was significantly higher in the Younger group than in the Older group, we assessed the effect of Ethnicity as a categorical covariate in these Rm-ANOVAs. We also re-calculated all the ICCs for the Younger group while controlling for Ethnicity.

To explore the roles of BDNF and APOE SNPs in the reliability of TMS measures, we calculated the ICC values of neurophysiological measures over the two visits separately for participants with BDNF Val/Val (Met–; n = 14) and Val/Met (Met+; n = 8) genotypes as well as for those with APOE ε2/ε3 or ε3/ε3 (ε4–; n = 12) and APOE ε2/ε4 or ε3/ε4 (ε4+; n = 10) genotypes.

Because BDNF was previously found to influence the cTBS aftereffect at T10 (Jannati et al., 2017), we assessed the cTBS aftereffects separately for BDNF Met– and Met+ participants in each visit and conducted a Rm-ANOVA with ΔMEP at T10 as dependent variable, BDNF Status (Met– vs. Met+) as a between-subjects factor, Visit (visit-A vs. visit-B) as a within-subject factor, and BDNF Status × Visit interaction. Further, we assessed the effect of BDNF Status as a covariate in the LME regression analyses at T10.

Results

Demographics, neuropsychological measures, inter-visit interval, starting times of the two visits, and inter-visit differences in starting time for individual participants are presented in Table 1. Statistical comparisons of these measures between the two age groups are presented in Table 2.

Genetic Analyses

Available BDNF and APOE results and comparisons of all available measures between BDNF/APOE subgroups are presented in Tables 1 and 2, respectively.

Among 22 participants with available DNA results, the frequencies of BDNF Val/Val and Val/Met genotypes were 0.64 and 0.36, respectively, while the frequencies of APOE ε2/ε3, ε3/ε3, and ε3/ε4 genotypes were 0.14, 0.41, and 0.46, respectively. BDNF and APOE subgroups were comparable in all available measures (Table 2).

Baseline Neurophysiological Measures

The RMT, AMT, and baseline MEP amplitude in each visit and their inter-visit differences are summarized in Tables 2 and 3, respectively.

There were no significant differences in any of the baseline neurophysiological measures in either visit between age or genetic subgroups (Table 2). There was also no significant difference between the two visits in any of the baseline neurophysiological measures for the whole sample (p’s > 0.14), for each age group (p’s > 0.17), or for each BDNF (p’s > 0.08) or APOE (p’s > 0.28) subgroup.

The LME regression analyses of ΔMEP at T10 found a significant, negative effect of Time Difference in all models (p’s < 0.025), but no significant effect of any of the baseline neurophysiological measures or Visit (p’s > 0.21). LME regression analyses of ΔMEP at no other time point found a significant effect of Time Difference (p’s > 0.41).

cTBS-Induced Plasticity Results

Grand-average ΔMEPs in visits A and B are shown in Figure 1. ΔMEP and |ΔMEP| values, maximum suppression and maximum modulation as well as their inter-visit differences are summarized in Table 3. Grand-average ΔMEPs did not significantly differ from zero at any time point in either visit (p’s > 0.11). There was also no significant difference in grand-average ΔMEP or |ΔMEP| between the two visits at any time point (p’s > 0.16).

FIGURE 1

Figure 1. Grand-average ΔMEPs recorded from the right FDI muscle at 5 to 60 min following cTBS of the left primary motor cortex in two identical visits. The ΔMEPs did not significantly differ from zero at any post-cTBS time point in either visit (p’s > 0.11). Error bars represent standard error of the mean. cTBS, continuous theta-burst stimulation; ΔMEP, natural log-transformed, baseline-corrected amplitudes of the motor evoked potential; FDI, first dorsal interosseous.

The ΔMEPs in the Older group were significantly greater than zero at T20 in visit-A (P_FDR = 0.029), but not at any other time point in either visit (p’s > 0.18). The ΔMEPs in the Younger group did not significantly differ from zero at any time point in either visit (p’s > 0.06). The ΔMEPs in the BDNF Met– group were significantly less than zero at T10 in visit-A (P_FDR = 0.042), but not at any other time point in either visit (p’s > 0.14). The ΔMEPs in the BDNF Met+ group were not significantly different from zero at any time point in either visit (P_FDR’s > 0.05). cTBS aftereffects in both visits for the two age subgroups and the two BDNF subgroups are presented in Figures 2 and 4 respectively.

FIGURE 2

Figure 2. Average ΔMEPs recorded from the right FDI muscle at 5 to 60 min following cTBS of the left primary motor cortex in two identical visits in the Younger (A) and Older (B) groups. The ΔMEPs did not significantly differ from zero at any time point in either visit in the Younger group (p’s > 0.06). ^∗The ΔMEPs in the Older group were significantly greater than zero at T20 in visit-A (P_FDR = 0.029), but not at any other time point in either visit (p’s > 0.18). Error bars represent standard error of the mean. cTBS, continuous theta-burst stimulation; ΔMEP, natural log-transformed, baseline-corrected amplitudes of the motor evoked potential; FDI, first dorsal interosseous; FDR, false discovery rate.

The Rm-ANOVA on the ΔMEP at T10 found a significant effect of BDNF status, F(1, 20) = 8.28, p = 0.009, ${\mathrm{\eta }}_{\mathrm{p}}^{\mathrm{2}}$ = 0.29, but no significant effects of Visit or BDNF × Visit interaction (p’s > 0.10). BDNF Met-carrier status had a significant positive effect in all LME regression analyses of ΔMEP at T10 (${\widehat{B}}^{\prime }$s > 0.28, p’s < 0.027). There was no other significant effect in any of the LME models (p’s > 0.08).

To control for potential effects of gender, race/ethnicity, and handedness on cTBS-induced plasticity measures, we calculated ΔMEP’s at T5–T60 in a subgroup of White, non-Hispanic, and right-handed males (n = 11, Table 1). In this smaller, but more-homogenous subsample, ΔMEPs did not significantly differ from zero at any time point in either visit (p’s > 0.23).

Test–Retest Reliability of TMS Measures

Measures of inter-visit variability and test–retest reliability for RMT, AMT, baseline MEP amplitude and cTBS measures, as well as reliability-adjusted effect sizes for each TMS measure in the whole group are presented in Table 3.

The ICCs of baseline neurophysiological measures was not significantly different between the two age groups (p’s > 0.19; Figure 3). In contrast, ΔMEPs in the Younger group were significantly more reliable than in the Older group at T10 and T60 (P_FDR’s < 0.015), but not at other individual time points (P_FDR’s > 0.058). Similarly, |AUC| measures were significantly more reliable in the Younger group than in the Older group over T0–T20 and beyond (P_FDR’s < 0.001; Figure 3). The reliability of other cumulative ΔMEP measures was not significantly different between the two age groups (p’s > 0.058). After adjusting for Ethnicity, none of the ICCs in the Younger group crossed our pre-defined boundaries for interpretation of ICC values (see “Materials and Methods” section).

FIGURE 3

Figure 3. Test–retest reliability of baseline neurophysiological measures and post-cTBS ΔMEP measures separately in the Younger (age < 35, n = 16) and Older (age ≥ 45, n = 12) groups. |AUC| of ΔMEPs were calculated as the summed products of the average |ΔMEP| across each two consecutive time-points and the time in minutes between them over T0–T10, T0–15, …, T0–T60 intervals (marked by their end time point on the abscissa). The ICCs of the RMT, AMT, and baseline MEP amplitude were not significantly different between the two age groups (p’s > 0.19). The |AUC| of ΔMEPs were significantly more reliable in the Younger group than in the Older group over T0–T15 and beyond (P_FDR’s < 0.001). The reliability of other cumulative ΔMEP measures was not significantly different between the two age groups (p’s > 0.058). AUC, area under-the-curve; cTBS, continuous theta-burst stimulation; FDR, false discovery rate; ΔMEP, natural log-transformed, baseline-corrected MEP amplitude; ICC, intraclass correlation coefficient; MEP, motor evoked potential; T0–Tn, over the first n minutes post-cTBS.

To control for potential effects of gender, race/ethnicity, and handedness on the test–retest reliability of cTBS aftereffects, we calculated the ICC values of baseline neurophysiological measures and ΔMEP’s at T5–T60 among White, non-Hispanic, and right-handed males (n = 11, Table 1). The ICC values of RMT, AMT, and baseline MEP amplitude in this subgroup were 0.86 (p < 0.001), 0.96 (p < 0.001), and 0.75 (p = 0.003), respectively. The ICC values of ΔMEP at T5, T10, T15, T20, T30, T40, T50, and T60 in this subgroup were 0.54 (p = 0.031), –0.09 (p = 0.602), 0.18 (p = 0.297), 0.71 (p = 0.005), 0.86 (p < 0.001), 0.67 (p = 0.008), 0.79 (p = 0.001), and 0.17 (p = 0.309), respectively.

The ICCs of baseline neurophysiological measures were not significantly different between either the BDNF or the APOE subgroups (p’s > 0.16; Figure 5). In contrast, ΔMEPs were significantly more reliable in BDNF Met– participants than in BDNF Met+ participants at T20–T40 (P_FDR’s < 0.023), but not at other time points (p’s > 0.24). Maximum suppression, maximum modulation, and AUC of ΔMEPs over T0–T30 and beyond were significantly more reliable in BDNF Met– participants than in BDNF Met+ participants (P_FDR’s < 0.032; Figure 5). The reliability of other cumulative ΔMEP measures was not significantly different between the two BDNF subgroups (P_FDR > 0.21).

ΔMEPs were significantly more reliable in APOE ε4– participants than in APOE ε4+ participants at T5 and T20–T40 (P_FDR’s < 0.024), but not at other time points (P_FDR’s > 0.07). All AUC measures were significantly more reliable in APOE ε4– participants than in APOE ε4+ participants (P_FDR’s < 0.021; Figure 5). There were no significant differences in the reliability of maximum suppression or maximum modulation between the two APOE subgroups (P_FDR’s > 0.27).

Discussion

Test–retest reliability of TMS measures influences their utility as potential neurophysiologic biomarkers or targets for therapeutic intervention. As the use of plasticity-inducing rTMS protocols becomes more common, it is necessary to investigate the magnitude and sources of their inter- and intra-individual variability. While some of the factors that contribute to the inter-individual variability of these types of plasticity metrics among healthy individuals have been identified (Cheeran et al., 2008, 2009; Antal et al., 2010; Hamada et al., 2013; Goldsworthy et al., 2014; López-Alonso et al., 2014; Nettekoven et al., 2014, 2015; Vallence et al., 2015; Suppa et al., 2016; Hordacre et al., 2017; Jannati et al., 2017), few studies have assessed the intra-individual reliability of cTBS responses (Vernet et al., 2014; Vallence et al., 2015) and no study, to our knowledge, has systematically assessed the test–retest reliability of cTBS aftereffects during 60 min post-cTBS. The present study was designed to fill this gap by assessing the test–retest reliability of cTBS aftereffects at 5- or 10-minute intervals (T5–T60) and of cumulative cTBS aftereffects during 60 min post-cTBS in healthy adults. Furthermore, in order to provide guidance and reference for future studies, we calculated adjusted effect sizes that take into account the test–retest reliability of cTBS measures. Finally, we explored the influences of age group and common SNPs in BDNF and APOE genes on the reliability of cTBS aftereffects.

Overall Reliability of Baseline Neurophysiological Measures

Resting motor threshold had high test–retest reliability (Table 3), which was comparable with the ICC values reported in most previous studies (Carroll et al., 2001; Kimiskidis et al., 2004; Livingston and Ingersoll, 2008; Bastani and Jaberzadeh, 2012; Ngomo et al., 2012; Hinder et al., 2014; Schambra et al., 2015; Fried et al., 2017; Davila-Pérez et al., 2018) and somewhat higher than other studies that found RMT ICCs in the 0.75–0.80 range (Fleming et al., 2012; Liu and Au-Yeung, 2014; Sankarasubramanian et al., 2015; Hermsen et al., 2016). The AMT also had high test–retest reliability (Table 3), which was comparable with the results of previous studies (Ngomo et al., 2012; Hinder et al., 2014; Fried et al., 2017).

Baseline MEP amplitude had moderate test–retest reliability (Table 3). The ICC of baseline MEP amplitude found in the present study (ICC = 0.70) was moderate compared to the wide range of ICC values for baseline MEP amplitude (–0.16 to 0.87) reported in previous studies (Kamen, 2004; McDonnell et al., 2004; Christie et al., 2007; Fleming et al., 2012; Ngomo et al., 2012; Hinder et al., 2014; Hermsen et al., 2016; Fried et al., 2017; Davila-Pérez et al., 2018). Variability of baseline MEP amplitude was previously found to be associated with variability of TBS aftereffects (Hordacre et al., 2017; Fried et al., 2017). The moderate reliability of baseline MEP amplitude in the present study suggests that such variability was not the main cause of the low reliability of some of the cTBS measures reported here.

Overall Reliability of cTBS Aftereffects

The finding that differences between the start times of the two visits influenced the cTBS aftereffects at T10 could be due to the effect of circadian rhythm on the neuromodulatory effects of rTMS arising from changes in cortical excitability and synaptic efficiency during the day (Cohen et al., 2005). While the present results cannot definitively conclude that circadian factors influenced the intra-individual variability in plasticity at T10, future studies could attempt to control for the time of day or, perhaps even better, to individualize visits to coincide with the same relative point in each subject’s circadian cycle.

The finding that T5 had one of the lowest between-visit variabilities among post-cTBS time points is consistent with the findings of a previous study (Vernet et al., 2014). Importantly, however, the low between-visit variability of cTBS aftereffects at T5 and T50 at the group level in the present study did not translate to high test–retest reliability measures, which take into account both within-individual and between-individuals variability; while T50 was the most reliable post-cTBS time point (ICC = 0.53), T5 had very low reliability (ICC = 0.16). This pattern of results underlines the importance of calculating the ICCs of TMS measures rather than relying only on measures of inter-visit variability at the group level. Further, the low ICCs at T5 and T10 indicate that the time points expected to show maximal effects of cTBS (Wischnewski and Schutter, 2015) do not necessarily exhibit high test–retest reliability. This remained true for the cumulative measures of cTBS aftereffects over the first 20 min post-cTBS.

The very low test–retest reliability of ΔMEPs at T10 (ICC = 0.11) could be due to two factors: (1) The ΔMEPs at T10 in both visits could be the most influenced by BDNF polymorphism (Jannati et al., 2017). Consistent facilitation of MEPs, at least numerically, in BDNF Met+ participants in both visits may have resulted in higher test–retest reliability of T10 ΔMEP in that subgroup (Figure 5). (2) Despite the relatively long inter-visit interval in the present study, T10 seemed to exhibit a metaplastic-like effect similar to those reported with shorter intervals (Maeda et al., 2000; Gentner et al., 2008; Valero-Cabré et al., 2008; Oberman et al., 2016) in the overall results (Figure 1), as well in the Younger group (Figure 2A) and the BDNF Met– subgroup (Figure 4A). In all three cases, the direction of the neuromodulatory effect of cTBS at T10 was reversed, at least numerically, from inhibitory in visit-A to facilitatory in visit-B. Such reversals, when predominant at the individual level, would substantially reduce the test–retest reliability of cTBS aftereffects at T10. Although a previous iTBS study found only inter-visit intervals shorter than 7 days to be associated with metaplastic changes after iTBS in aging adults (Fried et al., 2017), the initial cTBS in the present study may have set into motion subtle changes that were still present when the second cTBS was applied. Further, it is possible that demographic, genetic, and state-dependent factors modulate the metaplastic(-like) effects of successive TBS sessions (Opie et al., 2017).

FIGURE 4

Figure 4. Average ΔMEPs recorded from the right FDI muscle at 5 to 60 min following cTBS of the left primary motor cortex in two identical visits in the BDNF Met– (A) and Met+ (B) groups. ^∗The ΔMEPs in the BDNF Met– group were significantly less than zero at T10 in visit-A (P_FDR = 0.042), but not at any other time point in either visit (p’s > 0.14). The ΔMEPs in the BDNF Met+ group were not significantly different from zero at any time point in either visit (P_FDR’s > 0.05). Error bars represent standard error of the mean. BDNF, brain-derived neurotrophic factor; BDNF Met–, Val66Val; BDNF Met+, Val66Met; cTBS, continuous theta-burst stimulation; FDR, false discovery rate; ΔMEP, natural log-transformed, baseline-corrected; MEP amplitude; MEP, motor evoked potential; Met, metionine; Val, valine.

The low test–retest reliability of several cTBS aftereffects resulted in adjustment of large- and medium effect sizes to medium and small effect sizes, respectively (Table 3). While calculating the cumulative cTBS measures improved the overall ICCs over T0–T30 and beyond (Figures 3, 5), the ICCs of the cumulative measures for the whole sample remained below 0.5. Attenuation of idealized effect sizes by this level of reproducibility indicates that detecting significant differences in cTBS responses between healthy and clinical populations may require sample sizes that are substantially larger than those used in most previous cTBS studies (Wischnewski and Schutter, 2015; Chung et al., 2016; Suppa et al., 2016), unless steps can be implemented to improve the reliability of this technique.

FIGURE 5

Figure 5. Test–retest reliability of baseline neurophysiological measures and post-cTBS ΔMEP measures separately in the BDNF Met–/Met+ and APOE ε4–/ε4+ groups. AUC of ΔMEPs were calculated as the summed products of the average ΔMEP across each two consecutive time-points and the time in minutes between them over T0–T10, T0–15, …, T0–T60 intervals (marked by their end time point on the abscissa). The ICC values of RMT, AMT, and baseline MEP amplitude were not significantly different between the two BDNF groups (p’s > 0.19) or the two APOE groups (p’s > 0.16). Maximum suppression, maximum modulation, and the AUC of ΔMEPs over T0–T30 and beyond were significantly more reliable in the BDNF Met– group than in the BDNF Met+ group (P_FDR’s < 0.032). The reliability of other cumulative ΔMEP measures was not significantly different between the two BDNF groups (P_FDR > 0.208). All the AUC measures were significantly more reliable in the APOE ε4– group than in the APOE ε4+ group (P_FDR’s < 0.021). The maximum suppression, maximum modulation were not significantly different between the two APOE groups (P_FDR’s > 0.27). AMT, active motor threshold; APOE, apolipoprotein E; APOE ε4+, ε2/ε4 or ε3/ε4 genotype; APOE ε4–, ε2/ε3 or ε3/ε3 genotype; AUC, area under-the-curve; BDNF, brain-derived neurotrophic factor; BDNF Met–, Val66Val; BDNF Met+, Val66Met; cTBS, continuous theta-burst stimulation; FDR, false discovery rate; ICC, intraclass correlation coefficient; ΔMEP, natural log-transformed, baseline-corrected MEP amplitude; MEP, motor evoked potential; Met, metionine; RMT, resting motor threshold; T0–Tn, over the first n minutes following cTBS; Val, valine.

The finding that within White, non-Hispanic, and right-handed males (n = 11), none of the ΔMEPs at any time point in either visit differed significantly from zero suggests that controlling for demographic factors such as gender, race/ethnicity, and handedness is not enough to overcome the large inter-individual variability in cTBS responses in either visit. We found, despite comparable reliability of baseline neurophysiological measures, T5 and T20–T50 ΔMEPs were substantially more reliable in this more-homogenous subgroup, suggesting that heterogeneity of these demographic factors influence the reliability of cTBS aftereffects in the whole sample. While the small sample precludes definitive conclusions about the effects of gender, handedness, and race/ethnicity, these results nonetheless appear to suggest that demographic variation plays a role in the test–retest reliability of cTBS responses.

Age and Reliability of cTBS Aftereffects

The significant Age Group × Visit interaction effect on T10 ΔMEP indicates that cTBS aftereffects at T10 in the Younger group, but not in the Older group, switched from inhibitory in visit-A to facilitatory in visit-B (Figure 2). Such reversal could be due to metaplasticity or some other state-dependent factor. The distinct patterns of cTBS aftereffects at T10 among Younger and Older groups could be due to two factors: (i) Based on animal studies that have found an age-related reduction in the efficiency of gamma-aminobutyric acid- (GABA-) mediated inhibition (Milbrandt et al., 1994; Billard et al., 1995; McQuail et al., 2012), it is possible that older participants have less-efficient GABAergic synaptic transmission, which is presumed to be involved in cTBS-induced plasticity (Stagg et al., 2009; Trippe et al., 2009). The resulting reduced inhibitory effects of cTBS, potentially combined with cumulative facilitatory effects of successive single TMS pulses (Pellicciari et al., 2016), could have resulted in facilitation of MEPs in the Older group, at least in visit-A. (ii) The finding that the Younger group, but not the Older group, showed metaplastic-like changes at T10 could be due to age-related differences in the priming effect of TBS (Opie et al., 2017), i.e., a stronger priming effect of cTBS in visit-A among younger participants.

Similarly, the finding that despite comparable ICCs of baseline neurophysiological measures in the two age groups, cTBS aftereffects were substantially less reliable in older participants (Figure 3) could be due to the age-related decrease in the efficiency of GABAergic synaptic transmission reported in animal studies (Milbrandt et al., 1994; Billard et al., 1995; McQuail et al., 2012). These results indicate that in order to retain adequate power to detect differences in cTBS measures of plasticity in future studies, it may be necessary to adjust effect sizes separately for younger and older age groups.

BDNF and APOE Polymorphisms and Reliability of cTBS Aftereffects

Despite comparable ICC values of RMT, AMT, and baseline MEP amplitude in BDNF Met– and Met+ groups, cTBS aftereffects at several time points and most cumulative measures of cTBS aftereffects were substantially more reliable in BDNF Met– participants than in Met+ participants (Figure 5). This pattern of results could be due to the following: BDNF Met carrier status is known to be associated with impaired N-Methyl-D-aspartic acid- (NMDA-)dependent LTD (Woo et al., 2005), aberrant GABAergic synaptic transmission (Abidin et al., 2008), reduced cTBS-induced inhibition of MEPs (Chung et al., 2016), and “paradoxical” cTBS-induced facilitation of MEPs in visit-A reported in our previous study (Jannati et al., 2017) and a few other studies (Gentner et al., 2008; Goldsworthy et al., 2012; Hellriegel et al., 2012; Brownjohn et al., 2014). The finding that BDNF Met+ participants showed MEP facilitation, at least numerically, at T10 in both visits (Figure 4B) supports an association between BDNF Met+ status and facilitatory response to cTBS at T10 in the present sample. The noticeably lower test–retest reliability of cTBS aftereffects in BDNF Met+ participants (Figure 5) could be due to the less-efficient cTBS-induced plasticity caused by aberrant GABAergic inhibition (Abidin et al., 2008), assumed to be involved in the LTD-like effects of cTBS (Stagg et al., 2009; Trippe et al., 2009).

Despite comparable ICC values of RMT, AMT, and baseline MEP amplitude in APOE ε4– and ε4+ participants, cTBS aftereffects at several time points and most cumulative measures of cTBS aftereffects were substantially more reliable in APOE ε4– participants than in APOE ε4+ participants (Figure 5). These results could be due to the influence of APOE ε4 on NMDA-mediated synaptic plasticity, which has been found to be involved in TBS aftereffects (Huang et al., 2007; Chen et al., 2010). These results are also consistent with the less-efficient rTMS-induced activation of brain networks in APOE ε4 carriers (Peña-Gomez et al., 2012).

The small number of participants in the BDNF and APOE subgroups study limits the generalizability of the present findings on genetic influences on the test–retest reliability of cTBS measures. Assuming that the noticeable differences in reliability of cTBS aftereffects in BDNF and APOE subgroups observed here (Figure 5) are confirmed in future studies, it would be advantageous to consider the expected proportions of BDNF and APOE subgroups and adjust effect sizes for each SNP subgroup accordingly. For example, the minor allele frequencies of rs6265 (BDNF), rs429358 (APOE), and rs7412 (APOE) SNPs in the admixed American population in the 1000 Genomes Project (Auton et al., 2015) are 0.1527, 0.1037, and 0.0476, respectively. As long as that the SNP frequencies among participants do not significantly deviate from Hardy–Weinberg equilibrium proportions (Guo and Thompson, 1992; Wigginton et al., 2005), these frequencies provide good approximations to the frequencies of minor BDNF and APOE alleles in future cTBS studies.

In comparisons of test–retest reliability of cTBS measures between age or genetic subgroups, the finding that the subgroups in each case were comparable in gender, race/ethnicity, handedness, and the reliability of baseline neurophysiological measures indicates that differences in heterogeneity of demographic factors and baseline cortical excitability did not play a major role in the observed differences in test–retest reliability of cTBS aftereffects.

In addition to considering age, genetic polymorphisms, inter-visit interval (Fried et al., 2017), the time of day (Cohen et al., 2005), and the use of neuronavigation (Julkunen et al., 2009), other factors that could improve the test–retest reliability of TMS measures include: ensuring comparable blood glucose levels and caffeine intake before and during each visit (Specterman et al., 2005; Cerqueira et al., 2006; Badawy et al., 2013), comparable amount and quality of sleep the night before each visit (Civardi et al., 2001; Kreuzer et al., 2011), comparable intensity and duration of exercise before each visit (Samii et al., 1997; Lentz and Nielsen, 2002), comparable phase of the menstrual cycle across visits (Smith et al., 1999; Hattemer et al., 2007), the use of robotic arms such as the TMS-Robot (Axilum Robotics, Schiltigheim, France), which can reduce trial-to-trial variability of MEP amplitude (Foucher et al., 2012), comparable baseline MEP amplitude across plasticity visits (Fried et al., 2017), and implementing closed-loop systems that trigger TMS pulses timed to real-time, EEG-defined indices of brain states (Zrenner et al., 2016, 2018).

Conclusion

The present study assessed the test–retest reliability of cTBS aftereffects in healthy adults. cTBS aftereffects at most individual time points had low to moderate reliability. Cumulative cTBS measures over the first 30 min and beyond were relatively more reliable. Effect sizes adjusted for reliability of cTBS aftereffects are provided to help future studies retain adequate power for comparing M1 cTBS responses between healthy and clinical populations. Those calculations resulted in adjustment of several large and medium effect sizes to medium and small effect sizes, respectively, thereby substantially increasing the estimates of the required sample size to detect a significant difference in cTBS responses between healthy and clinical populations. Exploratory analyses found cTBS aftereffects were substantially more reliable in younger participants (age < 35 years) and those with BDNF Met– and APOE ε4– genotypes.

Ethics Statement

This study was approved by the Institutional Review Board at Beth Israel Deaconess Medical Center in accordance with the Declaration of Helsinki. All participants provided written informed consent prior to enrollment and received monetary compensation upon completion.

Author Contributions

AJ, AR, and AP-L conceived and designed the study. AJ and GB collected the data. AJ analyzed the data and drafted the manuscript. AJ, PF, AR, and AP-L interpreted the data. All authors revised the manuscript, approved the final version, and agreed to be accountable for the content of the work.

Funding

This study was primarily funded by the National Institutes of Health (NIH R01 MH100186). AJ was further supported by postdoctoral fellowships from the Natural Sciences and Engineering Research Council of Canada (NSERC 454617) and the Canadian Institutes of Health Research (CIHR 41791). PF was supported in part by the NIH (R21 NS082970, R21 AG051846). LO was further supported by the Simons Foundation Autism Research Initiative (SFARI) and the Nancy Lurie Marks Family Foundation. AR was further supported by the NIH (R01 NS088583), The Boston Children’s Hospital Translational Research Program, Autism Speaks, Massachusetts Life Sciences, The Assimon Family, Brainsway, CRE Medical, Eisai, Neuroelectrics, Roche, Sage Therapeutics and Takeda Medical. AP-L was further supported by the Sidney R. Baer Jr. Foundation, the NIH (R01 HD069776, R01 NS073601, R21 MH099196, R21 NS085491, R21 HD07616), the Football Players Health Study at Harvard University, and Harvard Catalyst | The Harvard Clinical and Translational Science Center (NIH NCRR and NCATS, UL1 RR025758). The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard Catalyst, Harvard University and its affiliated academic health care centers, or any of the listed granting agencies.

Conflict of Interest Statement

AP-L serves on the scientific advisory boards for Neuronix, Starlab Neuroscience, Neuroelectrics, Constant Therapy, Cognito, NovaVision, and Neosync, and is listed as an inventor on several issued and pending patents on real-time integration of TMS with EEG and MRI. AR is a founder and advisor for Neuromotion, serves on the medical advisory board for NeuroRex, and is listed as an inventor on a patent related to integration of TMS and EEG.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Stephanie Changeau, Aaron Boes, and Simon Laganiere (Beth Israel Deaconess Medical Center) for assistance with physical/neurological examinations, and Ann Connor and Joanna Macone (Beth Israel Deaconess Medical Center) for regulatory oversight and compliance, and for assistance with evaluation of participants’ health and medical history.

Abbreviations

%Δ, percent change from the baseline; Δ_B-A, visit-B minus visit-A; ΔMEP, natural log-transformed, baseline-corrected amplitude of motor evoked potentials; APOE, apolipoprotein E; AUC, area under-the-curve; BDNF, brain-derived neurotrophic factor; cTBS, continuous theta-burst stimulation; FDR, false discovery rate; GABA, gamma-aminobutyric acid; ICC, intraclass correlation coefficient; iTBS, intermittent theta-burst stimulation; LME, linear mixed-effects regression analysis; LTD, long-term depression; LTP, long-term potentiation; NMDA, N-Methyl-D-aspartic acid; PCR, polymerase chain reaction; Rm-ANOVA, repeated-measures analysis of variance; SNP, single-nucleotide polymorphism; T0–Tn, over the first n minutes following cTBS; TMS, transcranial magnetic stimulation; Tn, at n minutes post-cTBS.

References

Abidin, I., Eysel, U. T., Lessmann, V., and Mittmann, T. (2008). Impaired GABAergic inhibition in the visual cortex of brain-derived neurotrophic factor heterozygous knockout mice: GABA release is impaired in visual cortex of BDNF heterozygous KO mice. J. Physiol. 586, 1885–1901. doi: 10.1113/jphysiol.2007.148627

CrossRef Full Text | Google Scholar

Antal, A., Chaieb, L., Moliadze, V., Monte-Silva, K., Poreisz, C., Thirugnanasambandam, N., et al. (2010). Brain-derived neurotrophic factor (BDNF) gene polymorphisms shape cortical plasticity in humans. Brain Stimul. 3, 230–237. doi: 10.1016/j.brs.2009.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Auton, A., Abecasis, G. R., Altshuler, D. M., Durbin, R. M., Abecasis, G. R., Bentley, D. R., et al. (2015). A global reference for human genetic variation. Nature 526, 68–74. doi: 10.1038/nature15393

PubMed Abstract | CrossRef Full Text | Google Scholar

Badawy, R. A. B., Vogrin, S. J., Lai, A., and Cook, M. J. (2013). Cortical excitability changes correlate with fluctuations in glucose levels in patients with epilepsy. Epilepsy Behav. 27, 455–460. doi: 10.1016/j.yebeh.2013.03.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Barker, A. T., Jalinous, R., and Freeston, I. L. (1985). Non-invasive magnetic stimulation of human motor cortex. Lancet 325, 1106–1107. doi: 10.1016/S0140-6736(85)92413-4

CrossRef Full Text | Google Scholar

Bastani, A., and Jaberzadeh, S. (2012). A higher number of TMS-elicited mep from a combined hotspot improves intra- and inter-session reliability of the upper limb muscles in healthy individuals. PLoS One 7:e47582. doi: 10.1371/journal.pone.0047582

PubMed Abstract | CrossRef Full Text | Google Scholar

Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B 57, 289–300. doi: 10.1111/j.2517-6161.1995.tb02031.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Benjamini, Y., and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. stat. 29, 1165–1188. doi: 10.1186/1471-2105-9-114

PubMed Abstract | CrossRef Full Text | Google Scholar

Billard, J., Lamour, Y., and Dutar, P. (1995). Decreased monosynaptic GABAB-mediated inhibitory postsynaptic potentials in hippocampal CA1 pyramidal cells in the aged rat: pharmacological characterization and possible mechanisms. J. Neurophysiol. 74, 539–546. doi: 10.1152/jn.1995.74.2.539

PubMed Abstract | CrossRef Full Text | Google Scholar

Brownjohn, P. W., Reynolds, J. N. J., Matheson, N., Fox, J., and Shemmell, J. B. H. (2014). The Effects of individualized theta burst stimulation on the excitability of the human motor system. Brain Stimul. 7, 260–268. doi: 10.1016/j.brs.2013.12.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Cantone, M., Di Pino, G., Capone, F., Piombo, M., Chiarello, D., Cheeran, B., et al. (2014). The contribution of transcranial magnetic stimulation in the diagnosis and in the management of dementia. Clin. Neurophysiol. 125, 1509–1532. doi: 10.1016/j.clinph.2014.04.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Carrette, S., Boon, P., Dekeyser, C., Klooster, D. C. W., Carrette, E., Meurs, A., et al. (2016). Repetitive transcranial magnetic stimulation for the treatment of refractory epilepsy. Expert Rev. Neurother. 16, 1093–1110. doi: 10.1080/14737175.2016.1197119

PubMed Abstract | CrossRef Full Text | Google Scholar

Carroll, T. J., Riek, S., and Carson, R. G. (2001). Reliability of the input–output properties of the cortico-spinal pathway obtained from transcranial magnetic and electrical stimulation. J. Neurosci. Methods 112, 193–202. doi: 10.1016/S0165-0270(01)00468-X

CrossRef Full Text | Google Scholar

Cazzoli, D., Muri, R. M., Schumacher, R., von Arx, S., Chaves, S., Gutbrod, K., et al. (2012). Theta burst stimulation reduces disability during the activities of daily living in spatial neglect. Brain 135, 3426–3439. doi: 10.1093/brain/aws182

PubMed Abstract | CrossRef Full Text | Google Scholar

Cerqueira, V., de Mendonça, A., Minez, A., Dias, A. R., and de Carvalho, M. (2006). Does caffeine modify corticomotor excitability? Neurophysiol. Clin. 36, 219–226. doi: 10.1016/j.neucli.2006.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, W. H., Bang, O. Y., Shin, Y.-I., Lee, A., Pascual-Leone, A., and Kim, Y.-H. (2014). BDNF polymorphism and differential rTMS effects on motor recovery of stroke patients. Brain Stimul. 7, 553–558. doi: 10.1016/j.brs.2014.03.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheeran, B. J., Ritter, C., Rothwell, J. C., and Siebner, H. R. (2009). Mapping genetic influences on the corticospinal motor system in humans. Neuroscience 164, 156–163. doi: 10.1016/j.neuroscience.2009.01.054

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheeran, B. J., Talelli, P., Mori, F., Koch, G., Suppa, A., Edwards, M., et al. (2008). A common polymorphism in the brain-derived neurotrophic factor gene (BDNF) modulates human cortical plasticity and the response to rTMS: BNDF polymorphism modulates response to rTMS. J. Physiol. 586, 5717–5725. doi: 10.1113/jphysiol.2008.159905

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, Y., Durakoglugil, M. S., Xian, X., and Herz, J. (2010). ApoE4 reduces glutamate receptor function and synaptic plasticity by selectively impairing ApoE receptor recycling. Proc. Natl. Acad. Sci. U.S.A. 107, 12011–12016. doi: 10.1073/pnas.0914984107

PubMed Abstract | CrossRef Full Text | Google Scholar

Christie, A., Fling, B., Crews, R. T., Mulwitz, L. A., and Kamen, G. (2007). Reliability of motor-evoked potentials in the ADM muscle of older adults. J. Neurosci. Methods 164, 320–324. doi: 10.1016/j.jneumeth.2007.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Chuang, W.-L., Huang, Y.-Z., Lu, C.-S., and Chen, R.-S. (2014). Reduced cortical plasticity and GABAergic modulation in essential tremor. Mov. Disord. 29, 501–507. doi: 10.1002/mds.25809

PubMed Abstract | CrossRef Full Text | Google Scholar

Chung, S. W., Hill, A. T., Rogasch, N. C., Hoy, K. E., and Fitzgerald, P. B. (2016). Use of theta-burst stimulation in changing excitability of motor cortex: a systematic review and meta-analysis. Neurosci. Biobehav. Rev. 63, 43–64. doi: 10.1016/j.neubiorev.2016.01.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Civardi, C., Boccagni, C., Vicentini, R., Bolamperti, L., Tarletti, R., Varrasi, C., et al. (2001). Cortical excitability and sleep deprivation: a transcranial magnetic stimulation study. J. Neurol. Neurosurg. Psychiatry 71, 809–812. doi: 10.1136/jnnp.71.6.809

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, D. A., Pascual-Leone, A., Press, D. Z., and Robertson, E. M. (2005). Off-line learning of motor skill memory: a double dissociation of goal and movement. Proc. Natl Acad. Sci. U.S.A. 102, 18237–18241. doi: 10.1073/pnas.0506072102

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Hilsdale. NJ: Lawrence Earlbaum Associates.

Google Scholar

Cohen, J. (1992). A power primer. Psychol. Bull. 112, 155–159.

Google Scholar

Crum, R. M., Anthony, J. C., Bassett, S. S., and Folstein, M. F. (1993). Population-based norms for the mini-mental state examination by age and educational level. JAMA 269, 2386–2391. doi: 10.1001/jama.1993.03500180078038

CrossRef Full Text | Google Scholar

Davila-Pérez, P., Jannati, A., Fried, P. J., Cudeiro Mazaira, J., and Pascual-Leone, A. (2018). The effects of waveform and current direction on the efficacy and test–retest reliability of transcranial magnetic stimulation. Neuroscience 393, 97–109. doi: 10.1016/j.neuroscience.2018.09.044

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Lazzaro, V., Capone, F., Di Pino, G., Pellegrino, G., Florio, L., Zollo, L., et al. (2016). Combining robotic training and non-invasive brain stimulation in severe upper limb-impaired chronic stroke patients. Front. Neurosci. 10:88. doi: 10.3389/fnins.2016.00088

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Lazzaro, V., Pellegrino, G., Di Pino, G., Corbetto, M., Ranieri, F., Brunelli, N., et al. (2015). Val66Met BDNF gene polymorphism influences human motor cortex plasticity in acute stroke. Brain Stimul. 8, 92–96. doi: 10.1016/j.brs.2014.08.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Lazzaro, V., Rothwell, J. C., Talelli, P., Capone, F., Ranieri, F., Wallace, A. C., et al. (2013). Inhibitory theta burst stimulation of affected hemisphere in chronic stroke: a proof of principle, sham-controlled study. Neurosci. Lett. 553, 148–152. doi: 10.1016/j.neulet.2013.08.013

PubMed Abstract | CrossRef Full Text | Google Scholar

DNA Genotek Inc. (2015). Laboratory Protocol for Manual Purification of DNA from 0.5 mL of Sample. Available at: http://www.dnagenotek.com/US/pdf/PD-PR-006.pdf (accessed January 12, 2016).

Eberle, M.-C., Wildgruber, D., Wasserka, B., Fallgatter, A. J., and Plewnia, C. (2010). Relief from chronic intractable auditory hallucinations after long-term bilateral theta burst stimulation. Am. J. Psychiatry 167, 1410–1410. doi: 10.1176/appi.ajp.2010.10070988

PubMed Abstract | CrossRef Full Text | Google Scholar

Fleming, M. K., Sorinola, I. O., Newham, D. J., Roberts-Lewis, S. F., and Bergmann, J. H. M. (2012). The effect of coil type and navigation on the reliability of transcranial magnetic stimulation. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 617–625. doi: 10.1109/TNSRE.2012.2202692

PubMed Abstract | CrossRef Full Text | Google Scholar

Folstein, M. F., Folstein, S. E., and McHugh, P. R. (1975). Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 12, 189–198.

Google Scholar

Forogh, B., Yazdi-Bahri, S.-M., Ahadi, T., Fereshtehnejad, S.-M., and Raissi, G. R. (2014). Comparison of two protocols of transcranial magnetic stimulation for treatment of chronic tinnitus: a randomized controlled clinical trial of burst repetitive versus high-frequency repetitive transcranial magnetic stimulation. Neurol. Sci. 35, 227–232. doi: 10.1007/s10072-013-1487-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Foucher, J., Lorgouilloux, K., Turek, J., Pham, B.-T., Elowe, J., Bayle, B., et al. (2012). “Robotic assistance in coil positioning improves reliability and comfort,” in Proceedings of the 3rd Annual Conference of the German Society for Brain Stimulation - Modulating Emotions, (Berlin).

Google Scholar

Fox, M. D., Halko, M. A., Eldaief, M. C., and Pascual-Leone, A. (2012). Measuring and manipulating brain connectivity with resting state functional connectivity magnetic resonance imaging (fcMRI) and transcranial magnetic stimulation (TMS). NeuroImage 62, 2232–2243. doi: 10.1016/j.neuroimage.2012.03.035

PubMed Abstract | CrossRef Full Text | Google Scholar

Freitas, C., Mondragón-Llorca, H., and Pascual-Leone, A. (2011). Noninvasive brain stimulation in Alzheimer’s disease: systematic review and perspectives for the future. Exp. Gerontol. 46, 611–627. doi: 10.1016/j.exger.2011.04.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Fried, P. J., Jannati, A., Davila-Pérez, P., and Pascual-Leone, A. (2017). Reproducibility of single-pulse, paired-pulse, and intermittent theta-burst tms measures in healthy aging, type-2 diabetes, and alzheimer’s disease. Front. Aging Neurosci. 9:263. doi: 10.3389/fnagi.2017.00263

CrossRef Full Text | Google Scholar

Fried, P. J., Schilberg, L., Brem, A.-K., Saxena, S., Wong, B., Cypess, A. M., et al. (2016). Humans with type-2 diabetes show abnormal long-term potentiation-like cortical plasticity associated with verbal learning deficits. J. Alzheimer’s Dis. 55, 89–100. doi: 10.3233/JAD-160505

PubMed Abstract | CrossRef Full Text | Google Scholar

Friedman, H. (1968). Magnitude of experimental effect and a table for its rapid estimation. Psychol. Bull. 70, 245–251. doi: 10.1037/h0026258

PubMed Abstract | CrossRef Full Text | Google Scholar

Gentner, R., Wankerl, K., Reinsberger, C., Zeller, D., and Classen, J. (2008). Depression of human corticospinal excitability induced by magnetic theta-burst stimulation: evidence of rapid polarity-reversing metaplasticity. Cereb. Cortex 18, 2046–2053. doi: 10.1093/cercor/bhm239

PubMed Abstract | CrossRef Full Text | Google Scholar

Goldsworthy, M. R., Müller-Dahlhaus, F., Ridding, M. C., and Ziemann, U. (2014). Inter-subject variability of LTD-like plasticity in human motor cortex: a matter of preceding motor activation. Brain Stimul. 7, 864–870. doi: 10.1016/j.brs.2014.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Goldsworthy, M. R., Pitcher, J. B., and Ridding, M. C. (2012). The application of spaced theta burst protocols induces long-lasting neuroplastic changes in the human motor cortex. Eur. J. Neurosci. 35, 125–134. doi: 10.1111/j.1460-9568.2011.07924.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Guerra, A., López-Alonso, V., Cheeran, B., and Suppa, A. (2017). Variability in non-invasive brain stimulation studies: reasons and results. Neurosci. Lett. doi: 10.1016/j.neulet.2017.12.058 [Epub ahead of print],

PubMed Abstract | CrossRef Full Text | Google Scholar

Guo, S. W., and Thompson, E. A. (1992). Performing the exact test of Hardy-Weinberg proportion for multiple alleles. Biometrics 48, 361–372. doi: 10.2307/2532296

PubMed Abstract | CrossRef Full Text | Google Scholar

Hallett, M. (2007). Transcranial magnetic stimulation: a primer. Neuron 55, 187–199. doi: 10.1016/j.neuron.2007.06.026

PubMed Abstract | CrossRef Full Text | Google Scholar

Hamada, M., Murase, N., Hasan, A., Balaratnam, M., and Rothwell, J. C. (2013). The role of interneuron networks in driving human motor cortical plasticity. Cereb. Cortex 23, 1593–1605. doi: 10.1093/cercor/bhs147

PubMed Abstract | CrossRef Full Text | Google Scholar

Harrell, F. E. Jr. (2015). Regression Modeling Strategies: with Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. Berlin: Springer.

Google Scholar

Hattemer, K., Knake, S., Reis, J., Rochon, J., Oertel, W. H., Rosenow, F., et al. (2007). Excitability of the motor cortex during ovulatory and anovulatory cycles: a transcranial magnetic stimulation study. Clin. Endocrinol. 66, 387–393. doi: 10.1111/j.1365-2265.2007.02744.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Heidegger, T., Hansen-Goos, O., Batlaeva, O., Annak, O., Ziemann, U., and Lötsch, J. (2017). A data-driven approach to responder subgroup identification after paired continuous theta burst stimulation. Front. Hum. Neurosci. 11:382. doi: 10.3389/fnhum.2017.00382

PubMed Abstract | CrossRef Full Text | Google Scholar

Hellriegel, H., Schulz, E. M., Siebner, H. R., Deuschl, G., and Raethjen, J. H. (2012). Continuous theta-burst stimulation of the primary motor cortex in essential tremor. Clin. Neurophysiol. 123, 1010–1015. doi: 10.1016/j.clinph.2011.08.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Hermsen, A., Haag, A., Duddek, C., Balkenhol, K., Bugiel, H., Bauer, S., et al. (2016). Test–retest reliability of single and paired pulse transcranial magnetic stimulation parameters in healthy subjects. J. Neurol. Sci. 362, 209–216. doi: 10.1016/j.jns.2016.01.039

PubMed Abstract | CrossRef Full Text | Google Scholar

Hinder, M. R., Goss, E. L., Fujiyama, H., Canty, A. J., and Garry, M. I. (2014). Inter-and intra-individual variability following intermittent theta burst stimulation: implications for rehabilitation and recovery. Brain Stimul. 7, 365–371. doi: 10.1016/j.brs.2014.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Hordacre, B., Goldsworthy, M. R., Vallence, A.-M., Darvishi, S., Moezzi, B., Hamada, M., et al. (2017). Variability in neural excitability and plasticity induction in the human cortex: a brain stimulation study. Brain Stimul. 10, 588–595. doi: 10.1016/j.brs.2016.12.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, Y.-Z., Chen, R.-S., Rothwell, J. C., and Wen, H.-Y. (2007). The after-effect of human theta burst stimulation is NMDA receptor dependent. Clin. Neurophysiol. 118, 1028–1032. doi: 10.1016/j.clinph.2007.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, Y.-Z., Edwards, M. J., Rounis, E., Bhatia, K. P., and Rothwell, J. C. (2005). Theta burst stimulation of the human motor cortex. Neuron 45, 201–206. doi: 10.1016/j.neuron.2004.12.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Hunter, J. E., and Schmidt, F. L. (1994). “Correcting for sources of artificial variation across studies,” in The Handbook of Research Synthesis, eds H. Cooper and L. V. Hedge (New York, NY: Russell Sage Foundation), 323–336.

Google Scholar

Iezzi, E., Conte, A., Suppa, A., Agostino, R., Dinapoli, L., Scontrini, A., et al. (2008). Phasic voluntary movements reverse the aftereffects of subsequent theta-burst stimulation in humans. J. Neurophysiol. 100, 2070–2076. doi: 10.1152/jn.90521.2008

PubMed Abstract | CrossRef Full Text | Google Scholar

Jannati, A., Block, G., Oberman, L. M., Rotenberg, A., and Pascual-Leone, A. (2017). Interindividual variability in response to continuous theta-burst stimulation in healthy adults. Clin. Neurophysiol. 128, 2268–2278. doi: 10.1016/j.clinph.2017.08.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Julkunen, P., Säisänen, L., Danner, N., Niskanen, E., Hukkanen, T., Mervaala, E., et al. (2009). Comparison of navigated and non-navigated transcranial magnetic stimulation for motor cortex mapping, motor threshold and motor evoked potentials. NeuroImage 44, 790–795. doi: 10.1016/j.neuroimage.2008.09.040

PubMed Abstract | CrossRef Full Text | Google Scholar

Kamen, G. (2004). Reliability of motor-evoked potentials during resting and active contraction conditions. Med. Sci. Sports & Exercise 36, 1574–1579. doi: 10.1249/01.mss.0000139804.02576.6a

CrossRef Full Text | Google Scholar

Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., and Kashy, D. A. (2002). The statistical analysis of data from small groups. J. Personal. Soc. Psychol. 83, 126–137. doi: 10.1037//0022-3514.83.1.126

PubMed Abstract | CrossRef Full Text | Google Scholar

Kimiskidis, V. K., Papagiannopoulos, S., Sotirakoglou, K., Kazis, D. A., Dimopoulos, G., Kazis, A., et al. (2004). The repeatability of corticomotor threshold measurements. Neurophysiol. Clin. 34, 259–266. doi: 10.1016/j.neucli.2004.10.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Koch, G., Bonni, S., Giacobbe, V., Bucchi, G., Basile, B., Lupo, F., et al. (2012). Theta-burst stimulation of the left hemisphere accelerates recovery of hemispatial neglect. Neurology 78, 24–30. doi: 10.1212/WNL.0b013e31823ed08f

PubMed Abstract | CrossRef Full Text | Google Scholar

Koch, G., Brusa, L., Carrillo, F., Lo Gerfo, E., Torriero, S., Oliveri, M., et al. (2009). Cerebellar magnetic stimulation decreases levodopa-induced dyskinesias in Parkinson disease. Neurology 73, 113–119. doi: 10.1212/WNL.0b013e3181ad5387

PubMed Abstract | CrossRef Full Text | Google Scholar

Kreuzer, P., Langguth, B., Popp, R., Raster, R., Busch, V., Frank, E., et al. (2011). Reduced intra-cortical inhibition after sleep deprivation: a transcranial magnetic stimulation study. Neurosci. Lett. 493, 63–66. doi: 10.1016/j.neulet.2011.02.044

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, M., Kim, S. E., Kim, W. S., Lee, J., Yoo, H. K., Park, K.-D., et al. (2013). Interaction of motor training and intermittent theta burst stimulation in modulating motor cortical plasticity: influence of BDNF Val66Met polymorphism. PLoS One 8:e57690. doi: 10.1371/journal.pone.0057690

PubMed Abstract | CrossRef Full Text | Google Scholar

Lentz, M., and Nielsen, J. F. (2002). Post-exercise facilitation and depression of M wave and motor evoked potentials in healthy subjects. Clin. Neurophysiol. 113, 1092–1098. doi: 10.1016/S1388-2457(02)00031-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, C.-T., Chen, M.-H., Juan, C.-H., Huang, H.-H., Chen, L.-F., Hsieh, J.-C., et al. (2014). Efficacy of prefrontal theta-burst stimulation in refractory depression: a randomized sham-controlled study. Brain 137, 2088–2098. doi: 10.1093/brain/awu109

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, H., and Au-Yeung, S. S. Y. (2014). Reliability of transcranial magnetic stimulation induced corticomotor excitability measurements for a hand muscle in healthy and chronic stroke subjects. J. Neurol. Sci. 341, 105–109. doi: 10.1016/j.jns.2014.04.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Livingston, S. C., and Ingersoll, C. D. (2008). Intra-rater reliability of a transcranial magnetic stimulation technique to obtain motor evoked potentials. Int. J. Neurosci. 118, 239–256. doi: 10.1080/00207450701668020

PubMed Abstract | CrossRef Full Text | Google Scholar

López-Alonso, V., Cheeran, B., Río-Rodríguez, D., and Fernández-del-Olmo, M. (2014). Inter-individual variability in response to non-invasive brain stimulation paradigms. Brain Stimul. 7, 372–380. doi: 10.1016/j.brs.2014.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Maeda, F., Keenan, J. P., Tormos, J. M., Topka, H., and Pascual-Leone, A. (2000). Modulation of corticospinal excitability by repetitive transcranial magnetic stimulation. Clin. Neurophysiol. 111, 800–805. doi: 10.1016/s1388-2457%2899%2900323-5

PubMed Abstract | CrossRef Full Text | Google Scholar

McClintock, S. M., Freitas, C., Oberman, L. M., Lisanby, S. H., and Pascual-Leone, A. (2011). Transcranial magnetic stimulation: a neuroscientific probe of cortical function in schizophrenia. Biol. Psychiatry 70, 19–27. doi: 10.1016/j.biopsych.2011.02.031

PubMed Abstract | CrossRef Full Text | Google Scholar

McDonnell, M. N., Ridding, M. C., and Miles, T. S. (2004). Do alternate methods of analysing motor evoked potentials give comparable results? J. Neurosci. Methods 136, 63–67. doi: 10.1016/j.jneumeth.2003.12.020

PubMed Abstract | CrossRef Full Text | Google Scholar

McGraw, K. O., and Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychol. Methods 1, 30–46. doi: 10.1037/1082-989x.1.1.30

PubMed Abstract | CrossRef Full Text | Google Scholar

McQuail, J. A., Bañuelos, C., LaSarge, C. L., Nicolle, M. M., and Bizon, J. L. (2012). GABAB receptor GTP-binding is decreased in the prefrontal cortex but not the hippocampus of aged rats. Neurobiol. Aging 33, 1124.e1–1124.e12. doi: 10.1016/j.neurobiolaging.2011.11.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Milbrandt, J. C., Albin, R. L., and Caspary, D. M. (1994). Age-related decrease in GABAB receptor binding in the Fischer 344 rat i inferior colliculus. Neurobiol. Aging 15, 699–703.

Google Scholar

Mori, F., Rossi, S., Piccinin, S., Motta, C., Mango, D., Kusayanagi, H., et al. (2013). Synaptic plasticity and PDGF signaling defects underlie clinical progression in multiple sclerosis. J. Neurosci. 33, 19112–19119. doi: 10.1523/JNEUROSCI.2536-13.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Nettekoven, C., Volz, L. J., Kutscha, M., Pool, E.-M., Rehme, A. K., Eickhoff, S. B., et al. (2014). Dose-dependent effects of theta burst rTMS on cortical excitability and resting-state connectivity of the human motor system. J. Neurosci. 34, 6849–6859. doi: 10.1523/JNEUROSCI.4993-13.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

Nettekoven, C., Volz, L. J., Leimbach, M., Pool, E.-M., Rehme, A. K., Eickhoff, S. B., et al. (2015). Inter-individual variability in cortical excitability and motor network connectivity following multiple blocks of rTMS. NeuroImage 118, 209–218. doi: 10.1016/j.neuroimage.2015.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Ngomo, S., Leonard, G., Moffet, H., and Mercier, C. (2012). Comparison of transcranial magnetic stimulation measures obtained at rest and under active conditions and their reliability. J. Neurosci. Methods 205, 65–71. doi: 10.1016/j.jneumeth.2011.12.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Nichol, K., Deeny, S. P., Seif, J., Camaclang, K., and Cotman, C. W. (2009). Exercise improves cognition and hippocampal plasticity in APOE ε4 mice. Alzheimers Dementia 5, 287–294. doi: 10.1016/j.jalz.2009.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

NIH Office of Extramural Research (2001). NIH Policy and Guidelines on the Inclusion of Women and Minorities as Subjects in Clinical Research. Available at: https://grants.nih.gov/grants/funding/women_min/guidelines_amended_10_2001.htm (accessed February 1, 2016).

Google Scholar

Oberman, L., Ifert-Miller, F., Najib, U., Bashir, S., Woollacott, I., Gonzalez-Heydrich, J., et al. (2010). Transcranial magnetic stimulation provides means to assess cortical plasticity and excitability in humans with fragile X syndrome and autism spectrum disorder. Front. Synaptic Neurosci. 2:26. doi: 10.3389/fnsyn.2010.00026

PubMed Abstract | CrossRef Full Text | Google Scholar

Oberman, L. M., Eldaief, M., Fecteau, S., Ifert-Miller, F., Tormos, J. M., and Pascual-Leone, A. (2012). Abnormal modulation of corticospinal excitability in adults with Asperger’s syndrome: modulation of excitability in Asperger’s. Eur. J. Neurosci. 36, 2782–2788. doi: 10.1111/j.1460-9568.2012.08172.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Oberman, L. M., Ifert-Miller, F., Najib, U., Bashir, S., Gonzalez-Heydrich, J., Picker, J., et al. (2016). Abnormal mechanisms of plasticity and metaplasticity in autism spectrum disorders and fragile X syndrome. J. Child Adoles. Psychopharmacol. 26, 617–624. doi: 10.1089/cap.2015.0166

PubMed Abstract | CrossRef Full Text | Google Scholar

Oberman, L. M., Pascual-Leone, A., and Rotenberg, A. (2014). Modulation of corticospinal excitability by transcranial magnetic stimulation in children and adolescents with autism spectrum disorder. Front. Hum. Neurosci. 8:627. doi: 10.3389/fnhum.2014.00627

PubMed Abstract | CrossRef Full Text | Google Scholar

Opie, G. M., Vosnakis, E., Ridding, M. C., Ziemann, U., and Semmler, J. G. (2017). Priming theta burst stimulation enhances motor cortex plasticity in young but not old adults. Brain Stimul. 10, 298–304. doi: 10.1016/j.brs.2017.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Pascual-Leone, A., Amedi, A., Fregni, F., and Merabet, L. B. (2005). The plastic human brain cortex. Annu. Rev. Neurosci. 28, 377–401. doi: 10.1146/annurev.neuro.27.070203.144216

CrossRef Full Text | Google Scholar

Pascual-Leone, A., Freitas, C., Oberman, L., Horvath, J. C., Halko, M., Eldaief, M., et al. (2011). Characterizing brain cortical plasticity and network dynamics across the age-span in health and disease with TMS-EEG and TMS-fMRI. Brain Topogr. 24, 302–315. doi: 10.1007/s10548-011-0196-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Pascual-Leone, A., Valls-Solé, J., Wassermann, E. M., and Hallett, M. (1994). Responses to rapid-rate transcranial magnetic stimulation of the human motor cortex. Brain 117, 847–858. doi: 10.1093/brain/117.4.847

CrossRef Full Text | Google Scholar

Pellicciari, M. C., Miniussi, C., Ferrari, C., Koch, G., and Bortoletto, M. (2016). Ongoing cumulative effects of single TMS pulses on corticospinal excitability: an intra- and inter-block investigation. Clin. Neurophysiol. 127, 621–628. doi: 10.1016/j.clinph.2015.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Peña-Gomez, C., Solé-Padullés, C., Clemente, I. C., Junqué, C., Bargalló, N., Bosch, B., et al. (2012). APOE status modulates the changes in network connectivity induced by brain stimulation in non-demented elders. PLoS One 7:e51833. doi: 10.1371/journal.pone.0051833

PubMed Abstract | CrossRef Full Text | Google Scholar

Portney, L. G., and Watkins, M. P. (2009). Foundations of Clinical Research: Applications to Practice, 3rd Edn. Upper Saddle River, NJ: Prentice Hall.

Google Scholar

Rossi, S., Hallett, M., Rossini, P. M., and Pascual-Leone, A. (2009). Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research. Clin. Neurophysiol. 120, 2008–2039. doi: 10.1016/j.clinph.2009.08.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossini, P. M., Burke, D., Chen, R., Cohen, L. G., Daskalakis, Z., Di Iorio, R., et al. (2015). Non-invasive electrical and magnetic stimulation of the brain, spinal cord, roots and peripheral nerves: basic principles and procedures for routine clinical and research application. An updated report from an I.F.C.N. Committee. Clin. Neurophysiol. 126, 1071–1107. doi: 10.1016/j.clinph.2015.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Samii, A., Wassermann, E. M., and Hallett, M. (1997). Post-exercise depression of motor evoked potentials as a function of exercise duration. Electroencephalogr. Clin. Neurophysiol. 105, 352–356. doi: 10.1016/S0924-980X(97)00033-7

CrossRef Full Text | Google Scholar

Sankarasubramanian, V., Roelle, S. M., Bonnett, C. E., Janini, D., Varnerin, N. M., Cunningham, D. A., et al. (2015). Reproducibility of transcranial magnetic stimulation metrics in the study of proximal upper limb muscles. J. Electromyogr. Kinesiol. 25, 754–764. doi: 10.1016/j.jelekin.2015.05.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Schambra, H. M., Ogden, R. T., Martínez-Hernández, I. E., Lin, X., Chang, Y. B., Rahman, A., et al. (2015). The reliability of repeated TMS measures in older adults and in patients with subacute and chronic stroke. Front. Cell Neurosci. 9:335. doi: 10.3389/fncel.2015.00335

PubMed Abstract | CrossRef Full Text | Google Scholar

Schilberg, L., Schuhmann, T., and Sack, A. T. (2017). Interindividual variability and intraindividual reliability of intermittent theta burst stimulation-induced neuroplasticity mechanisms in the healthy brain. J. Cogn. Neurosci. 29, 1022–1032. doi: 10.1162/jocn_a_01100

PubMed Abstract | CrossRef Full Text | Google Scholar

Smith, M. J., Keel, J. C., Greenberg, B. D., Adams, L. F., Schmidt, P. J., Rubinow, D. A., et al. (1999). Menstrual cycle effects on cortical excitability. Neurology 53, 2069–2069. doi: 10.1212/wnl.53.9.2069

CrossRef Full Text | Google Scholar

Specterman, M., Bhuiya, A., Kuppuswamy, A., Strutton, P., Catley, M., and Davey, N. (2005). The effect of an energy drink containing glucose and caffeine on human corticospinal excitability. Physiol. Behav. 83, 723–728. doi: 10.1016/j.physbeh.2004.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Stagg, C. J., Wylezinska, M., Matthews, P. M., Johansen-Berg, H., Jezzard, P., Rothwell, J. C., et al. (2009). Neurochemical effects of theta burst stimulation as assessed by magnetic resonance spectroscopy. J. Neurophysiol. 101, 2872–2877. doi: 10.1152/jn.91060.2008

PubMed Abstract | CrossRef Full Text | Google Scholar

Suppa, A., Huang, Y.-Z., Funke, K., Ridding, M. C., Cheeran, B., Di Lazzaro, V., et al. (2016). Ten years of theta burst stimulation in humans: established knowledge, unknowns and prospects. Brain Stimul. 9, 323–335. doi: 10.1016/j.brs.2016.01.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Suppa, A., Marsili, L., Di Stasio, F., Berardelli, I., Roselli, V., Pasquini, M., et al. (2014). Cortical and brainstem plasticity in Tourette syndrome and obsessive-compulsive disorder. Mov. Disord. 29, 1523–1531. doi: 10.1002/mds.25960

PubMed Abstract | CrossRef Full Text | Google Scholar

Thorndike, R. L., Hagen, E. P., and Sattler, J. M. (1986). The Stanford-Binet Intelligence Scale: Guide for Administering and Scoring. Rolling Meadows, IL: Riverside Publishing Company.

Google Scholar

Trippe, J., Mix, A., Aydin-Abidin, S., Funke, K., and Benali, A. (2009). Theta burst and conventional low-frequency rTMS differentially affect GABAergic neurotransmission in the rat cortex. Exp. Brain Res. 199, 411–421. doi: 10.1007/s00221-009-1961-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Valero-Cabré, A., Amengual, J. L., Stengel, C., Pascual-Leone, A., and Coubard, O. A. (2017). Transcranial magnetic stimulation in basic and clinical neuroscience: a comprehensive review of fundamental principles and novel insights. Neurosci. Biobehav. Rev. 83, 381–404. doi: 10.1016/j.neubiorev.2017.10.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Valero-Cabré, A., Pascual-Leone, A., and Rushmore, R. J. (2008). Cumulative sessions of repetitive transcranial magnetic stimulation (rTMS) build up facilitation to subsequent TMS-mediated behavioural disruptions. Eur. J. Neurosci. 27, 765–774. doi: 10.1111/j.1460-9568.2008.06045.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Vallence, A.-M., Goldsworthy, M. R., Hodyl, N. A., Semmler, J. G., Pitcher, J. B., and Ridding, M. C. (2015). Inter- and intra-subject variability of motor cortex plasticity following continuous theta-burst stimulation. Neuroscience 304, 266–278. doi: 10.1016/j.neuroscience.2015.07.043

PubMed Abstract | CrossRef Full Text | Google Scholar

Vernet, M., Bashir, S., Yoo, W.-K., Oberman, L. M., Mizrahi, I., Ifert-Miller, F., et al. (2014). Reproducibility of the effects of theta burst stimulation on motor cortical plasticity in healthy participants. Clin. Neurophysiol. 125, 320–326. doi: 10.1016/j.clinph.2013.07.004

PubMed Abstract | CrossRef Full Text | Google Scholar

White, F., Nicoll, J. A. R., Roses, A. D., and Horsburgh, K. (2001). Impaired neuronal plasticity in transgenic mice expressing human apolipoprotein E4 compared to E3 in a model of entorhinal cortex lesion. Neurobiol. Dis. 8, 611–625. doi: 10.1006/nbdi.2001.0401

PubMed Abstract | CrossRef Full Text | Google Scholar

Wigginton, J. E., Cutler, D. J., and Abecasis, G. R. (2005). A note on exact tests of hardy-weinberg equilibrium. Am. J. Hum. Genet. 76, 887–893. doi: 10.1086/429864

PubMed Abstract | CrossRef Full Text | Google Scholar

Wischnewski, M., and Schutter, D. J. L. G. (2015). Efficacy and time course of theta burst stimulation in healthy humans. Brain Stimul. 8, 685–692. doi: 10.1016/j.brs.2015.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Woo, N. H., Teng, H. K., Siao, C.-J., Chiaruttini, C., Pang, P. T., Milner, T. A., et al. (2005). Activation of p75NTR by proBDNF facilitates hippocampal long-term depression. Nat. Neurosci. 8, 1069–1077. doi: 10.1038/nn1510

PubMed Abstract | CrossRef Full Text | Google Scholar

Wright, K. (2014). “Adjusting effect sizes in light of reliability estimates,” in Paper Presented at the Annual Meeting of the Southwest Educational Research Association, (New Orleans, LA).

Zrenner, C., Belardinelli, P., Müller-Dahlhaus, F., and Ziemann, U. (2016). Closed-loop neuroscience and non-invasive brain stimulation: a tale of two loops. Front. Cell. Neurosci. 10:92. doi: 10.3389/fncel.2016.00092

PubMed Abstract | CrossRef Full Text | Google Scholar

Zrenner, C., Desideri, D., Belardinelli, P., and Ziemann, U. (2018). Real-time EEG-defined excitability states determine efficacy of TMS-induced plasticity in human motor cortex. Brain Stimul. 11, 374–389. doi: 10.1016/j.brs.2017.11.016

PubMed Abstract | CrossRef Full Text | Google Scholar