Reliability of TMS measurements using conventional hand-hold method with different numbers of stimuli for tibialis anterior muscle in healthy adults

Objective: The objective of this study was to determine the reliability of corticomotor excitability measurements using the conventional hand-hold transcranial magnetic stimulation (TMS) method for the tibialis anterior (TA) muscle in healthy adults and the number of stimuli required for reliable assessment.

Methods: Forty healthy adults participated in three repeated sessions of corticomotor excitability assessment in terms of resting motor threshold (rMT), slope of recruitment curve (RC), peak motor evoked potential amplitude (pMEP), and MEP latency using conventional TMS method. The first two sessions were conducted with a rest interval of 1 h, and the last session was conducted 7–10 days afterward. With the exception of rMT, the other three outcomes measure elicited with the block of first 3–10 stimuli were analyzed respectively. The within-day (session 1 vs. 2) and between-day (session 1 vs. 3) reliability for all four outcome measures were assessed using intraclass correlation coefficient (ICC), standard error of measurement, and minimum detectable difference at 95% confidence interval.

Results: Good to excellent within-day and between-day reliability was found for TMS-induced outcome measures examined using 10 stimuli (ICC ≥ 0.823), except in pMEP, which showed between-day reliability at moderate level (ICC = 0.730). The number of three stimuli was adequate to achieve minimum acceptable within-day reliability for all TMS-induced parameters and between-day reliability for MEP latency. With regard to between-day reliability of RC slope and pMEP, at least seven and nine stimuli were recommended respectively.

Conclusion: Our findings indicated the high reliability of corticomotor excitability measurement by TMS with adequate number of stimuli for the TA muscle in healthy adults. This result should be interpreted with caveats for the specific methodological choices, equipment setting, and the characteristics of the sample in the current study.

Clinical Trial Registration: http://www.chictr.org.cn, identifier ChiCTR2100045141.

Introduction

Transcranial magnetic stimulation (TMS) is a non-invasive technique (Barker et al., 1985) and has been widely used in evaluating brain plasticity (Hallett, 1996; Rothwell, 2011). According to the principle of electromagnetic induction, magnetic stimulation acts on the primary motor cortex of the brain (M1) and induces neural axon depolarization, which can be recorded in corresponding peripheral muscle with surface electromyography; the detected signal is called motor-evoked potential (MEP; Rotenberg et al., 2014; Rossini et al., 2015). Hence, MEP and its related parameters induced by TMS can certainly reflect the excitability of cortico-spinal pathway and are commonly used in neurophysiological examination and research (Moscatelli et al., 2021).

Changes in measurements should reflect actual changes in patients’ conditions rather than those caused by measurement errors. Therefore, testing the reliability of an assessment tool should be the primary consideration in clinical and research; unreliable measurements are prone to large systematic and random errors (Atkinson and Nevill, 1998). TMS-related outcome measures are variable, and thus their reliability has attracted considerable interest, that is, TMS can provide accurate and consistent measurements for individuals without physiological changes. Although some new techniques such as neuronavigation, robot TMS coil holding arm, and EEG-TMS closed-loop triggering strategy which could improve the accuracy of hotspot localization and reduce the variability of TMS-induced MEP (Goetz et al., 2019), these advanced techniques are more widely applied in some high-level lab for research studies. For clinical evaluation, the conventional methods of TMS using hand-hold assessment are still used by most medical institutions. On the other hand, different shapes of coil can be used for TMS assessment. Although the double-cone coils have been around for decades, there is as yet few evidence of the reliability of lower limb assessment with double-cone coils compared to upper limb studies and numerous coil brands (Beaulieu et al., 2017a).

MEP-related parameters considerably vary with the number of stimuli delivered per site during TMS assessment (Kiers et al., 1993; Wassermann, 2002). Increasing the number of stimuli may reduce the influence of inherent variations in an individual (Bastani and Jaberzadeh, 2012). However, this approach brings another problem, that is, more testing time is needed. Prolonged TMS tests would make participants feel uncomfortable. Therefore, the minimum number of stimuli to obtain reliable MEP parameters must be investigated. For reliable MEP, a minimum of five stimuli are recommended for within-session comparisons, and 10 stimuli are recommended for between-session comparisons (Cavaleri et al., 2017). Lewis et al. (2014) demonstrated that at least six magnetic stimuli are required for the reliable MEP of the soleus muscle in the same session of stroke assessment, whereas the intersession reliability was poor even with ten stimuli applied for average MEP assessment. However, studies on the assessment of lower limbs subjected to different numbers of stimuli have mainly focused on MEP amplitude, and reports on the slope of recruitment curve (RC) and MEP latency are few (Lewis et al., 2014).

With respect to the foregoing, the purpose of this study was to investigate the reliability of TMS-induced parameters in measuring the cortico-spinal excitability of the lower limb using a double-cone coil and conventional hand-hold method, and determine the number of TMS stimuli required to achieve acceptable reliable measurement. The results of the present study would provide an evidence-based knowledge about TMS as a tool for evaluating the cortico-spinal excitability of the lower limb in clinical practice and research.

Materials and Methods

Participants

Forty healthy volunteers aged 19–36 years (23.05 ± 3.83 years; 21 males) were recruited using an advertising poster from January 1, 2021 to August 31, 2021. According to the safety guidelines of TMS (Rossi et al., 2021), all participants were screened for the contraindications of TMS. The inclusion criteria were as follows: (i) age ≥ 18 years; (ii) no central nervous system lesion; (iii) no serious mental or physical disease; and (iv) no drug or alcohol dependence history; and (v) no contraindications to TMS examinations. The exclusion criteria were as follows: (i) any neurological disease; (ii) history of seizures; (iii) previous adverse effects with TMS; (iv) taking medications that affect cortical excitability; (v) pregnancy; (vi) history of lower limb surgery; and (vii) poor compliance or failure to cooperate during testing. Participants were asked to have a good sleep the night before the test, avoid taking recreational drugs or alcohol on the day before the test and avoid drinking caffeinated drinks 2 h before the test.

Experimental design and measurement procedures

This study adopted a repeated-measures design and was approved by the Research Ethics Committee of the JORU Rehabilitation Hospital (No. 20210325B02, ChiCTR2100045141). After signing informed consent, all participants underwent three sessions of assessment on corticomotor excitability with TMS for the tibialis anterior (TA) muscles of the non-dominant leg. The dominant legs of the participants were defined as the legs they would use to kick a ball (van Melick et al., 2017). The first two sessions were conducted on the first day, with a rest interval of 1 h. The last session was conducted 7–10 days afterward at the same time of the day as the first session. For each participant, all three assessment sessions were performed from 15:00 to 18:00 of the testing day by the same rater, who was a physical therapist with extensive experience on corticomotor excitability measurement using TMS.

The strength of a magnetic field induced by TMS is attenuated as the distance from the scalp surface increases (Rothwell et al., 1991; Deng et al., 2013). However, the M1 areas of the lower limb are located at the paracentral lobules, which are deeper than those of the upper limb (Rothwell et al., 1991; Rossini et al., 2015). Therefore, an electric field induced by TMS using figure-of-eight or round coil may be inadequate to activate motor neurons that control the muscles of the lower limbs (Dharmadasa et al., 2019). The double-cone coil, a figure-of-eight-shaped coil with two components at each angle, increases the strength of a magnetic field (Hallett, 2007). The magnetic field penetration depth of a double-cone coil is significantly greater than that of a figure-of-eight coil (Deng et al., 2013, 2014; Lu and Ueno, 2017). Thus, a double-cone coil stimulates the M1 region of the lower limbs more easily and is the recommended tool for examining the neurophysiological status involving lower limbs. Hence, in the present study, the corticomotor excitability measurement for the TA muscle of the non-dominant leg was measured with a Magneuro100 stimulator and a matching double-cone coil (VCZ001; VISHEE Company Limited, Nanjing, China).

During the assessment, each participant sat comfortably upright in a high back chair, and the arms, back, and legs supported. After skin preparation, a pair of silica gel electrodes was placed over the belly of the TA muscle at the upper third of the lower leg. The distance between the electrodes was 2 cm. A ground electrode was applied ipsilaterally on the lateral malleolus. Electromyography (EMG) signals were amplified (gain 1,000×), filtered (bandpass: 20–500 Hz), and sampled at 2,000 Hz with a wireless portable motor evoked potential detection module. During the measurement, participants were instructed to stay relaxed, and the relaxed state was monitored through the visual inspection of the EMG recordings in real time. When obvious muscle activities were detected, the participants were reminded, and the procedure was suspended until the muscle relaxes again.

During the measurements, a positioning cap designed according to the international 10–20 EEG system was fixed on a participant’s head. Biphasic single magnetic pulses were generated at least every 5 s to stimulate the M1 area contralateral to the recorded TA muscle around the Cz point, and the coil handle was perpendicular to the scalp (Farzan, 2014; Figure 1). The current direction in the long axis of the two-wing intersection of the coil was produced at AP orientation which would induce a PA oriented current in the cortex of the lower limb (Groppa et al., 2012). The optimal stimulation site, which was called the “hotspot”, for TA muscle was identified by moving the double-cone coil in 1 cm steps at the initial stimulation intensity and a supposed suprathreshold level and marked using a pen as the easiest excitable site that can consistently elicit large MEP amplitudes at a relatively low TMS output intensity. For each session, the same procedure used for hotspot identification was conducted prior to the corticomotor excitability outcome measurement. After the hotspot for TA muscle was identified, the resting motor threshold (rMT) was determined with the TMS coil fixed on the hotspot, and the suprathreshold stimulator output intensity was reduced by 2% stepwise, and then by 1% when the intensity was near the threshold intensity. The rMT is the lowest TMS output intensity that can induce MEP amplitudes above 0.05 mV in at least five of 10 consecutive TMS stimuli when a TMS coil is fixed on the hotspot of the TA muscle (Liu and Au-Yeung, 2014). The MEPs were measured using TMS output intensities ranging from 90% rMT to 160% rMT, and the intensity was increased by 10% for every 10 stimuli. RCs were plotted in Microsoft Excel with the MEP amplitude averaged using different numbers of consecutive MEPs from the first three to ten evoked at the same intensity against a corresponding TMS output intensity level. The RC slope was obtained by adding the linear regression line to the curve (Liu and Au-Yeung, 2014). The MEP latency was defined as the time from the delivery of TMS stimuli to MEP onset which was identified by the cursor of the self-contained system of motor evoked potential detection module and inspected visually by an investigator.

FIGURE 1

Figure 1. Coil handle is perpendicular to the scalp.

The rMT, linear slope for RC, maximum mean MEP amplitudes that were reached during RC measurement (pMEP), and mean MEP latency evoked at 130% rMT were adopted as the outcome measures for the reliability test.

Data analysis and statistics

Data analyses were performed by using the software package of SPSS23.0. Demographic data were statistically described with means ± standard deviations to represent the characteristics of the participants. Within-day reliability (session 1 vs. 2) and between-day reliability (session 1 vs. 3) were evaluated using intraclass correlation coefficients (ICCs), and their 95% confidence interval limit was based on a two-way mix model (Model 3). The level of significance was set at 0.05. In the reliability analysis, the MEP latencies evoked by 130% rMT were averaged for 3–10 stimuli, whereas the RC slopes plotted from 90% rMT to 160% rMT were calculated for 3–10 stimuli delivered at each of the series of intensities. The pMEPs were the maximum MEP amplitudes during the RC procedure at different numbers of stimuli. The reliability of the measurements was assessed as follows: >0.90, excellent; 0.75–0.90, good; 0.50–0.75, moderate; and <0.50, poor (Portney and Watkins, 2009).

According to the reliability coefficient, the standard error of measurement [SEM = (pooled SD) × √(1-ICCs)] and the minimum detectable measurement difference (MDD₉₅ = 1.96 × SEM × √2) were calculated for the analysis of response stability and test responsiveness of TMS-induced corticomotor excitability. The SEM showed variation around a “true” score of excitability parameters for an individual when repeated measures were taken. The reliability of the measurement results increased with decreasing SEM value. The MDD₉₅ represents the smallest difference value, which can be confirmed as the “true change” in cortical motor excitability measurement values within the 95% confidence interval.

Results

All the participants completed the whole procedure of the study. The demographic characteristics of all participants are presented in Table 1. Table 2 lists the reliability analysis in terms of the ICC, SEM, and MDD₉₅ of the four TMS-induced outcome measures related to cortical excitability. The within-day comparison showed excellent reliability for rMT (ICC = 0.946, p < 0.001) and MEP latency (ICC ≥ 0.941; 3–10 MEPs; p < 0.001), good reliability for RC slope (ICC = 0.776–0.880; 3–10 MEPs; p < 0.001), and good to excellent reliability for pMEP (ICC = 0.867–0.901; 3–10 MEPs; p < 0.001). With regard to between-day reliability, the rMT (ICC = 0.936, p < 0.001) and latency (ICC ≥ 0.923; 3–10 MEPs; p < 0.001) displayed excellent reliability. However, the ICCs revealed moderate to good reliability for the RC slope (ICC = 0.541–0.823; 3–10 MEPs; p < 0.01) and poor to moderate reliability for pMEP (ICC = 0.454–0.730; 3–10 MEPs; p < 0.05). When the number of stimuli was greater than 7 and 9 for the repeated assessment on different days, the ICC values of the RC slope (ICC = 0.735) and pMEP (ICC = 0.720) achieved values higher than 0.7 respectively, which is defined as the minimum acceptable reliability for outcome measures (Matheson, 2019; Figures 2–4).

FIGURE 2

Figure 2. Within-day and between-day reliability of RC slope. ICC, intraclass correlation coefficient.

FIGURE 3

Figure 3. Within-day and between-day reliability of pMEP. ICC, intraclass correlation coefficient.

FIGURE 4

Figure 4. Within-day and between-day reliability of MEP latency. ICC, intraclass correlation.

TABLE 1

Table 1. Characteristics of participants.

TABLE 2

Table 2. Within-day and between-day reliability measures for the participants.

Discussion

In this study, we assessed the within-day and between-day reliability of TMS-related outcome measures induced by various numbers of stimuli. A double-cone coil was used for the TA muscle of the non-dominant lower limb of healthy young participants. The results demonstrated good to excellent within-day and between-day repeatability for all measurements examined by 10 stimuli except pMEP which showed moderate between-day reliability. Three stimuli were sufficient to achieve minimum acceptable within-day reliability for all TMS-induced parameters and between-day reliability for MEP latency. Regarding between-day reliability of the RC slope and pMEP, a minimum of seven and nine stimuli were recommended, respectively.

Early reports suggested that ICC, SEM, and MDD₉₅ are the most accurate indexes in reliability studies (Weir, 2005; Beaulieu et al., 2017a; Lewis et al., 2020). ICC was recommended as the most suitable statistical test method for reliability parameter as it reflects the correlation and consistency of the results among repeated measurements (Portney and Watkins, 2000). SEM determines the degree of variation of measurements in a sample of individuals (Atkinson and Nevill, 1998). The accuracy of measurement and the sensitivity to changes increases with decreasing measurement error. Understanding the measurement error associated with TMS-induced outcome measures provides the profile of TMS as a tool for assessing corticomotor excitability. Notably, the MDD₉₅ value for TMS-related parameters achieved in the present study can be used in determining the true change in the corticomotor excitability of the lower limb in a population of individuals with similar characteristics.

The ICC values for the repeatability of all TMS-related parameters in the between-day tests were lower than those of the within-day in the present study. This finding was coherent with the previous findings of studies, which have demonstrated that the MDD values of TMS-induced outcome measures when a test was repeated on the same day were lower than those in tests performed several days apart regardless of whether the test was conducted in the upper limb muscles or the lower limb muscles (Cacchio et al., 2009; Bastani and Jaberzadeh, 2012; Fisher et al., 2013). This result may be attributed to the larger time-variant neurophysiological fluctuations in the cortex on different days as compared with those on the same day (Wassermann, 2002). In addition, although hotspot measurements were repeated for each session, the EEG cap was not removed between session 1 vs. 2. Re-wearing the EEG cap in session 3 may have led to slight differences in the positioning of the hotspot and placement of the double-cone coil between measurements even when the international EEG 10–20 system was used in the alignment. Similarly, the difference in the placement of EMG electrodes between 2 days might have produced a bias in the results on MEP amplitudes. These methodological factors might also cause the lower reliability of between-day tests than the within-day tests.

The TMS-related parameters in the present study were recommended to reflect the different aspects of corticomotor excitability. rMT was defined as the lowest output intensity of TMS to induce the depolarization of neural axons and can reflect the excitability of the membrane of motor neurons. The RC slope was suggested to illustrate the degree of recruitment of the corticospinal pathway during stimulation with graded intensities (Devanne et al., 1997), whereas the pMEP amplitude reflects the maximum descending responses from the corticomotor neurons during the RC procedure (Liu and Au-Yeung, 2014). In addition, MEP latency is defined as the time from the delivery of TMS stimuli to MEP onset and reflects the conduction velocity of a motor efferent pathway. Compared with MEP amplitude, MEP latency has a lower level of variation (Kiers et al., 1993). Other studies on the lower limbs have demonstrated that MT and MEP latency for the TA muscles of healthy participants (Cacchio et al., 2009) or TA (Cacchio et al., 2011; Beaulieu et al., 2017b) and quadriceps muscles (Wheaton et al., 2009) of patients with stroke provide more reliable measurements than MEP amplitude. The present study showed that the ICCs for rMT and MEP latency exceeded 0.90, revealing excellent repeatability in the within-day or between-day comparison. Compared with rMT and MEP latency, the relatively lower ICCs of RC slope and pMEP indicated large variations in these two parameters, especially between-day reliability. This result was coherent with Cacchio’s study, which indicated excellent reliability for rMT (ICC = 0.98 and 0.97 for within-day and between-day reliability, respectively) and MEP latency (ICC = 0.93 and 0.92 for within-day and between-day, respectively) and good reliability for RC slope (ICC = 0.79 and 0.78 for within-day and between-day, respectively; Cacchio et al., 2009).

Variability in an MEP-related parameter can be attributed to intrinsic factors, such as physiological fluctuations in the excitability of cortical pyramidal neurons and spinal alpha motor neurons (Kiers et al., 1993; Darling et al., 2006). It is also related to other external factors, such as age, physiological status of participants, attention level, coil positioning, and assessment methodology (Ridding and Ziemann, 2010; Bhandari et al., 2016). In healthy participants, Cacchio et al. presented higher rMT (62.25% and 63.02% maximum stimulator output for two test-retest sessions respectively) and longer latency (32.20 and 32.44 ms for two test-retest sessions respectively) for TA muscle than those obtained in the present study (Cacchio et al., 2009). This inconsistency may be due to differences in equipment used between the two studies in terms of coil types, pulse waveform, current direction, and absolute magnetic output intensity of the stimulator. Cacchio’s group conducted the measurement with a circular coil, whereas the present study used a double-cone coil. Another reason may be the different characteristics of participants, such as age and height, in the two studies. Notably, the participants (mean age: 23.05 years) recruited in the present study were younger than those recruited in Cacchio’s study (mean age: 44.8 years), and whether the heights of the participants in the two studies varied is unknown.

The excitability of the motor cortex fluctuates with the level of awakening and attention of participants and may be observed by long-term studies (Classen et al., 1998). The fatigue and discomfort of participants are significantly related to the increment of the duration of assessment using TMS (van de Ruit et al., 2015). Thus, long-term testing will reduce participants’ compliance (Mead et al., 2012), and determining the minimum number of stimuli for TMS-induced outcome measures is reasonable. On the premise of ensuring accuracy, our study reduced the time cost and addressed these unfavorable factors during the assessment of the corticomotor excitability of the lower limb with TMS.

As predicted, when more TMS stimuli were provided, high ICCs and low SEM and MDD₉₅ values for the RC slope and pMEP amplitude were found. In an assessment tool in clinical practice, the minimum acceptable value of ICC is 0.7 (Nunnally and Bernstein, 1994; Shieh, 2016; Matheson, 2019). Our results showed no obvious fluctuation in ICCs for the RC slope, pMEP amplitude, and MEP latency induced by different quantities of stimuli (3–10) when the assessments were conducted on the same day. Thus, three TMS pulses were suitable for the three TMS-related parameters, and adequate repeatability of different assessment sessions on the same day was observed. However, this inference should be treated conservatively in research and clinical practice, especially the inference about pMEP amplitude. The pMEP measured in the present study was not the simple MEP induced by a fixed TMS intensity. Rather, it was the maximum averaged MEP that appears on the RC curve. Second, the three stimuli only achieved the minimum acceptable degree of reliability for these three outcomes measured on the same day. Thus, a larger number of stimuli is recommended if time consumption and participants’ conditions are considered. On the between-day comparison, MEP latency showed excellent ICC even when three stimuli were used in the measurement protocol. TMS is a reliable measurement tool only if at least seven or nine stimuli are used in the assessment of the RC slope and pMEP amplitude, respectively. This result was similar to those of previous studies. Lewis et al. examined the reliability of the amplitude and area of MEP on the soleus muscle in healthy participants (Lewis et al., 2014). The results demonstrated that the between-day reliability was acceptable for healthy participants with an ICC value at a mean MEP amplitude of 0.71 and area of 0.85 when six pulses were used as stimuli (Lewis et al., 2014). By contrast, Cuypers’s study revealed that MEP amplitude averaged on eight consecutive stimuli is sufficient to enter the confidence interval for the first dorsal interosseous muscle in healthy individuals (Cuypers et al., 2014). The recent systematic review and meta-analysis of Cavaleri showed that the reliability of MEP amplitude on healthy participants is influenced by participants’ characteristics, target muscle, interval time, and coil type (Cavaleri et al., 2017). Their results have explained why the number of stimuli for obtaining a reliable MEP-related outcome measure in different studies are inconsistent. Therefore, our findings are only applicable to the corticomotor excitability measurement with double-cone coil for TA muscle in young healthy adults.

Our results should be interpreted with caution because of some limitations. Although the sample size of this study was within the range of other TMS reliability studies (Beaulieu et al., 2017a), the number of participants was relatively small with referring to previous studies which reported that more than 50 participants are suitable for reliability examination (Hopkins, 2000; Terwee et al., 2007). Moreover, the TMS-related parameters assessed in the present study were only using single-pulse TMS. Thus, whether the parameters examined with paired-pulse TMS are reliable is unclear. Although the conventional hand-held system of TMS remains the most widely used in clinical assessment, it cannot constantly localize hotspots in a longitudinal measurement and enable coil position tracking during a measurement session in contrast to the automatic devices such as navigated TMS and robot TMS coil holding arm (Sparing et al., 2008; Fleming et al., 2012). In addition, implementation of the closed-loop trigger strategy of phase-locked EEG-TMS can reduce the variability of MEP size caused by the brain oscillating rhythms (Zrenner et al., 2018). Hence, the reliability of TMS-induced corticomotor excitability measurements using new devices like navigated systems, robot coil arm and TMS-EEG for heterogeneous patient populations in large samples should be investigated further.

Conclusion

Although our research is not entirely innovative in methods, it has provided novel insights into the reliability of TMS-related measurements for the lower limb in healthy participants. Determining the minimum number of stimuli provides an evidence-based meliorative protocol for assessing the corticomotor excitability of the lower limb. In addition, the reported SEM and MDD₉₅ values of the four TMS measurements should be used as references for evaluating the effects of clinical trials involving people without neurological pathology.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved and the study was approved by the Research Ethics Committee of the JORU Rehabilitation Hospital (No. 20210325B02, ChiCTR2100045141). The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

HL and LW designed the whole study. HL, LZ, and BS analyzed the data, and wrote the manuscript together. YJ, QS, DL, FG, and JW performed the study. YJ, GH, and YC participated in the collation of data. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by the Major Scientific Research Project of Wuxi Health Committee (Z202013), Top Talent Support Program for Young and Middle-aged People of Wuxi Health Committee (HB2020079), Wuxi “Taihu Talent Plan” medical and health high-level talents project (WXTTP2020008), Jiangsu Geriatrics Clinical Technology Application Research Project (LR2021040), Scientific and Technological Development Fund from Wuxi Science and Technology Bureau (Y20212008), and Clinical Teaching Base Scientific Research Development Project from Jiangsu Vocational College of Medicine.

Acknowledgments

We thank to all the participants for their contributions in completing this study.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Atkinson, G., and Nevill, A. M. (1998). Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Med. 26, 217–238. doi: 10.2165/00007256-199826040-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

Barker, A. T., Jalinous, R., and Freeston, I. L. (1985). Non-invasive magnetic stimulation of human motor cortex. Lancet 1, 1106–1107. doi: 10.1016/s0140-6736(85)92413-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Bastani, A., and Jaberzadeh, S. (2012). A higher number of TMS-elicited MEP from a combined hotspot improves intra- and inter-session reliability of the upper limb muscles in healthy individuals. PLoS One 7:e47582. doi: 10.1371/journal.pone.0047582

PubMed Abstract | CrossRef Full Text | Google Scholar

Beaulieu, L.-D., Flamand, V. H., Masse-Alarie, H., and Schneider, C. (2017a). Reliability and minimal detectable change of transcranial magnetic stimulation outcomes in healthy adults: a systematic review. Brain Stimul. 10, 196–213. doi: 10.1016/j.brs.2016.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Beaulieu, L.-D., Masse-Alarie, H., Ribot-Ciscar, E., and Schneider, C. (2017b). Reliability of lower limb transcranial magnetic stimulation outcomes in the ipsi- and contralesional hemispheres of adults with chronic stroke. Clin. Neurophysiol. 128, 1290–1298. doi: 10.1016/j.clinph.2017.04.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Bhandari, A., Radhu, N., Farzan, F., Mulsant, B. H., Rajji, T. K., Daskalakis, Z. J., et al. (2016). A meta-analysis of the effects of aging on motor cortex neurophysiology assessed by transcranial magnetic stimulation. Clin. Neurophysiol. 127, 2834–2845. doi: 10.1016/j.clinph.2016.05.363

PubMed Abstract | CrossRef Full Text | Google Scholar

Cacchio, A., Cimini, N., Alosi, P., Santilli, V., and Marrelli, A. (2009). Reliability of transcranial magnetic stimulation-related measurements of tibialis anterior muscle in healthy subjects. Clin. Neurophysiol. 120, 414–419. doi: 10.1016/j.clinph.2008.11.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Cacchio, A., Paoloni, M., Cimini, N., Mangone, M., Liris, G., Aloisi, P., et al. (2011). Reliability of TMS-related measures of tibialis anterior muscle in patients with chronic stroke and healthy subjects. J. Neurol. Sci. 303, 90–94. doi: 10.1016/j.jns.2011.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Cavaleri, R., Schabrun, S. M., and Chipchase, L. S. (2017). The number of stimuli required to reliably assess corticomotor excitability and primary motor cortical representations using transcranial magnetic stimulation (TMS): a systematic review and meta-analysis. Syst. Rev. 6:48. doi: 10.1186/s13643-017-0440-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Classen, J., Knorr, U., Werhahn, K. J., Schlaug, G., Kunesch, E., Cohen, L. G., et al. (1998). Multimodal output mapping of human central motor representation on different spatial scales. J. Physiol. 512, 163–179. doi: 10.1111/j.1469-7793.1998.163bf.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Cuypers, K., Thijs, H., and Meesen, R. L. (2014). Optimization of the transcranial magnetic stimulation protocol by defining a reliable estimate for corticospinal excitability. PLoS One 9:e86380. doi: 10.1371/journal.pone.0086380

PubMed Abstract | CrossRef Full Text | Google Scholar

Darling, W. G., Wolf, S. L., and Butler, A. J. (2006). Variability of motor potentials evoked by transcranial magnetic stimulation depends on muscle activation. Exp. Brain Res. 174, 376–385. doi: 10.1007/s00221-006-0468-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Deng, Z. D., Lisanby, S. H., and Peterchev, A. V. (2013). Electric field depth-focality tradeoff in transcranial magnetic stimulation: simulation comparison of 50 coil designs. Brain Stimul. 6, 1–13. doi: 10.1016/j.brs.2012.02.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Deng, Z. D., Lisanby, S. H., and Peterchev, A. V. (2014). Coil design considerations for deep transcranial magnetic stimulation. Clin. Neurophysiol. 125, 1202–1212. doi: 10.1016/j.clinph.2013.11.038

PubMed Abstract | CrossRef Full Text | Google Scholar

Devanne, H., Lavoie, B. A., and Capaday, C. (1997). Input-output properties and gain changes in the human corticospinal pathway. Exp. Brain Res. 114, 329–338. doi: 10.1007/pl00005641

PubMed Abstract | CrossRef Full Text | Google Scholar

Dharmadasa, T., Matamala, J. M., Howells, J., Simon, N. G., Vucic, S., and Kiernan, M. C. (2019). The effect of coil type and limb dominance in the assessment of lower-limb motor cortex excitability using TMS. Neurosci. Lett. 699, 84–90. doi: 10.1016/j.neulet.2019.01.050

PubMed Abstract | CrossRef Full Text | Google Scholar

Farzan, F. (2014). “Single-pulse transcranial magnetic stimulation (TMS) protocols and outcome measures,” in Transcranial Magnetic Stimulation, eds A. Rotenberg, J. Horvath and A. Pascual-Leone (New York, NY: Humana Press), 69–115.

Google Scholar

Fisher, B. E., Lee, Y. Y., Pitsch, E. A., Moore, B., Southam, A., Faw, T. D., et al. (2013). Method for assessing brain changes associated with gluteus maximus activation. J. Orthop. Sports Phys. Ther. 43, 214–221. doi: 10.2519/jospt.2013.4188

PubMed Abstract | CrossRef Full Text | Google Scholar

Fleming, M. K., Sorinola, I. O., Newham, D. J., Roberts-Lewis, S. F., and Bergmann, J. H. (2012). The effect of coil type and navigation on the reliability of transcranial magnetic stimulation. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 617–625. doi: 10.1109/TNSRE.2012.2202692

PubMed Abstract | CrossRef Full Text | Google Scholar

Goetz, S. M., Kozyrkov, I. C., Luber, B., Lisanby, S. H., Murphy, D. L. K., Grill, W. M., et al. (2019). Accuracy of robotic coil positioning during transcranial magnetic stimulation. J. Neural Eng. 16:054003. doi: 10.1088/1741-2552/ab2953

PubMed Abstract | CrossRef Full Text | Google Scholar

Groppa, S., Oliviero, A., Eisen, A., Quartarone, A., Cohen, L. G., Mall, V., et al. (2012). A practical guide to diagnostic transcranial magnetic stimulation: report of an IFCN committee. Clin. Neurophysiol. 123, 858–882. doi: 10.1016/j.clinph.2012.01.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Hallett, M. (1996). Transcranial magnetic stimulation: a useful tool for clinical neurophysiology. Ann. Neurol. 40, 344–345. doi: 10.1002/ana.410400303

PubMed Abstract | CrossRef Full Text | Google Scholar

Hallett, M. (2007). Transcranial magnetic stimulation: a primer. Neuron 55, 187–199. doi: 10.1016/j.neuron.2007.06.026

PubMed Abstract | CrossRef Full Text | Google Scholar

Hopkins, W. G. (2000). Measures of reliability in sports medicine and science. Sports Med. 30, 1–15. doi: 10.2165/00007256-200030010-00001

PubMed Abstract | CrossRef Full Text | Google Scholar

Kiers, L., Cros, D., Chiappa, K. H., and Fang, J. (1993). Variability of motor potentials evoked by transcranial magnetic stimulation. Electroencephalogr. Clin. Neurophysiol. 89, 415–423. doi: 10.1016/0168-5597(93)90115-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Lewis, A. F., Myers, M., Heiser, J., Kolar, M., Baird, J. F., and Stewart, J. C. (2020). Test-retest reliability and minimal detectable change of corticospinal tract integrity in chronic stroke. Hum. Brain Mapp. 41, 2514–2526. doi: 10.1002/hbm.24961

PubMed Abstract | CrossRef Full Text | Google Scholar

Lewis, G. N., Signal, N., and Taylor, D. (2014). Reliability of lower limb motor evoked potentials in stroke and healthy populations: how many responses are needed? Clin. Neurophysiol. 125, 748–754. doi: 10.1016/j.clinph.2013.07.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, H., and Au-Yeung, S. S. (2014). Reliability of transcranial magnetic stimulation induced corticomotor excitability measurements for a hand muscle in healthy and chronic stroke subjects. J. Neurol. Sci. 341, 105–109. doi: 10.1016/j.jns.2014.04.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Lu, M., and Ueno, S. (2017). Comparison of the induced fields using different coil configurations during deep transcranial magnetic stimulation. PLoS One 12:e0178422. doi: 10.1371/journal.pone.0178422

PubMed Abstract | CrossRef Full Text | Google Scholar

Matheson, G. J. (2019). We need to talk about reliability: making better use of test-retest studies for study design and interpretation. PeerJ 7:e6918. doi: 10.7717/peerj.6918

PubMed Abstract | CrossRef Full Text | Google Scholar

Mead, G., Bernhardt, J., and Kwakkel, G. (2012). Stroke: physical fitness, exercise and fatigue. Stroke Res. Treat. 2012:632531. doi: 10.1155/2012/632531

PubMed Abstract | CrossRef Full Text | Google Scholar

Moscatelli, F., Messina, A., Valenzano, A., Monda, V., Salerno, M., Sessa, F., et al. (2021). Transcranial magnetic stimulation as a tool to investigate motor cortex excitability in sport. Brain Sci. 11:432. doi: 10.3390/brainsci11040432

PubMed Abstract | CrossRef Full Text | Google Scholar

Nunnally, J., and Bernstein, I. (1994). Psychometric Theory. New York: McGraw-Hill.

Google Scholar

Portney, L. G., and Watkins, M. P. (2000). “Statistical measures of reliability,” in Foundations of Clinical Research: Applications to Practice (Upper Saddle River, NJ: Prentice-Hall), 557–586.

Google Scholar

Portney, L. G., and Watkins, M. P. (2009). Foundations of Clinical Research Applications to Practice, 3rd edition. Upper Saddle River, NJ: Pearson Education Inc.

Google Scholar

Ridding, M. C., and Ziemann, U. (2010). Determinants of the induction of cortical plasticity by non-invasive brain stimulation in healthy subjects. J. Physiol. 588, 2291–2304. doi: 10.1113/jphysiol.2010.190314

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossi, S., Antal, A., Bestmann, S., Bikson, M., Brewer, C., Brockmoller, J., et al. (2021). Safety and recommendations for TMS use in healthy subjects and patient populations, with updates on training, ethical and regulatory issues: expert Guidelines. Clin. Neurophysiol. 132, 269–306. doi: 10.1016/j.clinph.2020.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossini, P. M., Burke, D., Chen, R., Cohen, L. G., Daskalakis, Z., Di Iorio, R., et al. (2015). Non-invasive electrical and magnetic stimulation of the brain, spinal cord, roots and peripheral nerves: basic principles and procedures for routine clinical and research application. An updated report from an I.F.C.N. Committee. Clin. Neurophysiol. 126, 1071–1107. doi: 10.1016/j.clinph.2015.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Rotenberg, A., Horvath, J. C., and Pascual-Leone, A. J. N. (2014). The transcranial magnetic stimulation (TMS) device and foundational techniques. Neuromethods 89, 3–13. doi: 10.1007/978-1-4939-0879-0_1

CrossRef Full Text | Google Scholar

Rothwell, J. C. (2011). Using transcranial magnetic stimulation methods to probe connectivity between motor areas of the brain. Hum. Mov. Sci. 30, 906–915. doi: 10.1016/j.humov.2010.07.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Rothwell, J. C., Thompson, P. D., Day, B. L., Boyd, S., and Marsden, C. D. (1991). Stimulation of the human motor cortex through the scalp. Exp. Physiol. 76, 159–200. doi: 10.1113/expphysiol.1991.sp003485

PubMed Abstract | CrossRef Full Text | Google Scholar

Shieh, G. (2016). Choosing the best index for the average score intraclass correlation coefficient. Behav. Res. Methods 48, 994–1003. doi: 10.3758/s13428-015-0623-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Sparing, R., Buelte, D., Meister, I. G., Paus, T., and Fink, G. R. (2008). Transcranial magnetic stimulation and the challenge of coil placement: a comparison of conventional and stereotaxic neuronavigational strategies. Hum. Brain Mapp. 29, 82–96. doi: 10.1002/hbm.20360

PubMed Abstract | CrossRef Full Text | Google Scholar

Terwee, C. B., Bot, S. D., de Boer, M. R., van der Windt, D. A., Knol, D. L., Dekker, J., et al. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. J. Clin. Epidemiol. 60, 34–42. doi: 10.1016/j.jclinepi.2006.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

van de Ruit, M., Perenboom, M. J., and Grey, M. J. (2015). TMS brain mapping in less than two minutes. Brain Stimul. 8, 231–239. doi: 10.1016/j.brs.2014.10.020

PubMed Abstract | CrossRef Full Text | Google Scholar

van Melick, N., Meddeler, B. M., Hoogeboom, T. J., Nijhuis-van der Sanden, M. W. G., and van Cingel, R. E. H. (2017). How to determine leg dominance: the agreement between self-reported and observed performance in healthy adults. PLoS One 12:e0189876. doi: 10.1371/journal.pone.0189876

PubMed Abstract | CrossRef Full Text | Google Scholar

Wassermann, E. M. (2002). Variation in the response to transcranial magnetic brain stimulation in the general population. Clin. Neurophysiol. 113, 1165–1171. doi: 10.1016/s1388-2457(02)00144-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Weir, J. P. (2005). Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J. Strength Cond. Res. 19, 231–240. doi: 10.1519/15184.1

PubMed Abstract | CrossRef Full Text | Google Scholar

Wheaton, L. A., Villagra, F., Hanley, D. F., Macko, R. F., and Forrester, L. W. (2009). Reliability of TMS motor evoked potentials in quadriceps of subjects with chronic hemiparesis after stroke. J. Neurol. Sci. 276, 115–117. doi: 10.1016/j.jns.2008.09.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Zrenner, C., Desideri, D., Belardinelli, P., and Ziemann, U. (2018). Real-time EEG-defined excitability states determine efficacy of TMS-induced plasticity in human motor cortex. Brain Stimul. 11, 374–389. doi: 10.1016/j.brs.2017.11.016

PubMed Abstract | CrossRef Full Text | Google Scholar