Reliability of phase-specific outcome measurements in change-of-direction tests using a motorized resistance device

This study aims to determine test-retest reliability of phase-specific information during initial acceleration, deceleration, and re-acceleration phases of different change-of-direction (CoD) tests using a motorized resistance device (MRD). A total of 21 participants (16 males and five females, with mean age of 22.3 ± 3.9 years, body mass of 75.2 ± 6.9 kg, height of 177.9 ± 6.8 cm) completed the modified 505 (m505), 10-0-5, and 15-0-5 CoD tests on four different test sessions while exposed to an external load (3 kg) provided by the MRD. Outcome variables included overall and phase-specific kinetic (force, power, and impulse) and kinematic (time, distance, velocity, and acceleration/deceleration) data during the initial acceleration, deceleration, and re-acceleration phases. The deceleration and re-acceleration phases were further divided into two subphases, namely, early and late subphases, using 50% of maximum velocity. Reliability was assessed using an intraclass correlation coefficient (ICC), coefficient of variation (CV), typical error (TE), and minimal detectable change (MDC). Good to excellent ICC values (>0.75) and acceptable (<10%) to good (<5%) CV values were observed for most outcome measurements. Specifically, 80.1% (822 out of 1,026) of all variables showed good or better relative reliability (i.e., ICC ≥ 0.75), while 97.0% (995 out of 1,026) of all variables showed acceptable or better absolute reliability (i.e., CV < 10%). In conclusion, the present study demonstrates that the MRD can obtain reliable phase-specific outcome measurements across different CoD tests, providing coaches and researchers with new opportunities to advance our understanding of CoD ability and inform more advanced CoD training prescriptions.

1. Introduction

Change of direction (CoD) is frequently performed in many field and court-based sports (1) and crucial to many decisive on-field performance actions such as preventing and creating goal scoring opportunities (2). Given such importance, the assessment and monitoring of a player's CoD performance is a routine part of many performance teams' testing battery. The CoD performance requires “rapid and systematically coordinated force application during the braking, plant and propulsive phases of the movement while maintaining optimal body positioning” (3). Therefore, horizontal acceleration and deceleration are fundamental locomotor skills underpinning performance in many CoD maneuvers. Consequently, CoD testing methods should quantify the different phases (initial acceleration, deceleration, and re-acceleration) to help advance the understanding of the CoD performance and to inform more specific CoD training prescription for players.

Currently, a myriad of different tests are used to quantify CoD based on different movement patterns (i.e., sprint and side shuffle), angle of turn(s), number of turns, and duration (4). These differences make comparisons between tests difficult as CoD is a task-specific skill based on the angle of turn and entry velocity (4, 5). Furthermore, in most CoD tests, overall time is often used as the primary outcome measurement, which has some inherent problems. Firstly, the overall time does not quantify the initial acceleration, deceleration, and re-acceleration phases. Secondly, longer tests are not possibly representative of CoD but rather anaerobic capacity and linear sprint ability (6). In fact, even in shorter tests, such as the modified 505 (m505) which consists of two 5 m sprints with a 180-degree turn, superior sprint capacity can still mask the CoD ability (4, 7). In an attempt to mitigate this shortcoming, indirect measures such as CoD deficit have been developed to better quantify and isolate the CoD component (8), which in essence is an attempt to quantify deceleration and re-acceleration. However, the CoD deficit does not enable accurate identification of CoD phases, which is vitally important considering that some athletes have been shown to pace their run-up (initial acceleration to deceleration) based on the demand of the CoD (9). This is also in agreement with anecdotal field-based observations of the authors and colleagues.

Based on the shortcomings of the CoD testing mentioned above, it has been advocated that the CoD tests should directly quantify what happens during the tests and provide continuous measurements (4). Specifically, measurements of the velocity of the center of mass (COM) during the CoD testing have also been advocated (4). Such measurements can be obtained in a laboratory setting (i.e., motion capture). However, this is not practical, and in many cases, not feasible for coaches and other practitioners in the applied setting. Field-based technologies such as global navigation satellite systems (GNSS), local positioning systems (LPS) (10, 11) and laser devices (12) have been used to obtain instantaneous velocity data. However, GNSS and LPS have a limited validity and reliability for short CoD tests (10, 11, 13) with the laser-based system demonstrating highly inconsistent phase-specific measurements during a 90-degree CoD (12).

A recent development of motorized resistance technology may provide an opportunity to obtain continuous outcome measurements of how athletes are moving during the CoD test in both laboratory and field-based environments (14). Specifically, a motorized resistance device (MRD) quantifies time, continuous position, and force exerted on the machine while performing a CoD action. Furthermore, load can be prescribed not only in an absolute manner but also in a phase-specific manner since loads in one phase (i.e., initial acceleration to deceleration) can be set to be different than in another phase (i.e., re-acceleration). Continuous velocity measurements of one MRD have recently been validated against a three-dimensional optical motion analysis system (14), but reliability has not been established to date. With both valid and reliable continuous outcome measurements for CoD tests, more detailed insights can be obtained to direct individualized training prescription. For example, more detailed insights into the deceleration phase, as introduced by Harper and co-workers, with data from a MRD can be explored (15). Furthermore, since momentum could have a significant effect on the CoD performance (16), continuous velocity measurements would allow for the exploration of change in momentum capabilities during CoD, as previously advocated by Nimphius and co-authors (4).

Accordingly, the aim of this study was to assess test–retest reliability of overall and phase-specific information (time, velocity, and distance) with specific deceleration and re-acceleration analysis of time, distance, velocity, acceleration/deceleration, force, power, and impulse for different CoD tests.

2. Methods

2.1. Subjects

A total of 16 male (age, 23.0 ± 3.7 years; body mass, 77.3 ± 6.8 kg; height, 179.9 ± 3.7 cm) and five female participants (age, 20.0 ± 0.0 years; body mass, 68.6 ± 3.7 kg; height, 171.4 ± 9.5 cm) with experience in soccer (n = 8), handball (n = 8), and floorball (n = 5) completed the study. 19 of the 21 participants completed all four test sessions, whereas one male and one female participant completed only two and three test sessions, respectively (due to the COVID-19 pandemic). Inclusion criteria were familiarity with ball sports CoD movements and no musculoskeletal injury or illness at time of testing that would prevent maximum effort for all test sessions. This study was approved by the Local Ethical Committee and the National Data Protection Agency for Research (reference number: 148213) and conducted in accordance with the Declaration of Helsinki. Prior to participation, all participants provided a written informed consent after being given detailed verbal and written explanation of the purpose, procedures, and risks associated with their participation.

2.2. Procedures

Anthropometric measurements (height and body mass) were obtained prior to a standardized warm-up, which included dynamic lower extremity mobility exercises, jogging (forward and backward), butt kicks, front kicks, high knee lifts, side shuffle, carioca, unilateral anterior–posterior and lateral jumps, three progressive sprints (80%, 90%–95% of subjective maximal) with the last sprint having assistance, and two m505 tests on each limb both with and without MRD. In total, the warm-up lasted approximately 25 min. The same warm-up was used for all four sessions, and all participants were instructed to standardize their training 2 days prior to testing. There were seven days (median, interquartile range: 7 days) between test sessions. All participants were tested on the same time of the day (morning or afternoon) based on the first test session using the same footwear.

All test sessions took place in an indoor sports hall at the Norwegian School of Sport Sciences with the order and loads used for the different tests being consistent across sessions. Specifically, three different CoD tests were performed with increasing approach distance (5, 10, and 15 m). The m505 test was performed first, then the 10-0-5 (105), and lastly the 15-0-5 (155, the same as the traditional 505 test). During these three tests, the participants performed two successful trials on each limb in an alternate order (e.g., left, right, left, right). Thus, a total of 12 tests (m505L, m505R, 105L, 105R, 155L, and 155R) were performed. The procedures of the m505 test under loaded conditions were previously described in detail (14) but were summarized here for clarity as additional tests were used. A line defining the turning point was 15 cm wide and marked with three cones on each side of a 1.2-m-wide corridor. From the turning point, cones were placed at a distance of 5, 10, and 15 m to define the starting points (Figure 1). All tests were performed under externally loaded conditions provided by an MRD with an assisted start, which meant that the first phase (1a) was sprinting toward, while the second phase (1b) was sprinting away from the MRD past the 5 m mark. The same start position (two-point start) was used at a distance of 5, 10, and 15 m away from the turning point. The fiber cord from the MRD was attached to the participant using a carabiner onto a pulley (Cyclone 52, Purmotion, USA), which in turn was attached to a belt with two carabiners (1080 Vest and 1080 MAP AS, Oslo, Norway, USA). When turning off the left foot, the carabiners were attached over the right hip and for right foot turns vice versa. This was to ensure that the fiber cord from the resistance device was not in conflict with the CoD movement. As the initial acceleration was toward the MRD, a greater demand was placed on both the deceleration and re-acceleration. Left and right foot turns were performed in a randomized order and maintained for all test sessions. A successful trial was defined as a full effort with the final foot contact hitting the 15-cm-wide line between the cones defining the turning point. A minimum of 2-min rest period was given between trials. The trial with the best overall time from each limb during each CoD test was used for further analysis.

FIGURE 1

Figure 1. Laboratory set-up with placement of motorized resistance device and position of starting lines and CoD zone for the m505, 105, and 155 tests.

2.3. Equipment

A portable MRD (1080 Sprint; 1080 Motion, Lidingö, Sweden) was used to provide external resistance and measure time, distance as well as average and maximum velocity (V_avg and V_max), pulling force (F_avg and F_max), and power (P_avg and P_max). The 1080 Sprint has a servo motor (2,000 RPM OMRON G5 Series Motor; OMRON Corp., Kyoto, Japan) that is attached to a carbon fiber spool around which a fiber cord is wrapped. The device was positioned on a table at a height of 75 cm to approximately align with the hip height of the participants. Both assisted and resisted loads were set to 3 kg. The auto-start function of the MRD was used (onset of measurement with a speed of >0.2 m·s⁻¹) (17).

2.4. Data analysis

All CoD tests were quantified based on time and velocity. Specifically, CoD tests were divided into phase 1a (initial acceleration to deceleration) and phase 1b (re-acceleration), based on when the velocity changed direction (V₀) (14). Consequently, the overall time and phase times (1a and 1b) as well as the maximum and average velocity of phases 1a and 1b were measured.

In order to provide a more detailed description of phase 1a, the deceleration phase was analyzed based on the methods first described by Harper and co-authors (15) but summarized here for clarity. Similar to the radar device software used in the study by Harper et al. (2020), a fourth-order two-way Butterworth low-pass filter with cut-off frequency of 1.5 Hz was used. Specifically, data on the position, velocity, and force were filtered in this manner, while acceleration was calculated from the filtered velocity with a finite difference using a five-point stencil using MATLAB R2021a (The MathWorks Inc., Natick, MA, USA). From the filtered velocity, the maximum velocity (V_max) during phase 1a was identified. The deceleration phase was then defined from V_max to when the velocity changed direction V₀. This phase was then further divided into early deceleration phase [V_max to 50% V_max (V₅₀)] and late deceleration phase (V₅₀ to V₀). During these two phases, different time and distance variables were quantified. Then, based on the change in velocity during the deceleration phase, the pulling force exerted on the participant from the MRD, air resistance (based on the surface area from height measurement), 19-degree centigrade and atmospheric pressure at sea level, force, power, and impulse were calculated for the overall, early, and late deceleration phases. For a description of deceleration phases, see Figure 2.

FIGURE 2

Figure 2. Phase definitions with figures (top row) of the m505, 105, and 155 tests. The table in the bottom rows provides an overview of the phases and subphases with abbreviations used for the outcome measurements and definitions..

Contrary to the methods used by Harper and co-authors (15) where only the deceleration phase was used for analysis, we also analyzed the re-acceleration phase using the same methods as described for the deceleration phase. Specifically, based on the same filtered data, we identified V_{max_finish}—defined as V₀ to the finish line (i.e., end of test and thus measurements). As an athlete will not reach their maximum velocity during a 5-m acceleration, the point of the maximum velocity for the participants during the re-acceleration coincided with the end of the test if the athlete continues to maximally accelerate throughout the re-acceleration phase. Then, the re-acceleration phase was divided into two phases, namely, early phase and late phase. The early phase was defined from V₀ to reaching 50% of V_{max_finish} (V₅₀), while the late re-acceleration phase was defined from V₅₀ to V_{max_finish}. The filtered instantaneous velocity–time data were used to calculate force, power, and impulse for both the early and late re-acceleration and deceleration phases. The re-acceleration and deceleration phases are illustrated in Figure 2, while the complete summary of all phase-specific outcome measurements is presented in Table 1.

TABLE 1

Table 1. Descriptions and definitions of the outcome variables.

2.5. Statistical analysis

The statistical analysis was performed using JASP (JASP version 0.16.3.0, Amsterdam, the Netherlands), Statistical Package for Social Sciences (SPSS version 24.0, IBM Corp., Armonk, NY, USA), jamovi (The jamovi project, 2021, version 2.3.6), and RStudio (RStudio Team, 2022). Descriptive data were calculated in Microsoft Excel (version 2209, Microsoft Corp., Redmond, WA, USA) and were presented as mean and standard deviation (SD). Normality of data was assessed using Shapiro–Wilk's test (p < 0.05) and qualitatively by visual inspection of Q–Q plots. Furthermore, potential outliers in the data were also explored using a web-based outlier calculator (StatsKingdom, 2017) based on the Tukey's fences method, where k was set at 1.5 and indicative of a “regular outlier” (18).

To explore potential differences and learning effect between test sessions in an overall performance and the phase-specific outcome measurements, a repeated measures analysis of variances (RM ANOVAs) was conducted. Alternatively, the Friedman test was performed in cases where the data did not suffice to the assumption of normality. With many variables and therefore many tests performed, the p-values from the RM ANOVAs/Friedman tests were corrected with Holm's sequential Bonferroni procedure (19) by using a Microsoft Excel calculator (20) to keep the nominal alpha level at the desired level and avoid type I errors, as recommended by Wagoner (21, 22). The level of statistical significance for the resulting adjusted p-values was set at p = <0.10.

The assumption of sphericity for the RM ANOVA tests was tested with Mauchly's test of sphericity. If the test indicated that the assumption had been violated (p-value of <0.05), then a correction to the original unadjusted p-value (“sphericity assumed”) was applied and subsequently used for the analysis. Based on recommendations, if the Greenhouse–Geisser epsilon value (ε) from Mauchly's test of sphericity was below 0.75, then a Greenhouse–Geisser correction was applied to the uncorrected p-value. Conversely, if the value was ≥0.75, then the Huyndt–Feldt correction was applied (23–25). If a statistically significant difference was found (i.e., p = <0.10), then a correction was applied to the original unadjusted p-value irrespective of whether or not a violation was indicated by Mauchly's test of sphericity, as has been recommended (23, 24). Furthermore, it is also important to remark that a correction was only applied if the difference was statistically significant (i.e., uncorrected p-value = <0.10) in the first place, since a correction in this case will only make a large and non-significant p-value even larger and hence increase our change of making a type II error (23, 26).

An effect size was calculated using both omega squared and Kendall's W test. Specifically, the omega-squared effect size with a 95% confidence interval (CI) was presented for each RM ANOVA and was calculated using the MOTE Effect Size Calculator app from DOOM Lab (27). Omega-squared values were interpreted in the following manner (24, 28): 0.01 = “small,” 0.06 = “moderate,” and >0.14 = “large.” For the Friedman tests, the measure of effect size was given using the Kendall's W test, where based on guidelines (29) the values of ≤0.1–0.3, ≤0.3–0.5, ≤0.5–0.7, ≤0.7–0.9 and ≤0.9 were interpreted with indications of “very weak agreement”, “weak agreement”, “moderate agreement”, “strong agreement” and “unusually strong agreement”, respectively.

An RM ANOVA or Friedman's test states only whether there is a statistically significant difference somewhere between the testing sessions. Accordingly, if a statistically significant effect was found, then a post hoc pairwise comparison was performed to evaluate the statistically significant difference between the test sessions. These were performed on each outcome variable, between each pair of test sessions. For example, test session 1 was compared with test sessions 2, 3, and 4; session 2 was compared with test sessions 3 and 4; and session 3 was compared with session 4. This signified that for one variable, a total of six comparisons were made. Thus, to keep the nominal alpha level at 0.05 and to avoid type I errors, the Holm–Bonferroni correction for multiple comparisons (19) was applied to the p-values and used in JASP to adjust for a family of six estimates. Similarly, the 95% CIs for the mean differences given by JASP are also corrected for multiple comparisons using the Bonferroni method. In the case of non-normal data, and if a statistically significant differences based on the Friedman test was found, then the post hoc pairwise comparisons were performed with the Wilcoxon matched-pairs test. Accordingly, in this case, the result was presented with the Hodges–Lehmann estimate with a 95% CI as the location parameter, z-test statistic value, and p-value from the Wilcoxon test. Based on the discussion mentioned above, the p-values from the Wilcoxon tests were also corrected with Holm's sequential Bonferroni procedure in JASP, adjusting for a family of six estimates.

As it is recommended to report both unstandardized and standardized effect sizes (30), the standardized Hedges'g average (Hedges'g_av) was calculated with a 95% CI and provided on all pairs of variables that satisfied the assumption of normality. Magnitude of Hedges'g_av values was interpreted according to the guidelines by Hopkins (31, 32): ≤0.19 = “trivial,” 0.20–0.59 = “small,” 0.60–1.19 = “moderate,” 1.20–1.99 = “large,” 2.00–3.99 = “very large,” and ≥4.00 = “extremely large.”

Reliability statistics were obtained using a custom reliability spreadsheet (33). The relative reliability was assessed using the intraclass correlation coefficient (ICC) with a 95% CI. Specifically, the ICC were calculated based on the ICC (3,1) model, referred to as “two-way mixed effects, consistency, single rater/measurement” (34). Of note, this ICC model included only random error in the calculation and does not consider the systematic error. Hopkins noted that the ICC (3,1) model was preferable, as it was the “observed correlation between measurements in two real-life trials” (33). The ICC values were interpreted based on the guidelines presented by Koo and Li (34), where values of <0.50, 0.50–0.75, 0.75–0.90, and >0.90 were interpreted as poor, moderate, good, and excellent reliability, respectively.

The absolute reliability of outcome measurements was assessed using the typical error (TE), both in the raw unit of measurement and relative to the mean based on the log-transformed data (CV—%) with 95% CIs. The CV values of >15%, 10%–15%, 5%–10%, and <5% were used with indications of very poor, poor, acceptable, and good absolute reliability, respectively. Furthermore, to calculate what can be considered as a “real” change in the performance for each respective variable—that is, with a given level of confidence, a change that is outside the ranges of the typical variation (measurement error) of the test and therefore sufficient to be regarded as a “real” change or difference–estimation of the minimal detectable change (MDC) was calculated based on the TE with a 90% CI (MDC_90%). All reliability statistics and MDC_90% were calculated for each session-to-session comparison.

3. Results

The descriptive data for the m505, 105, and 155 tests were presented in Tables 2–4, respectively.

TABLE 2

Table 2. Phase-specific outcome measurements of the m505 test for all sessions.

TABLE 3

Table 3. Phase-specific outcome measurements of the 10-0-5 test for all sessions.

TABLE 4

Table 4. Phase-specific outcome measurements of the 15-0-5 test for all sessions.

Overall, 80.1% (822 out of 1,026) of all variables showed good or better relative reliability. Specifically, 50.7% (520 out of 1,026) showed excellent, 29.4% (302 out of 1,026) showed good, 14.2% (146 out of 1,026) showed moderate, and 5.7% (58 out of 1,026) of all variables showed poor relative reliability. For the absolute reliability, 97.0% (995 out of 1,026) of all variables showed acceptable or better absolute reliability. Specifically, 58.9% (604 out of 1,026) showed good, 38.1% (391 out of 1,026) showed acceptable, 2.7% (28 out of 1,026) showed poor, and 0.3% (3 out of 1,026) of the variables showed very poor absolute reliability.

3.1. M505 test

For the m505L test (Table 5), when combining the three comparisons, 87.7% (150 out of 171) of the variables displayed good or better relative reliability, with ICC values ranging from 0.18 to 0.99. Specifically, 50.3% (86 out of 171) of all variables displayed excellent, 37.4% (64 out of 171) displayed good, 8.8% (15 out of 171) displayed moderate, and 3.5% (6 out of 171) displayed poor relative reliability. With regard to CV values, 98.3% (168 out of 171) of all the variables had acceptable or better absolute reliability with a range from 0.8 to 16.0%. Specifically, 57.9% (99 out of 171) of all variables displayed good, 40.4% (69 out of 171) displayed acceptable, 1.2% (2 out of 171) displayed poor, and 0.6% (1 out of 171) displayed very poor absolute reliability. Furthermore, the CV values for the comparison of sessions 1–2 ranged from 0.8 to 16.0 (32/57 good), while the CV values for the comparison of sessions 3–4 ranged from 0.8 to 10.1 (32/57 good).

TABLE 5

Table 5. Test–retest reliability of overall and phase-specific outcome measurement for the m505 test with left foot ultimate step.

For the m505R test (Table 6), when combining all three comparisons, 76.6% (131 out of 171) of all the variables displayed good or better relative reliability, with ICC values ranging from 0.31 to 0.996. Specifically, 46.8% (80 out of 171) of all variables displayed excellent, 29.8% (51 out of 171) displayed good, 19.3% (33 out of 171) displayed moderate, and 4.1% (7 out of 171) displayed poor relative reliability. With regard to the CV values, 99.4% (170 out of 171) of all variables had acceptable or better absolute reliability, with a range from 0.7 to 15.7%. Specifically, 50.3% (86 out of 171) of all variables displayed good, 49.1% (84 out of 171) displayed moderate, and 0.6% (1 out of 171) displayed very poor absolute reliability. Furthermore, the CV values for the comparison of sessions 1–2 ranged from 1.3 to 15.7 (25/57 good), while the CV values for the comparison of sessions 3–4 ranged from 0.7 to 10.0 (31/57 good).

TABLE 6

Table 6. Test–retest reliability of overall and phase-specific outcome measurement for the m505 test with the right foot ultimate step.

3.2. 105 test

For the 105L test (Table 7), when combining all three comparisons, 77.2% (132 out of 171) of all variables displayed good or better relative reliability, with ICC values ranging from 0.34 to 0.99. Specifically, 50.9% (87 out of 171) of all variables displayed excellent, 26.3% (45 out of 171) good, 17.0% (29 out of 171) displayed moderate, and 5.6% (10 out of 171) displayed poor relative reliability. With regard to the CV values, 95.3% (163 out of 171) of all variables had acceptable or better absolute reliability, with a range from 0.7 to 12.2%. Specifically, 57.3% (98 out of 171) of all variables displayed good, 38.0% (65 out of 171) displayed acceptable, and 4.7% (8 out of 171) displayed poor absolute reliability. Consequently, none of the variables displayed very poor absolute reliability. Furthermore, the CV values for the comparison of sessions 1–2 ranged from 0.8 to 12.2 (27/57 good), while the CV values for the comparison of sessions 3–4 ranged from 0.7 to 8.2 (41/57 good).

TABLE 7

Table 7. Test–retest reliability of overall and phase-specific outcome measurement for the 10-0-5 test with the left foot ultimate step.

For the 105R test (Table 8), when combining all three comparisons, 79.5% (136 out of 171) of all the variables displayed good or better relative reliability, with ICC values ranging from 0.28 to 0.99. Specifically, 48.5% (83 out of 171) of all variables displayed excellent, 31.0% (53 out of 171) displayed good, 18.1% (31 out of 171) displayed moderate, and 2.3% (4 out of 171) displayed poor relative reliability. With regard to the CV values, 97.1% (166 out of 171) of all the variables had acceptable or better absolute reliability, with a range from 0.8 to 13.2%. Specifically, 59.1% (101 out of 171) of all variables displayed good, 38.0% (65 out of 171) displayed acceptable, and 2.9% (5 out of 171) displayed poor absolute reliability. Consequently, and similar to the 105L test, none of the variables displayed very poor absolute reliability. Furthermore, the CV values for the comparison of sessions 1–2 ranged from 1.2 to 11.8 (30/57 good), while the CV values for the comparison of sessions 3–4 ranged from 0.8 to 13.2 (36/57 good).

TABLE 8

Table 8. Test–retest reliability of overall and phase-specific outcome measurement for the 10-0-5 test with the right foot ultimate step.

3.3. 155 test

For the 155L test (Table 9), when combining all three comparisons, 84.2% (144 out of 171) of all the variables displayed good or better relative reliability, with ICC values ranging from 0.33 to 0.995. Specifically, 54.4% (93 out of 171) of all variables displayed excellent, 29.8% (51 out of 171) displayed good, 8.8% (15 out of 171) displayed moderate, and 7.0% (12 out of 171) displayed poor relative reliability. With regard to the CV values, 96.5% (165 out of 171) of all the variables had acceptable or better absolute reliability, with a range from 0.5 to 16.9%. Specifically, 70.8% (121 out of 171) of all variables displayed good, 25.7% (44 out of 171) displayed acceptable, 2.9% (5 out of 171) displayed poor, and 0.6% (1 out of 171) displayed very poor absolute reliability. Furthermore, the CV values for the comparison of sessions 1–2 ranged from 0.8 to 16.9 (39/57 good), while the CV values for the comparison of sessions 3–4 ranged from 0.5 to 8.1 (41/57 good).

TABLE 9

Table 9. Test–retest reliability of overall and phase-specific outcome measurement for the 15-0-5 test with the left foot ultimate step.

For the 155R test (Table 10), when combining all three comparisons, 75.4% (129 out of 171) of all the variables displayed good or better relative reliability, with ICC values ranging from 0.13 to 0.99. Specifically, 53.2% (91 out of 171) of all variables displayed excellent, 22.2% (38 out of 171) displayed good, 13.5% (23 out of 171) displayed moderate, and 11.1% (19 out of 171) displayed poor relative reliability. With regard to the CV values, 95.3% (163 out of 171) of all the variables had acceptable or better absolute reliability, with a range from 0.6 to 14.0%. Specifically, 57.9% (99 out of 171) of all variables displayed good, 37.4% (64 out of 171) displayed acceptable, and 4.7% (8 out of 171) displayed poor absolute reliability. None of the variables displayed very poor absolute reliability (i.e., a CV value of ≥15%). Furthermore, the CV values for the comparison of sessions 1–2 ranged from 0.7 to 12.9 (29/57 good), while the CV values for the comparison of sessions 3–4 ranged from 0.6 to 13.4 (42/57 good).

TABLE 10

Table 10. Test–retest reliability of overall and phase-specific outcome measurement for the 15-0-5 test with the right foot ultimate step.

3.4. Learning effect

Overall, the results from the RM ANOVAs/Friedman tests exhibited a statistically significant difference (defined as an adjusted p-value of <0.10) between the test sessions for 11 variables out of a total of 342 variables, based on the Holm–Bonferroni adjusted p-values (results not shown). Out of the 11 statistically significant variables (Table 11), 7 belonged to m505L, 2 to m505R, 1 to 105L, and 1 to 105R. Thus, no statistically significant results were found for any variables from the 155 tests. Test statistics [F-value with corresponding degrees of freedom, adjusted p-value, and omega-squared effect size (ω²) with a 95% CI] from each RM ANOVA are presented in Table 11 in the left column below their given variable name. However, based on MDC_90% comparisons, no “real” differences were found between the sessions for the 11 identified variables. Of note, only test statistics from the RM ANOVAs are presented in Table 11 as none of the Friedman tests demonstrated a statistically significant difference.

TABLE 11

Table 11. Post hoc pairwise comparison of variables where a statistically significant result was found from the RM ANOVA/Friedman tests.

4. Discussion

This study aims to explore the test–retest reliability, MDC, and differences between the test sessions of phase-specific outcome measurements of CoD tests using an MRD. Overall, the values of test–retest relative and absolute reliability observed were mostly good to excellent and mostly acceptable to good, respectively. These findings were similar or better than those presented using other technology and slightly different tests (12, 15). Based on the MDC from the reliability statistics, no learning effects were observed between the sessions for the different outcome measurements.

The phase-specific analysis applied to CoD tests in the present study was based on the work of Harper and co-authors (15). In their study, they analyzed a maximum horizontal deceleration following a maximal 20-m sprint using a radar technology. Therefore, the players in the study by Harper and co-authors reached higher speeds prior to commencing deceleration than those observed in the present study. Furthermore, in the study by Harper et al. (2020), the deceleration phase was followed by a backpedal and not a CoD whereby the athlete rotated their body to maintain a forward-facing motion like in the present study. The CV values for the deceleration phase in the study by Harper et al. (2020) ranged from acceptable (5.2%, average deceleration) to poor (20.4%, time-to-maximum deceleration) for intra-day reliability and acceptable (6.2%, maximum horizontal braking power) to poor (21.6%, average late horizontal braking power) for inter-day reliability (15). In the present study, almost all deceleration variables across all CoD tests had good to acceptable absolute reliability, which demonstrated the potential of MRD during various 180-degree CoD tests for assessing deceleration ability. Since good to acceptable levels of reliability were also demonstrated with both left and right foot turns across most deceleration-phase variables, inter-limb asymmetries based on the turning foot may also be explored.

It is interesting to note that a minimum of 31 of 57 outcome variables had a good absolute reliability for the comparison of sessions 3–4, with longer 155 CoD tests (41 and 42 variables) having more than the shorter m505 tests (32 and 31 variables). Specifically, phase 1a and initial acceleration (time, distance, and velocity) had consistently good absolute reliability across tests (Tables 5–10), while the overall deceleration and early deceleration outcome measurements (time, distance, velocity, acceleration, force, power, and momentum) had mostly good absolute reliability in the longer tests (155), with mostly acceptable values in the shorter tests (m505 and 105). On the contrary, no such differences were observed for the same outcome variables during the late deceleration phase with mostly acceptable absolute reliability across tests. Furthermore, both the overall and late re-acceleration phases had mostly good absolute reliability for time, distance, velocity, acceleration, force, power, and impulse except for maximum acceleration and force (acceptable absolute reliability). A similar pattern for early re-acceleration as for the late deceleration was observed with mostly acceptable absolute reliability for the different outcome measurements for all tests. Overall, it appeared that longer tests improved reliability, with late deceleration and re-acceleration not being impacted. This observation is possibly attributed to both greater distance and time available for possible pacing strategies in preparation for both the overall and early deceleration phases. However, this must be substantiated by future research. Nevertheless, apart from distance, acceptable reliability was observed during the late deceleration and re-acceleration phases. This possibly indicated that the participants chose slightly different late deceleration strategies between sessions, which in turn could influence early re-acceleration. See Tables 5–10 for details.

Phase-specific information has also been previously explored using two synchronized laser guns (12). In the study of Hader et al. (2015), the phase-specific information was explored using 45- and 90-degree CoD angles. This was different compared with the present study that used CoD tests involving 180-degree turns. Specifically, Hader and co-authors explored the reliability of maximum and distance-to-maximum acceleration, deceleration, and speed along with minimum speed and speed between 8 m and 12 m that represented the distance 2 m prior and 2 m after the CoD. The CV values for speed around the 8–12 m distance were ∼5%, and the CV values for phase-specific information ranged from 6.6% to 8.5% for peak acceleration, while peak and distance-to-peak deceleration ranged from 117% to 12.6%. Thus, the results observed for both speed and acceleration in their study were comparable with the present results, while a better absolute reliability was observed for deceleration-specific outcome measurements (good to acceptable vs. poor to very poor) in the present study (12). However, it is important to note that Hader and co-workers explored intra-session reliability, while inter-session reliability was explored in the current study making values not directly comparable. Nevertheless, having reliable assessment of deceleration specific outcome measurements is important considering the importance of deceleration in team and individual multi-directional speed sports (35).

Similar to the study of Harper et al. (2020), the protocols adopted in the present study permitted a more detailed analysis of the deceleration phase than that previously investigated during the CoD maneuvers. This could have a notable importance, since the deceleration phase has been associated with a superior CoD performance and major injuries such as anterior cruciate ligament ruptures in many multi-directional sports (34, 37). In the study by Harper and co-authors (15), the average V_max defining the start of the deceleration phase was between 7.19 m·s⁻¹ and 7.36 m·s⁻¹, which was most comparable with the values observed during the 155 test (6.78 m·s⁻¹), but not to those observed during neither the m505 (4.46 m·s⁻¹) nor the 105 test (5.94 m·s⁻¹). Thus, the comparisons with the deceleration outcome measurements were most appropriate for the 155 tests. Specifically, the overall and early deceleration outcome measurements (deceleration, force, and power) were slightly higher in magnitude in the present study compared with those presented by Harper and co-authors (15). However, large differences in deceleration are observed during the late deceleration phase. Specifically, Harper and co-authors reported a Dec_avg during the late deceleration of 5.53 m·s⁻²−5.55 m·s⁻², while our present findings were 9.80 m·s⁻¹ and 10.26 m·s⁻¹ for the left and right foot turn, respectively, during session 4. This is an interesting finding which could be due to the differences in both distance and time for the deceleration phase. Harper and co-authors reported a longer distance (6.53 m−6.71 m), while 5.70 m and 5.66 m were observed in the present study. In addition, the participants used a shorter time in the deceleration phase in the present study (1.29 s and 1.32 s) compared with 1.47 s–1.49 s reported by Harper and co-authors (15). Overall, it appeared that the deceleration from a 20-m sprint, compared with performing a CoD test, provided a relatively lower demand on the late deceleration phase. One plausible reason for the late deceleration differences was possibly that a CoD task had a different intent in that one was to complete a course in the shortest amount of time with a clearly defined turning point, whereas in a maximum deceleration test, the focus was to maximally decelerate from a 20-m sprint, thereby removing the skill required to change direction. Furthermore, an assisted deceleration prior to a CoD, as provided by an external load with a MRD, placed increased demands on the late deceleration. In addition, it is also possible that decelerating to a stop while facing the same direction has a decreased deceleration demand compared with the use of a plant step in a 180-degree CoD whereby a more horizontally orientated body posture may facilitate braking and thus deceleration prior to the CoD. This may be particularly the case when using the assisted load. Nevertheless, these findings have crucial implications on how one possibly uses different deceleration tasks in both training and rehabilitation.

Performance times of the m505 test in the present study are comparable with those that are previously reported (8). Specifically, slightly higher times for the m505 test (3.01 s to 3.04 s) were observed compared with the unloaded m505 in professional rugby players (2.73 s) (38) and netball players (2.82 s–2.88 s) (39). Furthermore, slightly lower times were observed compared with the same loaded conditions using the MRD (3.26 s) (14). However, it is important to consider that the measurements in the present study were obtained using the MRD with a 0.2 m·s⁻¹ start trigger, which is possibly more sensitive to capturing the start of the test compared with the data obtained from the other studies using timing gates. Specifically, the same MRD device had been found to measure greater 0 m–5 m sprint split times compared with photocells with a bias of 0.33–0.35 s (17). Furthermore, it is important to consider that the external load provided by the MRD will impact the overall CoD test performance time through providing “assistance” and “resistance” during different phases of the CoD. For example, during the initial acceleration-to-deceleration phase (1a) and due to the positioning of the MRD, the athlete is “assisted” as they accelerate and then decelerate prior to CoD. In contrast, during the re-acceleration phase (1b), the athlete is faced with a resisted load that would essentially reduce whole-body acceleration. This will make both deceleration and re-acceleration more demanding. Furthermore, the inclusion of both males and females and different sports (soccer, handball, and floorball) improves the reliability to encompass different ball sports and gender.

4.1. Limitations and implications for future research

Both limitations and possibilities for future research have been highlighted in the present study. One limitation is that a total of 57 variables were assessed for each test in three different tests. The variables in the different phases of a CoD task that are essential to the performance need to be analyzed, as documenting or monitoring 57 variables is impractical. However, the reliability statistics of all 57 variables allows for future exploration of which variables are important and how different phases determine the CoD performance. Furthermore, it is possible that phase-specific variables possibly dictate what phases to target in individualized training, which in turn could lead to phase-specific interventions (i.e., deceleration). Then, phase-specific outcome variables can be used to determine if there was a “real” change in performance and which variables responded to different types of training. In addition, three different tests were further assessed that increased the number of variables included. The reason for exploring these three different tests was to manipulate the entry velocity and thereby the momentum to progressively increase the deceleration-phase demands. Considering that the limited deceleration assessments are currently available, it is possible that using more than one test to assess the deceleration capacity is possibly important in future CoD assessment. Another limitation is that only a 180-degree CoD was used for the analysis of the CoD performance. However, the 180-degree CoD tasks require the players to perform the maximum deceleration prior to re-acceleration permitting the evaluation of the initial acceleration, deceleration, and re-acceleration capabilities of the players. Nonetheless, a future research should consider the evaluation of phase-specific CoD information during the performance of other CoD tasks that require different angles of CoD. Another limitation is that the CoD tasks were only performed with an assisted start, which increases the demands on the deceleration and re-acceleration phases due to the direction of the external load provided by the MRD. Future research should explore different external loads for both and assisted and resisted start relative to the MRD to allow for a better generalizability of the present findings as well as to further assess how this may change the demands in different subphases of CoD (i.e., early and late deceleration). Moreover, testing was performed in an indoor sports hall environment. Obtaining data from an outdoor setting and from an artificial turf would have improved the generalizability since applications to the CoD testing and training using an MRD apply to those environments. However, many team and individual ball sports are performed in indoor environments, and we wanted to explore the reliability in this environment.

5. Conclusion

The phase-specific outcome measurements of different CoD tests obtained and calculated from the data captured by using the MRD are mainly reliable. The analysis yielded mostly good to excellent ICC values and mostly acceptable to good CV values for the test–retest reliability across multiple sessions. The test–retest reliability of different CoD tests provides coaches and researchers alike with new opportunities to further assess and understand this important athletic quality.

6. Practical applications

Our findings may influence not only the lab but also the field-based testing and training of CoD. In combination with the previously established validity of velocity measurements of the MRD (14), the present findings provide coaches with important information that was previously only available in a laboratory setting. Based on the valid and reliable information that can be obtained by the MRD, we now have an opportunity to obtain more in-depth insights into an athletes CoD performance that will enable more advanced athlete specific CoD training prescriptions.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Local Ethical Committee of the Norwegian School of Sport Sciences. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

OE and ØG were responsible for the original study design of the manuscript. FW, OE and HL contributed to the data collection. All authors contributed to the interpretation, and revision of the manuscript. All authors contributed to the article and approved the submitted version.

Acknowledgments

The authors would like to thank all participants that participated in this study. Your contribution was invaluable as without you this study would have not been possible.

Conflict of interest

OE is a shareholder in 1080 Motion AB and 1080 MAP AS.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Taylor JB, Wright AA, Dischiavi SL, Townsend MA, Marmon AR. Activity demands during multi-directional team sports: a systematic review. Sports Med. (2017) 47(12):2533–51. doi: 10.1007/s40279-017-0772-5

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Martinez-Hernandez D, Quinn M, Jones P. Linear advancing actions followed by deceleration and turn are the most common movements preceding goals in male professional soccer. Sci Med Footb. (2023) 1:25–33. doi: 10.1080/24733938.2022.2030064

CrossRef Full Text | Google Scholar

3. Dos'Santos T, Thomas C, Jones PA, Comfort P. Mechanical determinants of faster change of direction speed performance in male athletes. J Strength Cond Res. (2017) 31(3):696–705. doi: 10.1519/JSC.0000000000001535

CrossRef Full Text | Google Scholar

4. Nimphius S, Callaghan SJ, Bezodis NE, Lockie RG. Change of direction and agility tests: challenging our current measures of performance. J Strength Cond Res. (2017) 40(1):1–13. doi: 10.1519/SSC.0000000000000309

CrossRef Full Text | Google Scholar

5. Dos'Santos T, Thomas C, Comfort P, Jones PA. The effect of angle and velocity on change of direction biomechanics: an angle-velocity trade-off. Sports Med. (2018) 48(10):2235–53. doi: 10.1007/s40279-018-0968-3

CrossRef Full Text | Google Scholar

6. Vescovi JD, McGuigan MR. Relationships between sprinting, agility, and jump ability in female athletes. J Sports Sci. (2008) 26(1):97–107. doi: 10.1080/02640410701348644

PubMed Abstract | CrossRef Full Text | Google Scholar

7. DeWeese B, Nimphius S. Speed and agility program design and technique. In: Triplett NT, Haff GG, editors. Essentials of strength and conditioning. Champaign, IL: Human Kinetics (2016). p. 521–57.

8. Nimphius S, Callaghan SJ, Spiteri T, Lockie RG. Change of direction deficit: a more isolated measure of change of direction performance than total 505 time. J Strength Cond Res. (2016) 30(11):3024–32. doi: 10.1519/JSC.0000000000001421

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Nimphius S, McGuigan MR, Newton RU. Is there a pacing strategy during a 505 change of direction test in adolescents? J Strength Cond Res. (2013) 27:S104–S5.

Google Scholar

10. Luteberget LS, Gilgien M. Validation methods for global and local positioning-based athlete monitoring systems in team sports: a scoping review. BMJ Open Sport Exerc Med. (2020) 6(1):e000794. doi: 10.1136/bmjsem-2020-000794

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Meylan C, Trewin J, McKean K. Quantifying explosive actions in international women's Soccer. Int J Sports Physiol Perform. (2017) 12(3):310–5. doi: 10.1123/ijspp.2015-0520

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Hader K, Palazzi D, Buchheit M. Change of direction speed in soccer: how much braking is enough? Kinesiology. (2015) 47(1):67–74.

Google Scholar

13. Buchheit M, Samozino P, Glynn JA, Michael BS, Al Haddad H, Mendez-Villanueva A, et al. Mechanical determinants of acceleration and maximal sprinting speed in highly trained young soccer players. J Sports Sci. (2014) 32(20):1906–13. doi: 10.1080/02640414.2014.965191

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Eriksrud O, Ahlbeck F, Harper D, Gloersen O. Validity of velocity measurements of a motorized resistance device during change of direction. Front Physiol. (2022) 13:824606. doi: 10.3389/fphys.2022.824606

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Harper DJ, Morin JB, Carling C, Kiely J. Measuring maximal horizontal deceleration ability using radar technology: reliability and sensitivity of kinematic and kinetic variables. Sports Biomech. (2023) 22(9):1192–208. doi: 10.1080/14763141.2020.1792968

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Fernandes R, Bishop C, Turner AN, Chavda S, Maloney SJ. Train the engine or the brakes? Influence of momentum on the change of direction deficit. Int J Sports Physiol Perform. (2020) 16(1):90–6. doi: 10.1123/ijspp.2019-1007

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Rakovic E, Paulsen G, Helland C, Haugen T, Eriksrud O. Validity and reliability of a robotic resistance device. J Strength Cond Res. (2022) 36(8):2335–8. doi: 10.1519/JSC.0000000000003830

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Tukey JW. Exploratory data analysis. 1st ed. London: Pearson (1977).

19. Holm S. A scale of magnitudes for effect statistics. A new view of statistics. J Stat. (1979) 6(2):65–70.

Google Scholar

20. Gaetano J. Holm-Bonferroni sequential correction: an excel calculator (1.3). 1.3 ed. (2018). Available at: https://www.researchgate.net/publication/322569220_Holm-Bonferroni_sequential_correction_An_Excel_calculator_13

21. Wagoner KD, editor. Descriptive discriminant analysis: A follow-up procedure to a “significant” Manova. Denver, CO, USA: American Alliance for Health, Physical Education, Recreation and Dance Convention (1994).

22. Weir JP, Vincent WJ. Statistics in kinesiology. 5th ed. Campaign, IL: Human Kinetics (2021).

23. Atkinson G. Analysis of repeated measurements in physical therapy research. Phys Ther Sport. (2001) 2(4):194–208. doi: 10.1054/ptsp.2001.0071

CrossRef Full Text | Google Scholar

24. Field A. Discovering statistics using IBM SPSS statistics. 5th ed. Sage Publications Ltd. (UK) (2018).

25. Girden ER. ANOVA: Repeated measures (quantitative applications in the social sciences). 1st ed. Sage Publications, Inc. (1991).

26. O’Donoghue P. Statistics for sport and exercise studies—an introduction. London: Routledge (2012).

27. Buchanan E, Scofield J, Valentine KD. Mote: effect size and confidence interval calculator. 0.0.0.9100 ed: R (2017).

28. Kirk RE. Practical significance: a concept whose time has come. Educ Psychol Meas. (1996) 56(5):746–59. doi: 10.1177/0013164496056005002

CrossRef Full Text | Google Scholar

29. Schmidt RC. Managing delphi surveys using nonparametric statistical techniques. Decision Sciences. (1997) 28(3):763–74. doi: 10.1111/j.1540-5915.1997.tb01330.x

CrossRef Full Text | Google Scholar

30. Fritz CO, Morris PE, Richler JJ. Effect size estimates: current use, calculations, and interpretation. J Exp Psychol Gen. (2012) 141(1):2–18. doi: 10.1037/a0024338

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Hopkins WG. A scale of magnitudes for effect statistics. A new view of statistics. (2002). Available at: http://sportsci.org/resource/stats/effectmag.html

32. Hopkins WG, Marshall SW, Batterham AM, Hanin J. Progressive statistics for studies in sports medicine and exercise science. Med Sci Sports Exerc. (2009) 41(1):3–13. doi: 10.1249/MSS.0b013e31818cb278

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Hopkins WG. Spreadsheets for analysis of validity and reliability. Sportscience. (2015) 19:36–42.

Google Scholar

34. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. (2016) 15(2):155–63. doi: 10.1016/j.jcm.2016.02.012

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Harper DJ, McBurnie AJ, Santos TD, Eriksrud O, Evans M, Cohen DD, et al. Biomechanical and neuromuscular performance requirements of horizontal deceleration: a review with implications for random intermittent multi-directional sports. Sports Med. (2022) 52(10):2321–54. doi: 10.1007/s40279-022-01693-0

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Della Villa F, Buckthorpe M, Grassi A, Nabiuzzi A, Tosarelli F, Zaffagnini S, et al. Systematic video analysis of acl injuries in professional male football (soccer): injury mechanisms, situational patterns and biomechanics study on 134 consecutive cases. Br J Sports Med. (2020) 54(23):1423–32. doi: 10.1136/bjsports-2019-101247

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Takahashi S, Nagano Y, Ito W, Kido Y, Okuwaki T. A retrospective study of mechanisms of anterior cruciate ligament injuries in high school basketball, handball, judo, soccer, and volleyball. Medicine (Baltimore). (2019) 98(26):e16030. doi: 10.1097/MD.0000000000016030

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Gabbett TJ, Kelly JN, Speed SJ. Change of direction speed, and reactive agility of rugby league players. J Strength Cond Res. (2008) 22(1):174–81. doi: 10.1519/JSC.0b013e31815ef700

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Barber OR, Thomas C, Jones PA, McMahon JJ, Comfort P. Reliability of the 505 change-of-direction test in netball players. Int J Sports Physiol Perform. (2016) 11(3):377–80. doi: 10.1123/ijspp.2015-0215

PubMed Abstract | CrossRef Full Text | Google Scholar