Performance Variability During Motor Learning of a New Balance Task in a Non-immersive Virtual Environment in Children With Hemiplegic Cerebral Palsy and Typically Developing Peers

Background: Motor impairments contribute to performance variability in children with cerebral palsy (CP) during motor skill learning. Non-immersive virtual environments (VEs) are popular interventions to promote motor learning in children with hemiplegic CP. Greater understanding of performance variability as compared to typically developing (TD) peers during motor learning in VEs may inform clinical decisions about practice dose and challenge progression.

Purpose: (1) To quantify within-child (i.e., across different timepoints) and between-child (i.e., between children at the same timepoint) variability in motor skill acquisition, retention and transfer in a non-immersive VE in children with CP as compared to TD children; and (2) To explore the relationship between the amount of within-child variability during skill acquisition and learning outcomes.

Methods: Secondary data analysis of 2 studies in which 13 children with hemiplegic CP and 67 TD children aged 7–14 years undertook repeated trials of a novel standing postural control task in acquisition, retention and transfer sessions. Changes in performance across trials and sessions in children with CP as compared to TD children and between younger (7–10 years) and older (11–14 years) children were assessed using mixed effects models. Raw scores were converted to z-scores to meet model distributional assumptions. Performance variability was quantified as the standard deviation of z-scores.

Results: TD children outperformed children with CP and older children outperformed younger children at each session. Older children with CP had the least between-child variability in acquisition and the most in retention, while older TD children demonstrated the opposite pattern. Younger children with CP had consistently high between-child variability, with no difference between sessions. Within-child variability was highest in younger children, regardless of group. Within-child variability was more pronounced in TD children as compared to children with CP. The relationship between the amount of within-child variability in performance and performance outcome at acquisition, retention and transfer sessions was task-specific, with a positive correlation for 1 study and a negative correlation in the other.

Conclusions: Findings, though preliminary and limited by small sample size, can inform subsequent research to explore VE-specific causes of performance variability, including differing movement execution requirements and individual characteristics such as motivation, attention and visuospatial abilities.

Introduction

Cerebral palsy (CP) is the leading cause of physical disability in childhood (1–4). Unilateral spastic CP, or hemiplegia, is the most common subtype, representing 35–40% of new diagnoses (4, 5). Children with hemiplegia have motor, cognitive, sensory and perceptual challenges that limit postural control and activities of daily living, reducing functional independence (6–10). Assisting children to learn new motor skills, improve existing skills, and transfer skills to enhanced function in the real world is a primary goal of rehabilitation (11–13). However, much remains to be understood about motor learning impairments in children with CP (11–13).

Information-processing, attention, motor planning, and motor execution impairments can differ in children with CP as compared to typically developing (TD) peers, influencing the rate and extent of motor learning (12, 13). While children with CP improve in new motor task learning with practice (14–16), they may require greater duration of practice to achieve competency, while demonstrating lower accuracy and greater variability in task performance outcomes over repeated trials as compared to TD peers (14, 16–25). The heterogeneous nature of motor and cognitive impairments in CP allows for significant variability in performance outcomes in children of the same age and Gross Motor Function Classification System Level (24, 26–30).

Variability, traditionally conceptualized as the opposite of stability, is a fundamental characteristic of human performance (31, 32). Sternad defines variability as an umbrella term for “all sets or series of observations that are non-constant” (32). Variability is usually reduced with practice of a new motor skill. A prevalent view is that variability impedes the accuracy and precision required for skill attainment (31, 32). In contrast, some amount of variability may be beneficial to support the search for optimal solutions in differing task conditions (33–35). Indeed, Hadders-Algra defines variability as “the capacity to select from the repertoire the motor strategy that fits the situation best” (36). Ranganathan et al. (31) relate this view of variability to behavioral flexibility, which they define as “the ability to achieve the same task outcome using different movement solutions.” Whether adaptive or detrimental, variability can occur at the level of task performance (in performance outcomes) and at the level of movement execution (in kinematic strategies used to achieve the outcome). Exploring both within-child (e.g., variability across different timepoints) and between-child variability (e.g., variability between children at the same timepoint) at both task and movement levels is important to understand differences in children's responses to interventions (37).

Movement execution variability in children with CP is highly correlated with severity of motor impairment (38, 39). Children with CP may demonstrate more movement execution variability because of challenges suppressing normal intrinsic motor system noise (40, 41). Their motor learning impairments may also affect the formation of internal models of movement and the interpretation of feedback mechanisms that could reduce variability (42). Movement execution variability has been investigated in gait (40, 43) and speech kinematics (44–46) in this population. For example, children with CP have a higher stride to stride variability, with more variation in muscle synergies during walking (39). In speech kinematics, Chen et al. (45) found longer coefficients of variation of utterance duration for short speech tasks in children with CP as compared to TD children, with greater variability as task complexity increased. However, movement execution variability in children with CP can decrease with training. For example, interventions in which children adapt to different gait speeds in each leg using a split-belt treadmill can decrease stride to stride variability in children with hemiplegic CP in ways that are significantly correlated with learning improvements (41).

Non-immersive virtual environments (VEs) in which children use body movements to interact with virtual objects displayed on a 2-dimensional (2D) flat-screen display are pediatric rehabilitation interventions that can support motor skill improvement and motor learning (47). There is strong evidence for the effectiveness of VE-based interventions to improve upper extremity functioning (48–51) and postural control (52–54) in children with hemiplegic CP. The unique practice conditions of non-immersive VEs may impact movement execution and performance variability. For example, interactions with virtual objects involve differing perceptual-motor affordances as compared to interaction with objects in the physical environment (55, 56). A lack of 3D depth cues in a non-immersive VE influences distance estimates of where objects are in space, which may increase uncertainty about movement accuracy. In addition, hand-held peripheral controllers that track movement (such as the one required by the Nintendo Wii or the HTC VIVE) may influence task interaction (56) as opposed to direct motion tracking.

Several studies have explored changes in movement execution variability in children with CP who learn a seated reaching task in a non-immersive VE as compared to in a physical environment. Children reduce their movement execution variability in repeated training in non-immersive VE, with some kinematic improvements transferring to improved performance of the same movement in the physical environment (57). Robert et al. (16) undertook reach-to-grasp training in both physical and 2D flat-screen VEs in children with CP, finding similar improvements in kinematic variables in both training groups. Robert and Levin (58) compared reaching movement kinematics in 2D virtual reality and physical environment in typically developing children and children with CP. Several kinematic variables differed between reaches in the VE and the physical environment for children with CP, with only small clinically insignificant differences between CP and TD children. Overall, children moved more slowly in the VE. When children with CP use a hand-held game controller to interact with a non-immersive VE, they demonstrate both within- and between-child variability in terms of upper-extremity movement patterns used to play a single 2D active video game (59, 60). Some of this variability may be explained by personal and predisposing factors, such as gender, experience with video game play, and upper extremity impairment level (52).

Less is known about task performance variability in non-immersive VEs. Exploring this issue is important to contribute to the ongoing discussion about variability as both an adaptive and detrimental characteristic of motor learning (31, 32). Greater knowledge about within- and between-child variability in performance can provide new information relevant to conclusions about intervention effectiveness and inform sample size considerations for clinical trials. With greater understanding of variability in new task learning in non-immersive VEs, we can better guide therapists who endeavor to adhere to motor learning principles underlying experience-dependent neuroplasticity in rehabilitative strategies (61). For example, decisions about practice dosage, amount of repetition, and timing of progression of difficulty and challenge levels are often made on the basis of consistent performance improvements (i.e., a lack of variability). Understanding children's variability in performance over time is important because non-immersive VEs are often used as home intervention programs (62) and are not directly supervised by therapists; therefore, they cannot observe children's performance to understand how they perform over repeated trials.

The purpose of this study is to (1) Quantify within- and between-child performance variability in motor skill acquisition, retention and transfer in a non-immersive VE in children with hemiplegic CP and TD children; and (2) Explore the relationship between the amount of within-child variability during skill acquisition and retention performance. We hypothesize that children with CP will demonstrate greater between- and within-child variability than TD children, that younger children will demonstrate greater variability as compared to older children, and that variability will differ by learning stage and demonstrate a relationship with learning outcomes.

Study Design and Methods

We undertook a secondary data analysis of 2 studies undertaken in our lab in which children with hemiplegic CP at Gross Motor Function Classification System (GMFCS) Levels I and II and typically developing children acquired one of 2 new balance skills in a non-immersive VE [the Stability and Balance Learning Environment (STABLE; Motekforce Link, The Netherlands), a 130 degree projection flat-screen VE in which interaction is via a force plate and motion capture cameras]. Forty-seven children participated in Study 1 and 33 children participated in Study 2. All children undertook baseline postural control tests (eyes closed stance, single leg stance, tandem stance, and mediolateral and anteroposterior limits of stability) on the STABLE prior to beginning the task. In both Study 1 and Study 2, children practiced a task requiring them to move their center of pressure (CoP) within a static base of support to control a virtual avatar (Acquisition). Children used CoP movements to control the avatar to follow a predetermined path displayed in the non-immersive VE as closely as possible. In Study 1 (Figure 1), the avatar moved along a path in a first-person perspective such that view of the path in the VE emerged according to the children's movements. In contrast, in Study 2 (Figure 2); the full path was always visible to the child in a 3rd person perspective. In both studies, VE visual and auditory feedback changed according to children's movements. The tasks are described in more detail in (63, 64). Acquisition involved 20 trials of practice; children returned 2–7 days after acquisition for a retention and transfer session, in which they performed the task in the same condition as acquisition (Retention; 10 trials) and in a more motorically-challenging condition (Transfer; 10 trials).

FIGURE 1

Figure 1. Displays the Study 1 virtual environment, showing the path emerging in front of the participant (the white dots).

FIGURE 2

Figure 2. Displays the Study 2 virtual environment, showing the entire path visible to the participant (the blue line).

Analyses

Changes in performance score across trials and sessions, as well as differences in performance between children with CP and TD children, and children differing in age (7–10 vs. 11–14 years), were assessed using mixed effects models via the lme4 package in R version 3.6.0. Raw performance scores were converted to z-scores to more closely meet distributional assumptions of the models. A z-score of 0 represents mean performance. Model selection was conducted using an information-theoretic approach based on Akaike's Information Criterion (AIC) (65) [as described in (66)]. Briefly, a set of models is generated based on explanatory variables and higher-order effects (e.g., interactions and/or polynomial terms) of interest. A reduced set of explanatory variables is selected, to prevent overfitting and determine the most important effects in the model. The reduced model is selected using AIC, in which models are ranked in increasing order by AIC values, which are then used to calculate “Akaike weights” for each model. These are commonly interpreted as the probability that the given model is the “best” in the set in terms of minimizing loss of Kullback-Leibler (66) information, providing a straightforward means of comparing relative model fits.

Models in our initial comparison set included both least-square and mixed effects implementations. The full set of explanatory variables and higher-order effects tested included trial number (linear only vs. second-order polynomial), group (TD vs. CP), age group (7–10 vs. 11–14 years), and interactions between group and both age group and the polynomial term for trial. For the top selected models, study was also added as an explanatory variable, as both a main effect and interaction with group and age group, in order to quantify the size of the difference between studies compared with other sources of variation.

Mixed models initially included random parameters for both slope and intercept across trials for each subject. However, for models that included random slope parameters, the numerical search method failed to converge on a maximum-likelihood solution, likely as a result of our relatively small sample size, particularly for the CP group. Mixed-effects implementations of our comparison models therefore only included a random intercept parameter. Differences in mean z-scores among sessions were examined separately using a mixed-effects model that included group, age group, and session as fixed effects, and a random intercept term for subject. Pair-wise differences between groups were examined using Tukey-adjusted post-hoc comparisons of estimated marginal means in the “emmeans” package in R.

Because AIC and statistics derived from it only assess relative fit among different models, absolute model fit was also assessed using R² values calculated from model deviances using the r.squaredGLMM command from the MuMIn package in R. This generates a “marginal” R² which expresses the variance explained by the fixed effects only, as well as a “conditional” R² that reflects the variance explained by the whole model (fixed + random factors).

Between-child variability was quantified as the standard deviation (SD = square root of the variance) for each of the 4 groups. We quantified within-child variability as the SD of the trial-to-trial difference in individual z-scores (sdDiff). We tested equality of variance using Levene's test between group and age group among sessions and at each session. Mean sdDiff was compared among sessions, and between groups and age groups, using ANOVA followed by Tukey-adjusted post-hoc tests. To examine the relationship between within-child variability (sdDiff) in a session and 2 performance outcomes of that session (maximum z-score and the mean z-score) we ran a multiple regression model for each correlation per study, with group as an explanatory variable, both as a main effect and as an interaction with the performance measure.

Results

Performance Differences Between Children With CP and TD Children and Between Older and Younger Children at Each Session

Table 1 provides the mean and SD of z-scores for each age group across sessions and at each session. Table 2 presents the mean and SD of baseline postural control tests for each age group. Table 3 presents the effect sizes [Cohen's d (67) and Hedges g (68)] for each age group at each session.

TABLE 1

Table 1. Mean and SD of Z-scores for each age group across sessions and within each session.

TABLE 2

Table 2. Mean and SD of baseline postural control tests for each age group.

TABLE 3

Table 3. Effect sizes for between-group differences at each session.

Figure 3 illustrates that children with CP have consistently lower scores as compared to TD children (t = −7.102, p < 0.001, estimate (CP) = −1.015, mean difference = 0.809), while the performance for younger participants is consistently below that for older participants (t = 4.604, p < 0.001, estimate (older) = 0.5206, mean difference = 0.780). Across sessions, mixed effects models show that the largest effects are associated with trial (t = 21.027, p < 0.001), with a positive linear effect indicating that most participants improve over time. However, the negative non-linear effect indicates this tendency toward improvement tends to diminish or even reverse as trial number increases within a session. For example, in the acquisition session, performance of the younger participants with CP drops off pronouncedly at the end of the session. Within age and group, the relationship with trial varies widely among participants, ranging from linear (both positive and negative slopes) to unimodal. The models show that the R² almost doubles (R² = 0.304 vs. R² = 0.541) when accounting for these effects of random between-child variation.

FIGURE 3

Figure 3. Mean z-score by group and age group across all trials at each session. Error bars use sdDiff (the standard deviation of the pair-wise trial-to-trial differences). SE is calculated by devising the SD by the square root of the number of subjects in each trial.

Between-Child Variability Across Sessions and Per Session: Group (TD vs. CP) and Age (Younger vs. Older) Differences

There is a significant difference in the amount of between-child variability between children with CP and TD children across all sessions [F_{(1, 81)} = 9.254, p < 0.001, mean difference = −1.071]. Younger children with CP demonstrate the most between-child variability, while older children with CP have the least [F_{(3, 81)} = 1.888, p < 0.001, mean difference = −0.706]. For TD children, there is no difference in the amount of between-child variability between younger and older children [F_{(1, 65)}= 1.018, p = 0.758, mean difference = −0.517]. Older TD children and children with CP differ significantly in amount of between-child variability [F_{(1, 47)} = 1.627, p < 0.001, mean difference = 1.023]. There is no difference in between-child variability between the 3 sessions in younger children with CP [F_{(2, 5)} = 0.568, p = 0.568], while younger TD children demonstrated significant differences between sessions [F_{(2, 23)} = 5.375, p = 0.005].

Older TD children show significant differences [F_{(2, 38)} = 11.843, p < 0.001] between sessions, with the highest variability during acquisition, lowest during retention, and an increase again during transfer. Older children with CP also show significant differences [F_{(2, 5)} = 9.924, p < 0.001], but opposite to the pattern in the older TD children: lowest variation during acquisition, highest during retention, then a decrease during transfer. Younger TD children show significant differences [F_{(2, 23)} = 5.375, p = 0.005], with another distinct pattern: a peak in between-child variability during acquisition, followed by a steady decrease throughout retention and transfer. Younger children with CP do not show significant differences [F_{(2, 5)} = 0.568, p = 0.568] in variability between sessions. There are no significant differences in between-child variability between TD children and children with CP or between younger or older children in the transfer session.

Within-Child Variability Across Sessions and Per Session: Group (TD vs. CP) and Age (Younger vs. Older) Differences

The strongest difference in within-child variability was associated with age, with younger children demonstrating greater within-child variability as compared to older children when pooled across sessions and group (t = 3.3, p = 0.001, mean difference = 0.157). Children with CP showed a non-significant trend toward lower within-child variability as compared to TD children (t = 1.9, p = 0.06, mean difference = −0.131). Within-child variability did not differ significantly among sessions (p > 0.1). When comparing across groups and age groups within each individual session, there were no significant differences, possibly due to high variability among individuals and low sample sizes. Older TD children displayed a trend toward higher within-child variability as compared to older children with CP in the acquisition session (t = 1.7, p = 0.10, mean difference = 0.237) and younger children with CP in the transfer session (t = 1.7, p = 0.09, mean difference = 0.216).

Table 4 provides between- and within-child variability for each age group overall and for each age group at each session.

TABLE 4

Table 4. Within- and between-child variability for each age group and for each age group at each session.

Figure 4 illustrates the performance score for each individual participant across all trials of the 3 sessions, fit with a quadratic curve.

FIGURE 4

Figure 4. Performance score for each individual participant across all trials of the 3 sessions, fit with a quadratic curve.

Relationship Between the Amount of Within-Participant Variability in Acquisition and Performance Outcomes at Each Session

These analyses revealed a strong effect of study as an explanatory variable. In Study 1, there was a significant positive relationship of sdDiff during acquisition with MaxZ score at acquisition (R² = 0.485, p < 0.001); this relationship did not differ between TD children and children with CP. There was no significant relationship with MeanZ. The amount of variability (sdDiff) in acquisition is not significantly correlated with MaxZ or MeanZ in retention or transfer sessions for Study 1.

For study 2, there was no significant negative relationship of amount of variability in acquisition with maxZ in acquistion (R² = 0.237, p = 0.100). There was a significant negative relationship of amount of variability in acquisition with meanZ (R² = 0.442, p < 0.001) in acquisition, with no difference between TD children and children with CP. In Study 2, the amount of variability in retention is significantly negatively correlated with MaxZ (R² = 0.373, p = 0.035) and with MeanZ (R² = 0.293, p = 0.0216) at retention. The amount of variability in the transfer session is significantly negatively correlated with MaxZ (R² = 0.336, p = 0.010), and MeanZ (R² = 0.503, p < 0.001) at transfer, with no difference in this relationship between TD children and children with CP.

Discussion

This secondary data analysis explored within- and between-child performance variability during practice of a novel postural control task in a non-immersive VE at acquisition, retention and transfer sessions in children with hemiplegic CP and TD children. Consistent with evidence for motor skill acquisition with practice in children with CP [e.g., (69–71)], performance on the task improved over repeated trials during the acquisition session, although it did not reach the same success level as TD children. We observed expected age differences in performance with older children outperforming younger children in each group. Performance decreased at the end of each practice session of our standing postural control tasks, especially for younger children with CP: e.g., from trial 19 to trial 20, the final trial of the acquisition stage, performance decreases by ≥0.2, enough for the error bars to exclude the mean curve in both CP age groups, but neither TD age group (Figure 3). Fatigue is one possible explanation for this observation. Children with CP demonstrate greater energy expenditure in ambulatory tasks as compared to TD children (72). Bronton and Bartlett (73) surveyed fatigue in 130 young adults with CP at all GMFCS levels, finding that while fatigue was highest in individuals at higher GMFCS levels (II-V), the majority (92%) of participants reported being fatigued at least a quarter of the day or more.

The amount of between-child variability in performance differed between groups, age groups and sessions. As hypothesized, younger children with CP had the highest between-child variability at each session, likely reflecting the known heterogeneity in motor abilities in children with CP compounded by a lesser amount of motor skill experience at this age. Contrary to our hypothesis, children with CP did not always demonstrate more between-child variability as compared to TD children. Patterns of variability in each session differed for older TD children and children with CP. Older children with CP had the least between-child variability in acquisition and transfer sessions but had highest between-child variability during retention. In contrast, TD children had the most variability in acquisition and the least variability in retention. This finding is explained by the fact that children with CP had consistently lower scores across participants.

The greater amount of between-child variability in retention in children with CP, combined with lower scores in the retention session, may reflect motor learning impairments in children with CP as compared to TD children, who more consistently retained task performance improvements. Information about children's postural control abilities obtained from baseline testing postural control abilities (Table 1) shows expected differences between TD children and children with CP due to motor impairment. However, postural control results were not especially heterogeneous among older children with CP, which further explains the low between-child variability. We did not collect detailed demographic data from children as to their current physical activity or sports participation that could help to elucidate between-child variability in task performance at retention in terms of movement experience. High between-child variability in all groups and age groups in the transfer session also suggests the importance of exploring other child factors that influence motor learning. For example, factors such as attention or motivation that were unmeasured here may illuminate between-child performance differences.

With respect to within-child variability, individual children were least variable in their performance across trials in the retention, suggesting that children achieved sufficient task competence to maintain stable performance after a period of no practice. As hypothesized, younger children demonstrated greater within-child variability than older children at each session; however, we were surprised to see lower within-child variability in children with CP as compared to TD children in acquisition and transfer. This finding may be explained again by the overall consistently poorer performance (i.e., lower scores) of children with CP as compared to TD children, limiting the range of scores across which subjects can vary. This is particularly true in the transfer session, which had the lowest scores for children with CP (especially for younger children). A less challenging task and a larger sample size may have resulted in greater information about within-child variability in performance over time. Simple prospective power calculations indicate that, for a balanced design, a sample size of 25 subjects per age group and developmental group would be required to detect our observed difference in the transfer session at alpha = 0.05. Subsequent studies can evaluate within-child variability over longer durations of practice, while considering the challenge of fatigue and motor endurance in this population.

Movement execution variability is one contributor to performance variability across repeated trials. In our studies, performance score was directly based on movement execution: the precision of controlling weight-shifting of the CoP over a static base of support. Studies involving repeated task practice of seated reaching tasks in non-immersive VEs demonstrate that children with CP reduce their movement execution variability with practice (16, 57, 74). However, these studies did not include retention or transfer tasks. In a game play situation, children with CP playing active video games in a non-immersive VE demonstrate within- and between-child variability at a movement execution level during repetitive game play (59, 60). This game play situation has some similarity to our study tasks in having greater opportunities for exploration in movement strategies as compared to studies involving restricted single arm reaching tasks in a seated position. This could have influenced the amount of between-child variability as children tried different strategies to achieve the task goal.

The relationship of within-child variability to performance differed between the 2 studies, with a positive correlation (greater variability in acquisition associated with better scores) in Study 1 and a negative correlation in Study 2. Both studies had similar movement requirements for success and similar visual feedback about how avatar position determined score. However, the visual display differed between the 2 studies, with the full path visible in a 3rd person perspective in Study 2 and the path emerging with movement in a first-person perspective in Study 1. Children with CP found Study 2 more challenging, as scores were lower as compared to Study 1. The path width was narrower in Study 2 as compared to study 1, leading to more penalty for increased variation in CoP position. In Study 1, we can speculate that with a slightly wider path that constantly revealed itself in a first person perspective, children had more tolerance for variation and that those who took advantage of this may have found a more optimal strategy that resulted in higher scores. Differences in the relationship between amount of variability and performance according to task requirements and VE visual display perspective suggest the need for subsequent research to explore the influence of these and other factors on variability.

Our variability metric did not enable us to partition variability into adaptive or error components. Using more sophisticated statistical models to understand the structure of variability can provide more insight into randomness and exploration patterns (31, 32). Example methods that could be useful for exploring the structure of different solutions in simple redundant tasks include the Tolerance, Noise and Covariation Approach (TNC), the Uncontrolled Manifold Approach, and the Goal-Equivalent Manifold approach; readers are directed to Sternad (32) for an overview. Of these, only the TNC method was purposefully developed to evaluate learning processes in changes in variability over time; however, it has not yet been applied to complex 3D tasks, as the model assumes task outcome redundancy from two precisely quantified input variables.

Exploring variability in movement execution during new task learning in non-immersive VEs through kinematic analyses can provide additional insight into this important source of performance variability. To understand how variability differs between virtual and physical environments, subsequent research can use within-participant designs to compare variability in the same task in a VE and an equivalent physical environment. An unexplored area of future research is whether VEs are relevant training paradigms to encourage development of variability/behavioral flexibility (75). Given that task features and task challenge can be precisely manipulated in a VE, these features could elicit practice of variable responses to differing task conditions and adapting to different task constraints. To achieve this goal, more knowledge about the similarity of movements in VEs to the physical environment and how learning transfers to the physical environment in children with CP is important to understand the degree of similarity required to facilitate transfer. Other hypothesized intertwined factors that might influence variability include children's attention, fatigue, effort, and motivation. Indeed, a predominant rationale for the use of VEs is that they elicit and sustain children's motivation and attention to participate in repetitive training (76).

This study has several limitations. Conclusions about variability in performance at retention and transfer sessions are limited by inconsistent rest periods between acquisition and retention/transfer sessions between participants, ranging from 2 to 7 days. These periods were necessary to accommodate family schedules in data collection. Our sample size was small and unbalanced, with the CP group having an especially low number of participants. While the mixed model approach utilized in the lme4 package is designed to be robust to unbalanced data (77), we interpret our results cautiously and would encourage follow-up studies with larger sample sizes.

Conclusion

Performance variability can be an important source of information about differences in children's responses to interventions and should be considered in the design of rehabilitation protocols. This study is the first to specifically investigate performance variability over time during learning of standing postural control tasks in a non-immersive VE in children with hemiplegic CP. Findings contribute to the evidence base about differences in motor skill learning in children with CP as compared to TD peers in these novel intervention environments. Between- and within-child performance variability in children with CP is consistent with expected challenges with task performance due to motor impairments and age. A greater understanding of variability in motor skill learning in VEs is important to advance the debate as to the benefits and disadvantages of variability in motor skill learning and to understand whether the affordances of non-immersive VEs may make these interventions appropriate for training behavioral flexibility. Given that the relationship of within-child variability in skill acquisition differs according to the specific demands of the task, other factors that influence performance variability should be explored in subsequent studies, including differences in movement execution in VEs and cognitive factors such as attention and motivation. The design of VE-based interventions for children with hemiplegic CP can consider all these factors and their implications in order to maximize therapeutic benefit.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Northeastern University Institutional Review Board. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin. Written informed consent was obtained from the minor(s)' legal guardian/next of kin for the publication of any potentially identifiable images or data included in this article.

Author Contributions

DL conceived of the study. MA and MC undertook the analyses and prepared the figures. MC, DL, and MA wrote and edited the manuscript. All authors agree to be accountable for the content of the work.

Funding

DL was supported by a K01 Mentored Research Scientist Development Grant (K01HD093838) from the Eunice Kennedy Shriver National Institute of Child Health and Human Development. One of the projects described was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, Award Number UL1TR002544. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The second project was supported by a Tier 1 Grant from Northeastern University.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank the children and families who participated in this research and the research assistants who participated in data collection for the 2 studies.

References

1. Van Naarden Braun K, Doernberg N, Schieve L, Christensen D, Goodman A, Yeargin-Allsopp M. Birth prevalence of cerebral palsy: a population-based study. Pediatrics. (2016) 137:1–9. doi: 10.1542/peds.2015-2872

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Maenner MJ, Blumberg SJ, Kogan MD, Christensen D, Yeargin-Allsopp M, Schieve LA. Prevalence of cerebral palsy and intellectual disability among children identified in two US National Surveys, 2011–2013. Ann Epidemiol. (2016) 26:222–6. doi: 10.1016/j.annepidem.2016.01.001

CrossRef Full Text | Google Scholar

3. Pulgar S, Bains S, Gooch J, Chambers H, Noritz GH, Wright E, et al. Prevalence, patterns, and cost of care for children with cerebral palsy enrolled in medicaid managed care. J Manag Care Special Pharm. (2019) 25:817–22. doi: 10.18553/jmcp.2019.25.7.817

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Oskoui M, Coutinho F, Dykeman J, Jette N, Pringsheim T. An update on the prevalence of cerebral palsy: a systematic review and meta-analysis. Dev Med Child Neurol. (2013) 55:509–19. doi: 10.1111/dmcn.12080

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Shevell MI, Dagenais L, Hall N. The relationship of cerebral palsy subtype and functional motor impairment: a population-based study. Dev Med Child Neurol. (2009) 51:872–7. doi: 10.1111/j.1469-8749.2009.03269.x

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Russell JC, Bjornson K. Participation in daily life: Influence on quality of life in ambulatory children with cerebral palsy. PM R. (2016) 8:S151–S. doi: 10.1016/j.pmrj.2016.07.019

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Himmelmann K, Hagberg G, Beckung E, Hagberg B, Uvebrant P. The changing panorama of cerebral palsy in Sweden. IX. Prevalence and origin in the birth-year period 1995–1998. Acta Paediatr. (2005) 94:287–94. doi: 10.1111/j.1651-2227.2005.tb03071.x

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Pavao SL, Arnoni JL, de Oliveira AK, Rocha NA. Impact of a virtual reality-based intervention on motor performance and balance of a child with cerebral palsy: a case study. Rev Paul Pediatr. (2014) 32:389–94. doi: 10.1590/S0103-05822014000400016

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Pavao SL, Nunes GS, Santos AN, Rocha NA. Relationship between static postural control and the level of functional abilities in children with cerebral palsy. Braz J Phys Ther. (2014) 18:300–7. doi: 10.1590/bjpt-rbf.2014.0056

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Cohen-Holzer M, Sorek G, Kerem J, Schless S, Freedman R, Rotem H, et al. The influence of intense combined training on upper extremity function in children with unilateral cerebral palsy: does initial ability matter? Phys Occup Ther Pediatr. (2016) 36:376–87. doi: 10.3109/01942638.2015.1108379

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Reid LB, Rose SE, Boyd RN. Rehabilitation and neuroplasticity in children with unilateral cerebral palsy. Nat Rev Neurol. (2015) 11:390–400. doi: 10.1038/nrneurol.2015.97

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Gordon AM, Magill RA. Motor learning: application of principles to pediatric rehabilitation. In: Campbell SK, Palisano RJ, Orlin MN, editors. Physical Therapy for Children. 4th ed. Philadelphia: Saunders. (2012). p. 151–74.

Google Scholar

13. Kantak SS, Sullivan KJ, Burtner P. Motor learning in children with cerebral palsy: implications for rehabilitation. In: Eliasson AC, Burtner PA, editors. Improving Hand Function in Children With Cerebral Palsy: Theory, Evidence and Intervention. London, GBR: Mac Keith Press (2008).

Google Scholar

14. Burtner PA, Leinwand R, Sullivan KJ, Goh HT, Kantak SS. Motor learning in children with hemiplegic cerebral palsy: feedback effects on skill acquisition. Dev Med Child Neurol. (2014) 56:259–66. doi: 10.1111/dmcn.12364

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Chu VW, Park S-W, Sanger TD, Sternad D. Children with dystonia can learn a novel motor skill: strategies that are tolerant to high variability. IEEE Trans Neural Syst Rehabil Eng. (2016) 24:847–58. doi: 10.1109/TNSRE.2016.2521404

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Robert MT, Guberek R, Sveistrup H, Levin MF. Motor learning in children with hemiplegic cerebral palsy and the role of sensation in short-term motor training of goal-directed reaching. Dev Med Child Neurol. (2013) 55:1121–8. doi: 10.1111/dmcn.12219

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Harbourne RT. Accuracy of movement speed and error detection skills in adolescents with cerebral palsy. Percept Mot Skills. (2001) 93:419–31. doi: 10.2466/pms.2001.93.2.419

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Thorpe DE, Valvano J. The effects of knowledge of performance and cognitive strategies on motor skill learning in children with cerebral palsy. Pediatr Phys Ther. (2002) 14:2–15. doi: 10.1097/00001577-200214010-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Hung YC, Gordon AM. Motor learning of a bimanual task in children with unilateral cerebral palsy. Res Dev Disabil. (2013) 34:1891–6. doi: 10.1016/j.ridd.2013.03.008

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Mutsaarts M, Steenbergen B, Bekkering H. Anticipatory planning deficits and task context effects in hemiparetic cerebral palsy. Exp Brain Res. (2006) 172:151–62. doi: 10.1007/s00221-005-0327-0

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Gagliardi C, Tavano A, Turconi AC, Pozzoli U, Borgatti R. Sequence learning in cerebral palsy. Pediatr Neurol. (2011) 44:207–13. doi: 10.1016/j.pediatrneurol.2010.10.004

CrossRef Full Text | Google Scholar

22. Gofer-Levi M, Silberg T, Brezner A, Vakil E. Deficit in implicit motor sequence learning among children and adolescents with spastic cerebral palsy. Res Dev Disabil. (2013) 34:3672–8. doi: 10.1016/j.ridd.2013.07.029

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Gofer-Levi M, Silberg T, Brezner A, Vakil E. Cognitive procedural learning among children and adolescents with or without spastic cerebral palsy: the differential effect of age. Res Dev Disabil. (2014) 35:1952–62. doi: 10.1016/j.ridd.2014.04.017

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Tieman B, Palisano RJ, Gracely EJ, Rosenbaum PL. Variability in mobility of children with cerebral palsy. Pediatr Phys Ther. (2007) 19:180–7. doi: 10.1097/PEP.0b013e31811ec795

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Prado MTA, Fernani DCGL, Silva TDd, Smorenburg ARP, Abreu LCd, Monteiro CBdM. Motor learning paradigm and contextual interference in manual computer tasks in individuals with cerebral palsy. Res Dev Disabil. (2017) 64:56–63. doi: 10.1016/j.ridd.2017.03.006

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Sakash A, Broman AT, Rathouz PJ, Hustad KC. Executive function in school-aged children with cerebral palsy: relationship with speech and language. Res Dev Disabil. (2018) 78:136–44. doi: 10.1016/j.ridd.2018.05.015

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Majnemer A, Shevell M, Hall N, Poulin C, Law M. Developmental and functional abilities in children with cerebral palsy as related to pattern and level of motor function. J Child Neurol. (2010) 25:1236–41. doi: 10.1177/0883073810363175

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Oeffinger D, Gorton G, Bagley A, Nicholson D, Barnes D, Calmes J, et al. Outcome assessments in children with cerebral palsy, part I: descriptive characteristics of GMFCS levels I to III. Dev Med Child Neurol. (2007) 49:172–80. doi: 10.1111/j.1469-8749.2007.00172.x

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Clutterbuck GL, Auld ML, Johnston LM. Performance of school-aged children with cerebral palsy at GMFCS levels I and II on high-level, sports-focussed gross motor assessments. Disabil Rehabil. (2019) 1–9. doi: 10.1080/09638288.2019.1650964

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Hassani S, Krzak J, Flanagan A, Bagley A, Gorton G, Romness M, et al. Assessment of strength and function in ambulatory children with cerebral palsy by GMFCS level and age: a cross-sectional study. Crit Rev Phys Rehabil Med. (2011) 23:1–14. doi: 10.1615/CritRevPhysRehabilMed.v23.i1-4.10

CrossRef Full Text | Google Scholar

31. Ranganathan R, Lee M-H, Newell KM. Repetition without repetition: challenges in understanding behavioral flexibility in motor skill. Front Psychol. (2020) 11:2018. doi: 10.3389/fpsyg.2020.02018

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Sternad D. It's not (only) the mean that matters: variability, noise and exploration in skill learning. Curr Opin Behav Sci. (2018) 20:183–95. doi: 10.1016/j.cobeha.2018.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Muller H, Sternad D. Motor learning: changes in the structure of variability in a redundant task. Adv Exp Med Biol. (2009) 629:439–56. doi: 10.1007/978-0-387-77064-2_23

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Sternad D, Abe MO, Hu X, Muller H. Neuromotor noise, error tolerance and velocity-dependent costs in skilled performance. PLoS Comput Biol. (2011) 7:e1002159. doi: 10.1371/journal.pcbi.1002159

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Sternad D, Huber ME, Kuznetsov N. Acquisition of novel and complex motor skills: stable solutions where intrinsic noise matters less. Adv Exp Med Biol. (2014) 826:101–24. doi: 10.1007/978-1-4939-1338-1_8

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Hadders-Algra M. Variation and variability: key words in human motor development. Phys Ther. (2010) 90:1823–37. doi: 10.2522/ptj.20100006

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Senn S. Mastering variation: variance components and personalised medicine. Stat Med. (2016) 35:966–77. doi: 10.1002/sim.6739

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Braendvik SM, Goihl T, Braaten RS, Vereijken B. The effect of increased gait speed on asymmetry and variability in children with cerebral palsy. Front Neurol. (2019) 10:1399. doi: 10.3389/fneur.2019.01399

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Kim Y, Bulea TC, Damiano DL. Children with cerebral palsy have greater stride-to-stride variability of muscle synergies during gait than typically developing children: implications for motor control complexity. Neurorehabil Neural Repair. (2018) 32:834–44. doi: 10.1177/1545968318796333

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Davies BL, Kurz MJ. Children with cerebral palsy have greater stochastic features present in the variability of their gait kinematics. Res Dev Disabil. (2013) 34:3648–53. doi: 10.1016/j.ridd.2013.08.012

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Mawase F, Bar-Haim S, Joubran K, Rubin L, Karniel A, Shmuelof L. Increased adaptation rates and reduction in trial-by-trial variability in subjects with cerebral palsy following a multi-session locomotor adaptation training. Front Hum Neurosci. (2016) 10:203. doi: 10.3389/fnhum.2016.00203

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Gordon AM. Impaired voluntary movement control and its rehabilitation in cerebral palsy. Adv Exp Med Biol. (2016) 957:291–311. doi: 10.1007/978-3-319-47313-0_16

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Prosser LA, Lauer RT, VanSant AF, Barbe MF, Lee SC. Variability and symmetry of gait in early walkers with and without bilateral cerebral palsy. Gait Posture. (2010) 31:522–6. doi: 10.1016/j.gaitpost.2010.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Wohlert AB, Smith A. Developmental change in variability of lip muscle activity during speech. J Speech Lang Hear Res. (2002) 45:1077–87. doi: 10.1044/1092-4388(2002/086)

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Chen C-l, Chen H-c, Hong W-h, Yang F-pG, Yang L-y, Wu C-y. Oromotor variability in children with mild spastic cerebral palsy: a kinematic study of speech motor control. J Neuroeng Rehabil. (2010) 7:1–10. doi: 10.1186/1743-0003-7-54

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Allison KM, Hustad KC. Data-driven classification of dysarthria profiles in children with cerebral palsy. J Speech Lang Hear Res. (2018) 61:2837–53. doi: 10.1044/2018_JSLHR-S-17-0356

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Weiss PL, Rand D, Katz N, Kizony R. Video capture virtual reality as a flexible and effective rehabilitation tool. J Neuroeng Rehabil. (2004) 1:12. doi: 10.1186/1743-0003-1-12

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Chen Y, Fanchiang HD, Howard A. Effectiveness of virtual reality in children with cerebral palsy: a systematic review and meta-analysis of randomized controlled trials. Phys Ther. (2018) 98:63–77. doi: 10.1093/ptj/pzx107

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Ravi D, Kumar N, Singhi P. Effectiveness of virtual reality rehabilitation for children and adolescents with cerebral palsy: an updated evidence-based systematic review. Physiotherapy. (2017) 103:245–58. doi: 10.1016/j.physio.2016.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Golomb MR, McDonald BC, Warden SJ, Yonkman J, Saykin AJ, Shirley B, et al. In-home virtual reality videogame telerehabilitation in adolescents with hemiplegic cerebral palsy. Arch Phys Med Rehabil. (2010) 91:1–8.e1. doi: 10.1016/j.apmr.2009.08.153

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Jannink MJ, van der Wilden GJ, Navis DW, Visser G, Gussinklo J, Ijzerman M. A low-cost video game applied for training of upper extremity function in children with cerebral palsy: a pilot study. Cyberpsychol Behav. (2008) 11:27–32. doi: 10.1089/cpb.2007.0014

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Jelsma J, Pronk M, Ferguson G, Jelsma-Smit D. The effect of the Nintendo Wii Fit on balance control and gross motor function of children with spastic hemiplegic cerebral palsy. Dev Neurorehabil. (2013) 16:27–37. doi: 10.3109/17518423.2012.711781

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Brien M, Sveistrup H. An intensive virtual reality program improves functional balance and mobility of adolescents with cerebral palsy. Pediatr Phys Ther. (2011) 23:258–66. doi: 10.1097/PEP.0b013e318227ca0f

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Deutsch JE, Borbely M, Filler J, Huhn K, Guarrera-Bowlby P. Use of a low-cost, commercially available gaming console (Wii) for rehabilitation of an adolescent with cerebral palsy. Phys Ther. (2008) 88:1196–207. doi: 10.2522/ptj.20080062

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Levin MF, Deutsch JE, Kafri M, Lieberman DG. Validity of virtual reality environments for motor rehabilitation. In: Weiss PL, Keshner EA, LevinMF, editors. Virtual Reality for Physical and Motor Rehabilitation. New York, NY: Springer (2014). p. 95–118.

Google Scholar

56. Drew SA, Awad MF, Armendariz JA, Gabay B, Lachica IJ, Hinkel-Lipsker JW. The trade-off of virtual reality training for dart throwing: a facilitation of perceptual-motor learning with a detriment to performance. Front Sports Act Living. (2020) 2:59. doi: 10.3389/fspor.2020.00059

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Sandlund M, Domellof E, Grip H, Ronnqvist L, Hager CK. Training of goal directed arm movements with motion interactive video games in children with cerebral palsy - a kinematic evaluation. Dev Neurorehabil. (2014) 17:318–26. doi: 10.3109/17518423.2013.776124

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Robert MT, Levin MF. Validation of reaching in a virtual environment in typically developing children and children with mild unilateral cerebral palsy. Dev Med Child Neurol. (2018) 60:382–90. doi: 10.1111/dmcn.13688

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Berry T, Howcroft J, Klejman S, Fehlings D, Wright V, et al. Variations in movement patterns during active video game play in children with cerebral palsy. J Bioeng Biomed Sci. (2011) S1:1. doi: 10.4172/2155-9538.S1-001

CrossRef Full Text | Google Scholar

60. Howcroft J, Klejman S, Fehlings D, Wright V, Zabjek K, Andrysek J, et al. Active video game play in children with cerebral palsy: potential for physical activity promotion and rehabilitation therapies. Arch Phys Med Rehabil. (2012) 93:1448–56. doi: 10.1016/j.apmr.2012.02.033

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Kleim JA, Jones TA. Principles of experience-dependent neural plasticity: implications for rehabilitation after brain damage. (2008) 51:S225–39. doi: 10.1044/1092-4388(2008/018)

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Beckers LW, Geijen MM, Kleijnen J, Rameckers EA, Schnackers ML, Smeets RJ, et al. Feasibility and effectiveness of home-based therapy programmes for children with cerebral palsy: a systematic review. BMJ Open. (2020) 10:e035454. doi: 10.1136/bmjopen-2019-035454

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Levac DE, Lu AS. Does narrative feedback enhance children's motor learning in a virtual environment? J Mot Behav. (2019) 51:199–211. doi: 10.1080/00222895.2018.1454398

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Levac DE, Taylor MM, Payne B, Ward N. Influence of virtual environment complexity on motor learning in typically developing children and children with cerebral palsy. In: 2019 International Conference on Virtual Rehabilitation (ICVR). IEEE (2019).

Google Scholar

65. Akaike H. Information Theory and an Extension of the Maximum Likelihood Principle. In: Parzen E, Tanabe K, Kitagawa G, editors. Selected Papers of Hirotugu Akaike. New York, NY: Springer New York (1998). p. 199–213.

Google Scholar

66. Burnham KP, Anderson DR. A Practical Information-Theoretic Approach. Model Selection and Multimodel Inference. 2nd ed. Vol. 2. New York, NY; Springer (2002).

Google Scholar

67. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. New York, NY: Routledge (1988).

PubMed Abstract | Google Scholar

68. Hedges LV. Distribution theory for Glass's estimator of effect size and related estimators. J Educ Stat. (1981) 6:107–28. doi: 10.3102/10769986006002107

CrossRef Full Text | Google Scholar

69. Hemayattalab R, Rostami LR. Effects of frequency of feedback on the learning of motor skill in individuals with cerebral palsy. Res Dev Disabil. (2010) 31:212–7. doi: 10.1016/j.ridd.2009.09.002

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Hemayattalab R, Arabameri E, Pourazar M, Ardakani MD, Kashefi M. Effects of self-controlled feedback on learning of a throwing task in children with spastic hemiplegic cerebral palsy. Res Dev Disabil. (2013) 34:2884–9. doi: 10.1016/j.ridd.2013.05.008

PubMed Abstract | CrossRef Full Text | Google Scholar

71. de Mello Monteiro CB, Massetti T, da Silva TD, van der Kamp J, de Abreu LC, Leone C, et al. Transfer of motor learning from virtual to natural environments in individuals with cerebral palsy. Res Dev Disabil. (2014) 35:2430–7. doi: 10.1016/j.ridd.2014.06.006

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Bottos M, Gericke C. Ambulatory capacity in cerebral palsy: prognostic criteria and consequences for intervention. Dev Med Child Neurol. (2003) 45:786. doi: 10.1017/S0012162203001452

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Brunton LK, Bartlett DJ. Profiles of fatigue severity and variability among adolescents and young adults with cerebral palsy. Fatigue Biomed Health Behav. (2016) 5:5–14. doi: 10.1080/21641846.2017.1264950

CrossRef Full Text | Google Scholar

74. Chen Y, Garcia-Vergara S, Howard AM. Effect of a home-based virtual reality intervention for children with cerebral palsy using super pop vr evaluation metrics: a feasibility study. Rehabil Res Pract. (2015) 2015:812348. doi: 10.1155/2015/812348

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Levac DE, Huber ME, Sternad D. Learning and transfer of complex motor skills in virtual reality: a perspective review. J Neuroeng Rehabil. (2019) 16:121. doi: 10.1186/s12984-019-0587-8

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Rohrbach N, Chicklis E, Levac DE. What is the impact of user affect on motor learning in virtual environments after stroke? A scoping review. J Neuroeng Rehabil. (2019) 16:1–14. doi: 10.1186/s12984-019-0546-4

PubMed Abstract | CrossRef Full Text | Google Scholar

77. Bates DM. lme4: Mixed-Effects Modeling With R. New York, NY: Springer (2010).

Google Scholar