Valter R. Fernandes
Derek R. Becker
Megan M. McClelland
Andrea C. Deslandes1- 1Exercise Neuroscience Laboratory, Institute of Psychiatry, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil
- 2Department of Human Services, College of Education and Allied Professions, Western Carolina University, Cullowhee, NC, United States
- 3Hallie E. Ford Center for Healthy Children & Families, Oregon State University, Corvallis, OR, United States
Executive function (EF) is a foundational cognitive construct, which is linked to better cognitive and physical health throughout development. The present study examines the construct validity of an EF task, the Head-Toes-Knees-Shoulders task (HTKS) that was initially developed for young children, in a sample of adolescents. We investigate the initial validity and range in scores between 54 adolescents from Brazil (mean age 12.58) and 56 US adolescents (mean age 12.48) from different socioeconomic contexts. Results indicated that the HTKS showed sufficient variability in both samples, especially for a measure of HTKS efficiency (completion time divided by the total score). The US sample performed better on all cognitive measures. For the Brazilian sample, regression models controlling for age and sex showed a significant relationship between the digit span working memory task, the HTKS total score, and the HTKS efficiency score. The Heart and Flowers cognitive flexibility measure was also included as an independent variable only for the Brazil sample, showing a significant relationship with both HTKS scores. For the US sample, results showed that only the HTKS efficiency score was significantly related to the digit span working memory task. This study highlights the importance of cognitive efficiency measures to achieve greater validity, as they can assess a broader range of performance with different populations. The HTKS showed good ecological validity with two adolescent samples, as it differentiated between populations with high and low socioeconomic status from different cultural contexts.
Introduction
Recent reports document the importance of children’s self-regulation and executive function for a variety of important life outcomes such as educational success and indicators of health and well-being (McClelland et al., 2013; Robson et al., 2020). These skills lay the foundation for school success and predict long-term educational outcomes starting in early childhood and through adolescence (Duckworth et al., 2010; McClelland et al., 2013). Self-regulation consists of the capacity to consciously control thoughts, feelings, and behavior and stems from executive function (EF) skills that experience rapid growth in early adolescence and which correspond to rapid brain development during this time (Zelazo and Carlson, 2012).
EF is recognized to include the cognitive processes of working memory, inhibitory control, and cognitive flexibility (McClelland et al., 2015). It has been described in different ways by the scientific literature. Researchers have described “Hot EF” as the emotional and motivational abilities that mediate cognitive control, likely engaging the prefrontal cortex orbitomedial and ventromedial, and its connections to limbic structures (Rubia, 2011). In contrast, “Cool EF” refers to more cognitive abilities, such as working memory, inhibitory control, shifting, planning, and fluid intelligence, mainly demanding dorso and frontolateral prefrontal cortex (Zelazo and Carlson, 2012). The ability to use perception and knowledge to select actions and thoughts coherent with our emotions and motivation allows us to respond accurately to what the environment demands (Friedman and Robbins, 2022). Working memory allows individuals to remember and process information, inhibitory control includes the ability to inhibit and manage thoughts and behavior, and cognitive flexibility enables individuals to focus on a task and switch attention when needed (Blair and Ku, 2022). During development, working memory and cognitive flexibility are particularly important, showing consistent correlations across middle childhood, adolescence, and early adulthood (Best and Miller, 2010). Together, these cognitive processes allow for the top-down regulation of thinking and behavior and help individuals to follow through on instructions, demonstrate self-control, and effectively manage tasks. In the present study, we focus on a measure of EF in adolescents that requires the integration of cognitive aspects of executive functioning in a real-life behavioral situation. The Head-Toes-Knees-Shoulders (HTKS) task was developed for young children and has recently been extended to measure EF in older adults (Cerino et al., 2018) and in adolescents (Dutra et al., 2022). In this study, we analyze the HTKS and its relations to other measures of EF in a sample of adolescents in Brazil and the United States, including the variability, range in scores, and construct validity.
The importance of EF for children and adolescents
During early childhood, EF becomes increasingly important and are crucial for children in the social environment of school and for academic success. They are described as a set of cognitive skills that are essential for social interaction, daily activities, work and learning (Diamond, 2013). A child’s ability to control and regulate their behavior predicts economic and health outcomes, and academic achievement and lowers the chance of having a criminal record (Moffitt et al., 2011).
Although most research on EF is focused on early childhood, it is a set of cognitive skills linked to several indicators of positive development between early childhood and adolescents (Best et al., 2011; Ahmed et al., 2019). Indeed, strong adolescent EF is associated with a lower body mass index (Reinert et al., 2013; Ronan et al., 2020), lower levels of media (Baumgartner et al., 2014) and substance use (Nigg et al., 2006), and better academic outcomes (Latzman et al., 2010; Samuels et al., 2016). The literature has pointed to different developmental trajectories for components of EF, but working memory may have a slower maturation process (Igazság et al., 2019). Although there is mixed evidence about whether adult-level performance is reached between 14 (Luna et al., 2004) and 17 years of age (Igazság et al., 2019), there is agreement on the rapid development of working memory and cognitive flexibility during adolescence (Luna et al., 2004; Best and Miller, 2010; Igazság et al., 2019).
Working memory and cognitive flexibility are highly involved in general and mathematical problem solving (Raghubar et al., 2010; Carden and Cline, 2015). They are critical for response inhibition and decision making (Best et al., 2011; McClelland et al., 2014; Ludyga et al., 2022), which are also required in sports tasks and real-life situations such as children’s play. The HTKS requires working memory to inhibit responses and also asks participants to switch rules using cognitive flexibility. Given the importance of working memory and cognitive flexibility in middle childhood and adolescence, the present study compared performance on the HTKS with measures of working memory and cognitive flexibility.
Exposure to different social and educational environments and links to EF in children from demographically-diverse communities
Academic performance is a multifactorial outcome that can be influenced by several variables, such as school and family structure (O’Malley et al., 2015), school and community resources for leisure time physical activity (Centeio et al., 2018), parental education, criminality and violence levels (Ruediger, 2017). Unfortunately, children who do not get their basic needs met in their social and family environments are more likely to face challenges in their cognitive development (Lipina and Segretin, 2019). In Brazil, as in other countries, educational and income inequalities are strongly correlated (de Souza and De Carvalhaes, 2014). Low levels of parent education can reduce a child’s opportunity to receive a quality education and achieve a higher education and income level (Medeiros et al., 2020). For example, in Brazil, children that live in low SES neighborhoods in Rio de Janeiro often suffer from high levels of criminality, which can affect their mental health and negatively impact academic performance (de Moraes et al., 2021). Nonetheless, even in neighborhoods with higher levels of criminality and violence, children’s EF can be positively impacted by school interventions (Watts et al., 2018).
In the United States (US), educational and income inequalities are also strongly correlated and predict lower EF and academic outcomes (Hackman and Farah, 2009; Ahmed et al., 2019). Neighborhood risk factors, such as high rates of crime and limited access to positive community resources (e.g., parks and green spaces) are linked to low levels of school success and poor health (Dadvand et al., 2015; Yu and Lippert, 2016; Wende et al., 2021). Children growing up in rural settings in the US, such as those in the present paper, are also at increased risk for low levels of physical activity and high rates of obesity (Joens-Matre et al., 2008; Gunter et al., 2015). Brazil’s Human Development Index (HDI), adjusted for inequality, is medium, while the US is classified as very high. This characterizes a large difference between the two countries in terms of the social context for children’s development [UNDP (United Nations Development Programme), 2022]. These findings highlight the importance of using sensitive and ecologically valid measures to assess children’s EF in different contexts.
Measures of EF in childhood and adolescence
Several EF assessments have been developed and adapted to capture better ecological validity and for children to be assessed in naturalistic rather than laboratory settings (Watts et al., 2018; Diamond et al., 2019; Gentile et al., 2020). Computerized EF tasks usually alternate between congruent and incongruent visual stimuli, with rules that alternate according to such stimuli. For example, the Hearts and Flowers test uses this principle, requiring working memory to maintain the rules related to each visual stimulus, inhibiting responses and cognitive flexibility for more accurate decisions in incongruent tasks (Wright and Diamond, 2014). Such tasks permit the registry of different measures of accuracy and reaction times in the different conditions, as well as in blocks that alternate between incongruent and congruent tasks.
Another popular EF measure, the Stroop test, requires saying colors in different conditions. As in the Heart and Flowers measure, an interference cost variable has been calculated and used by several authors (Lee et al., 2019). Another way to analyze performance in EF tests is cognitive efficiency, defined as the increase in knowledge gained concerning the amount of time invested in its acquisition (Hoffman, 2012). In this way, researchers have looked for ways to interpret scores, including a view that includes both accuracy and reaction time scores. For example, cognitive efficiency measures have been calculated considering the number of accurate responses per minute (Coffman et al., 2017; Miller et al., 2019). Lower scores indicate better performance as they represent lower costs of time to produce correct answers. In the present study, to better capture variability in the HTKS with adolescents, a measure of HTKS efficiency was created using the HTKS completion time (Cerino et al., 2018) and dividing it by the total HTKS score. We used both the HTKS efficiency score and the HTKS total score to provide a broader understanding of the variability and cognitive performance measured by the HTKS.
Different measures for the HTKS task for different populations
Given the association between EF with long-term health and cognitive development it is important to understand its developmental trajectory from early childhood into adolescents. The work that has assessed longitudinal measurement invariance with tasks that assess EF (Willoughby et al., 2012; Ahmed et al., 2019), has often employed assessments that require special software and a time-consuming application. This restricts how and where the assessment can be given.
The Head-Toes-Knees-Shoulders (HTKS) task is a brief and easy-to-administer measure of EF that has been extensively used with preschool and kindergarten-age children (Cameron Ponitz et al., 2007; McClelland et al., 2014; Kenny et al., 2023). The HTKS demonstrates strong reliability and validity (McClelland et al., 2014) and is related to academic and health-related outcomes during early childhood (Cameron Ponitz et al., 2009; Becker and Nader, 2021). Although the HTKS was primarily developed for younger children, recent work has extended the use of the task to older populations.
In a study with children in middle childhood and early adolescence, Dutra et al. (2022) analyzed the construct validity of the HTKS with measures of short-term memory and working memory, finding both tasks related to higher HTKS total scores. However, results also showed potential ceiling effects and low variability that might have impaired the construct validity of the HTKS. A related study with older adults used completion time on the HTKS and found evidence of construct validity with significant correlations with the NIH Toolbox and stronger correlations using the completion time variable relative to the HTKS total score (Cerino et al., 2018). Using a measure of cognitive efficiency could help avoid potential ceiling with the HTKS and be more closely aligned with traditional EF assessments used with older children.
The HTKS challenges executive control through incongruent motor responses to stimuli, and asks children to do the opposite to a given command. The assessment requires all aspects of EF – working memory to remember the rules, inhibitory control when asked to do the opposite of a command and cognitive flexibility when the rules change. The present study will build on the work of Dutra et al. (2022) and Cerino et al. (2018) by developing a measure of efficiency to help improve the sensitivity of the HTKS for use with older populations.
In the present study, we examined range in scores and construct validity of the HTKS in two samples of adolescents in the U.S. and Brazil. We extended the scoring on the HTKS, and included a measure of HTKS efficiency, which was the HTKS completion time (Cerino et al., 2018) divided by the total HTKS score. In addition, we analyzed the differences in the HTKS measures between both samples. In developing countries, there are more high school students who repeat grades, which can lead to greater age variability during the school years. This highlights the potential importance of using the HTKS efficiency measure (which may be more comprehensive and have greater variability) given the expected heterogeneity between the two samples in Brazil and the US.
We hypothesized that both samples of adolescents would show variability in the HTKS and EF measures. The HTKS efficiency score was expected to demonstrate stronger correlations with a measure of working memory than the HTKS total score. Differences between Brazilian and US children’s performances were expected since both countries are in different socioeconomic and cultural environments, and this could impact children’s cognitive development and performance. Considering these different contexts, we expected lower scores in the Brazilian sample.
Methods
Participants
This study examined the variability in scores and initial validity of the HTKS in adolescents in the United States (US) and Brazil. Participants included a combined 111 children who ranged in age between 10- to 15-years of age. We performed a post-hoc G-power sample size analysis, which showed that both samples could achieve significant results for an arbitrary target power of 95%.
Brazil sample
The Brazilian sample consisted of 55 students (65% male) from public schools aged 10 to 15 years, taken from a longitudinal study focused on the effect of physical exercise interventions on children’s cognition. Fifty-four percent of them were in grade six (mean age 12.58), 43% were in grade seven (mean age 12.91), and 4% were in grade eight (mean age 13.50). Participants and their guardians were informed about the study and gave informed consent. Children were excluded from this study if they had any hearing or visual impairments. Physical evaluations and HTKS were performed and computerized cognitive assessments were carried out on a subsequent day. This project was approved by the Ethics and Research Committee of the Centro Universitário Augusto Motta (UNISUAM) (CAAE 11980119.4.0000.5235).
United States sample
The U.S. sample included 56 adolescents (60% male) between the ages of 11 and 14 years attending a public lab-based middle school (grade 6 through grade 8) in the Southeast region of the U.S. The lab-based middle school was run by the public Regional University in collaboration with the local school district. Parents gave permission for their children to participate in research as a part of attending the lab school with the understanding that their child could opt out of any study. Fourteen percent of participating adolescents were in grade six (mean age 11.2), 42.5% were in grade seven (mean age 12.5), and 41.5% were in grade eight (mean age 13.5). This project was approved by Western Carolina University IRB PROJECT TITLE: [1156016–4] Catamount H.I.I.T.
Overall EF measure
Head toes knees shoulders (HTKS)
Children from the U.S. and Brazil both participated in the HTKS. The Portuguese version was used in the Brazilian sample and the English version was used in the US. The HTKS has 17 practice items and 30 test items that measure EF (working memory inhibitory control, and cognitive flexibility) with high ecological validity (McClelland et al., 2014, 2021). The task includes three sections. In part one, children have to touch their heads when instructed to touch their toes and, touch their toes when instructed to touch their heads. Part two includes a new command pairing shoulder with knees. In part 3, the four paired commands are reversed (head = knees, shoulders = toes). Correct responses are scored as a 2, self-correct are scored as a 1, and incorrect responses receive a 0.
The HTKS efficiency variable was calculated by combining accuracy and the time to complete the HTKS [(time/accuracy)/100]. The efficiency variable was created by first obtaining a total score on the HTKS by summing the three parts (range 0–60). The time variable assessed the amount of time in seconds children took to complete the HTKS using a digital stopwatch. The stopwatch was started at the point where the HTKS administrator began reading the instructions and stopped once the assessment was complete. Lower HTKS efficiency scores indicate better EF.
Working memory
Digit span task (forward-order and reverse-order)
In Brazil and the US, adolescents were administered either a computerized version (Brazil) or a paper and pencil version (US) of the digit span working memory task. The Forward Digit Task tests short-term memory and serves as an introduction to the Backward Digit Task, which is considered a working memory task. The sum of the two sub-items has been described as a measure of working memory (Gentile et al., 2020), with a high degree of reliability and validity (Conway et al., 2005). In both steps, participants hear the digit strings and need to retrieve them (directly or inversely) by selecting the digits from a circle of digits by touching the touchscreen. Depending on performance, participants move up or down a level. The evaluation ends after 14 attempts. The measure of the maximum length of up to two errors of the reverse order phase responses was used as a working memory measure (Woods et al., 2011). For adolescents in the US, the task was given in the same format as the computerized task completed by the Brazilian children, with the forward-order digit task followed by the reverse-order task. Adolescents in the US sample gave verbal responses. The task took approximately 5 minutes to complete and was administered in a quiet location.
Cognitive flexibility
Hearts and flowers task (H&F)
In Brazil, participants completed the Hearts and Flowers Task (H&F) and received information from congruent and incongruent tasks. When a heart appears on the screen, it must touch the same side as it and when a flower appears, it should touch on the opposite side. The H&F task includes the following phases: blocks with only hearts, blocks with only flowers, and blocks with a mix of hearts and flowers. All blocks had an initial moment of task learning, in which feedback was provided for each child’s response, reminding them, in case of error, of the task objective (congruent in the case of the apparition of the heart, and incongruent in the case of the flower apparition). In this study, we use the mixed block phase as a measure of cognitive flexibility (Stein et al., 2017), reaction time and accuracy were registered (Davidson et al., 2006) and used to calculate the H&F efficiency score [(time/accuracy)/100]. The H&F task is validated and widely used in the evaluation of children’s executive function in different countries (Spiegler and Leyendecker, 2017; Watts et al., 2018; Rosas et al., 2019).
Analytic plan
Descriptive statistics, tests of normality, and validity analyzes were completed separately in Stata 17 for the US and Brazilian samples. Normality was assessed by estimating skewness and kurtosis for both the HTKS and HTKS efficiency scores. Differences in HTKS measures between both samples were analyzed using independent samples t-tests. Bivariate correlations were used to assess validity for all variables of interest and multivariate regression, controlling for age and sex, were assessed separately for the HTKS and HTKS efficiency scores.
The percentage of missing data was small and was specific to age for the U.S. sample (n = 9, 17%) and Hearts and Flowers for the Brazilian sample (n = 3, 6%). Full Information Maximum Likelihood estimation (FIML) (Schafer and Graham, 2002) was used to appropriately address missing data for all regression models with the US and Brazilian samples.
Results
All research questions including descriptive statistics were analyzed for both samples. Table 1 includes the means and standard deviations for all the study variables, as well as the comparison of the results between the groups of Brazilian and US children. Measures of skewness and kurtosis are displayed in Table 2 with correlations displayed in Table 3.
Table 1. Descriptive analysis of the Brazilian and US children at baseline.
Table 2. HTKS ceiling effect analysis.
Table 3. Pearson correlations.
Does the HTKS demonstrate sufficient variability and range in scores in two samples of adolescents in Brazil and the US?
Results showed that HTKS scores demonstrated sufficient variability and range in scores for both samples, especially when using the HTKS efficiency score. For the total HTKS score and the HTKS efficiency score, 2% of Brazilian children scored at the ceiling. Brazilian children showed normally distributed HTKS scores and a significant positive skew for the HTKS efficiency score (see Table 2; Figure 1). In the sample of US children, 39% scored at the ceiling with the total HTKS score and less than 4% of US children scored at the ceiling with the efficiency score. Scores on the HTKS were significantly negatively skewed and scores on the HTKS efficiency measure were significantly positively skewed for US children (Table 2 and Figure 1).
Figure 1. Histograms for HTKS total score on top and HTKS Efficiency on the bottom, for United States and Brazilian samples.
Does exposure to different social and educational environments influence EF performance?
T-tests showed significant mean differences on the HTKS, with the U.S. sample showing better performance on both the HTKS and HTKS efficiency scores. However, no significant mean differences were found for age and the distribution of boys and girls between Brazilian and US samples.
Does the HTKS demonstrate initial construct validity in two samples of adolescents in Brazil and in the US?
We looked at the initial validity of the HTKS in samples of adolescents in the US and Brazil. To test HTKS construct validity, we first analyzed correlations between the measures of working memory collected in both samples (digit span) with both the HTKS and HTKS efficiency scores in both samples (see Table 3).
Results indicated significant correlations between measures of working memory and EF expressed by accuracy (HTKS total score) and a measure of cognitive efficiency (HTKS efficiency), but only for the Brazilian sample. As can be seen in Table 3, for Brazilian children, the HTKS and digit span task showed a small positive correlation (r = 0.33, p < 0.05), with the HTKS efficiency task significantly inversely correlated with the digit span task (r = −0.28, p < 0.05). The correlations between the digit span and the HTKS and HTKS efficiency score were non-significant for US children, but the correlation between the digit span and the HTKS efficiency score approached significance (r = −0.25, p < 0.10) and was similar in magnitude to the correlation in the Brazilian sample.
The Heart and Flower efficiency score was also included as an additional dependent variable for the Brazilian sample as a measure of cognitive flexibility. Results showed significant positive correlations for the HTKS total score (r = −0.36, p < 0.01) and the HTKS efficiency score (r = 0.30, p < 0.05) with the Heart and Flower efficiency score.
To further assess the HTKS construct validity, a series of regression models were run with digit span as the dependent variable for both Brazilian and US children. All models controlled for age and sex. For the Brazilian sample, a significant positive relationship (HTKS, ß = 0.36, p = 0.024) and a significant negative relationship (HTKS Efficiency, ß = −0.34, p = 0.019) were found for the digit span task. Results for the Heart and Flower efficiency score in the Brazil sample were also significant and in the same direction for the HTKS (ß = 0.29, p = 0.047) and the HTKS efficiency score (ß = −0.41, p = 0.012). Results for the HTKS and HTKS efficiency score for the U.S. sample were in the same direction as the Brazilian sample, but only the HTKS efficiency score was significantly negatively related to the digit span task (HTKS efficiency, ß = −0.29, p = 0.024, see Table 4).
Table 4. Linear regressions controlling for sex and age.
Discussion
In this study, we investigated the initial variability and range in scores for Brazilian and US adolescent children on the HTKS, a well-validated measure of EF used in preschool and kindergarten populations. We also expanded past work with the HTKS (e.g., McClelland et al., 2014; Kenny et al., 2023) by creating an efficiency score, reducing the potential ceiling effects found with older populations. Results indicated that there was sufficient variability and range in HTKS efficiency scores, with the Brazilian and US sample scoring at the ceiling for only 2 and 4%, respectively. For the HTKS total score, 39% of the US children scored at the ceiling, while only 4% of Brazilian children scored at ceiling levels. It has been suggested that psychometric instruments must avoid ceiling and floor effects in order to produce more reliable measures (Diamond, 2012). The Dutra et al. (2022) study included a Brazilian sample of adolescents, and the mean HTKS total score approached the ceiling. This was cited as a possible reason for the small correlations found in the study. Ceiling or floor effects may prevent the detection of significant patterns of correlations among different cognitive measures, especially in children from developing countries (Spiegel and Lonigan, 2018; Gonzales et al., 2021). In this sense, the HTKS efficiency score, which indicates faster processing for accurate answers and better cognitive functioning, proved to be a more satisfactory measure for both samples in the present study.
The T-test results reported significant differences between both samples and significantly lower performance in the Brazilian sample. This finding corroborates previous literature that has reported lower levels of EF in children from low socioeconomic status countries (Sarsour et al., 2010; Lipina and Segretin, 2019). Crowded home environments and dangerous neighborhoods may contribute to lower EF skills (Ribeiro et al., 2016).
Second, we examined whether the HTKS was associated with measures of EF in two samples of adolescents, from Brazil and the US. We found small significant correlations between measures of EF (working memory), in the HTKS total score and when using the measure of HTKS efficiency with the Brazilian sample. This was not the case with the American sample, which showed a trend in the correlation for the HTKS efficiency measure (p < 0.1) although the magnitude of the correlation was similar to that of the Brazilian sample. These results could be explained by the larger variability with lower scores in Brazil compared to the US and the better educational and socioeconomic environment in the US.
Regression analyzes that controlled for sex and age indicated that the HTKS efficiency score was negatively and significantly related to the digit span measures in both Brazil and US samples. Shorter time spent on accurate responses was significantly related to better working memory. The importance of analyzing cognitive efficiency was corroborated by the results with the Heart and Flower score as the dependent variable in the Brazil sample. The measures of accuracy and efficiency of the HTKS were correlated with Heart and Flower efficiency, but the stronger correlation was with the HTKS efficiency score. The HTKS efficiency score was consistently related to working memory, and with the Heart and Flower task, which is a measure of cognitive flexibility. At school, cognitive efficiency is crucial in learning in classrooms, where there can be a lot of content covered in a limited time (Hoffman, 2012). These results show that scores on the HTKS were related to scores on measures of WM and cognitive flexibility and give initial support for the construct validity of the HTKS in older children in the US and Brazil.
Implications for research and practice
Our study highlights the importance of using measures of cognitive efficiency with older children and supported the initial validity of a measure of EF with strong ecological validity (the HTKS) to other measures of EF (e.g., working memory and cognitive flexibility). Researchers and educators can assess cognitive performance in a naturalistic setting, with an easy-to-use and psychometrically valid measure of EF. In addition, results supported using the HTKS in diverse populations from two different countries, showing that it can be a useful tool to be applied in low and high socioeconomic status countries.
Although the study had a number of strengths, there were also limitations. First, although the study was one of the first to look at the initial reliability and validity of the HTKS with adolescents, the sample sizes for the Brazil and US samples were small, and this limited the power to detect effects. Future research should replicate these findings with larger samples from multiple cultural contexts. Second, the digit span task used in the Brazilian sample was computerized with the US sample using a pencil and paper version. However, in both samples, the two error maximum length digit span was used in the analysis. Third, both samples had a lack of additional demographic data, such as family socioeconomic status, and ethnicity making it difficult to draw conclusions about the social context that could be influencing the outcomes. Inferences and tests of construct validity in the US sample are limited by the lack of an additional measure of EF, such as the Heart and Flowers in the Brazilian sample.
Conclusion
Results support the variability and initial validity of the HTKS in two samples of adolescents from Brazil and the US. Including a cognitive efficiency score in the HTKS provided greater validity compared to the total HTKS accuracy score. This allowed the HTKS to assess a broader range of performance in diverse populations of adolescents. Lower educational and socio-economic contexts can negatively influence children’s EF, however the HTKS efficiency score provided sufficient range to assess performance for both populations. Future studies should investigate connections among the HTKS efficiency score with different EF assessments and with measures of academic performance.
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Ethics statement
The studies involving humans were approved by Comitê de Ética e Pesquisa do Centro Universitário Augusto Motta (UNISUAM) (CAAE 11980119.4.0000.5235). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by the participants' legal guardians/next of kin.
Author contributions
VF lead efforts on conceptualizing, writing, reviewing results, and revising drafts. DB lead efforts on data analyzing and writing the results section. MM contributed to conceptualizing, writing, and reviewing drafts. AD assisted with data analyzes, methods, and reviewing drafts. All authors contributed to the article and approved the submitted version.
Funding
The study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Participants and their parents were informed about the experimental procedure and signed an Informed Consent Form (ICF) and Statement of Consent.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Ahmed, S. F., Tang, S., Waters, N. E., and Davis-Kean, P. (2019). Executive function and academic achievement: longitudinal relations from early childhood to adolescence. J. Educ. Psychol. 111, 446–458. doi: 10.1037/EDU0000296
Baumgartner, S. E., Weeda, W. D., van der Heijden, L. L., and Huizinga, M. (2014). The relationship between media multitasking and executive function in early adolescents. J. Early Adolesc. 34, 1120–1144. doi: 10.1177/0272431614523133
Becker, D. R., and Nader, P. A. (2021). Run fast and sit still: connections among aerobic fitness, physical activity, and sedentary time with executive function during pre-kindergarten. Early Child. Res. Q. 57, 1–11. doi: 10.1016/j.ecresq.2021.04.007
Best, J. R., and Miller, P. H. (2010). A developmental perspective on executive function. Child Dev. 81, 1641–1660. doi: 10.1111/j.1467-8624.2010.01499.x
Best, J. R., Miller, P. H., and Naglieri, J. A. (2011). Relations between executive function and academic achievement from ages 5 to 17 in a large, representative national sample. Learn. Individ. Differ. 21, 327–336. doi: 10.1016/j.lindif.2011.01.007
Blair, C., and Ku, S. (2022). A hierarchical integrated model of self-regulation. Front. Psychol. 13:245. doi: 10.3389/FPSYG.2022.725828
Cameron Ponitz, C. E., McClelland, M. M., Jewkes, A. M., Connor, C. M., Farris, C. L., and Morrison, F. J. (2007). Touch your toes! Developing a direct measure of behavioral regulation in early childhood. Early Child. Res. Q. 23, 141–158. doi: 10.1016/j.ecresq.2007.01.004
Carden, J., and Cline, T. (2015). Problem solving in mathematics: the significance of visualisation and related working memory. Educ. Psychl. Prac. 31, 235–246. doi: 10.1080/02667363.2015.1051660
Centeio, E. E., Somers, C. L., Moore, E. W. G., Kulik, N., Garn, A., Martin, J., et al. (2018). Relationship between academic achievement and healthy school transformations in urban elementary schools in the United States. Phys. Educ. Sport Pedagog. 23, 402–417. doi: 10.1080/17408989.2018.1441395
Cerino, E. S., Hooker, K., Stawski, R. S., and McClelland, M. (2018). A new brief measure of executive function: adapting the head-toes-knees-shoulders task to older adults. Gerontologist 59, e258–e267. doi: 10.1093/geront/gny028
Coffman, I., Resnick, H. E., Drane, J., and Lathan, C. E. (2017). Computerized cognitive testing norms in active-duty military personnel: Potential for contamination by psychologically unhealthy individuals. App. Neuropsych.: Adults. 25, 497–503. doi: 10.1080/23279095.2017.1330749
Conway, A. R. A., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., and Engle, R. W. (2005). Working memory span tasks: A methodological review and user’s guide. Psychon. Bull. Rev. 12, 769–786. doi: 10.3758/BF03196772
Dadvand, P., Nieuwenhuijsen, M. J., Esnaola, M., Forns, J., Basagaña, X., Alvarez-Pedrerol, M., et al. (2015). Green spaces and cognitive development in primary schoolchildren. Proc. Natl. Acad. Sci. U. S. A. 112, 7937–7942. doi: 10.1073/PNAS.1503402112/SUPPL_FILE/PNAS.1503402112.SAPP.PDF
Davidson, M. C., Amso, D., Anderson, L. C., and Diamond, A. (2006). Development of cognitive control and executive functions from 4 to 13 years: evidence from manipulations of memory, inhibition, and task switching. Neuropsychologia 44, 2037–2078. doi: 10.1016/j.neuropsychologia.2006.02.006
de Moraes, C. L., dos Santos, É. B., Reichenheim, M. E., Taquette, S. R., Stochero, L., and Marques, E. S. (2021). Perceived experiences of community violence among adolescents: A school survey from Rio de Janeiro, Brazil. J. Interpers. Violence. 37, NP14588–NP14609. doi: 10.1177/08862605211005132
de Souza, P. H. G. F., and De Carvalhaes, F. A. O. (2014). Estrutura de Classes, Educação e Queda da Desigualdade de Renda (2002-2011). Dados 57, 101–128. doi: 10.1590/S0011-52582014000100004
Diamond, A. (2012). Activities and programs that improve children’s executive functions. Curr. Dir. Psychol. Sci. 21, 335–341. doi: 10.1177/0963721412453722
Diamond, A. (2013). Executive functions. Annu. Rev. Psychol. 64, 135–168. doi: 10.1146/annurev-psych-113011-143750
Diamond, A., Lee, C., Senften, P., Lam, A., and Abbott, D. (2019). Randomized control trial of tools of the mind: marked benefits to kindergarten children and their teachers. PLoS One 14:e0222447. doi: 10.1371/journal.pone.0222447
Duckworth, A. L., Tsukayama, E., and May, H. (2010). Establishing causality using longitudinal hierarchical linear modeling: an illustration predicting achievement from self-control. Soc. Psychol. Personal. Sci. 1, 311–317. doi: 10.1177/1948550609359707
Dutra, N. B., Chen, L., Anum, A., Burger, O., Davis, H. E., Dzokoto, V. A., et al. (2022). Examining relations between performance on non-verbal executive function and verbal self-regulation tasks in demographically-diverse populations. Dev. Sci. 25:e13228. doi: 10.1111/desc.13228
Friedman, N. P., and Robbins, T. W. (2022). The role of prefrontal cortex in cognitive control and executive function. Neuropsychopharmacology 47, 72–89. doi: 10.1038/s41386-021-01132-0
Gentile, A., Boca, S., Şahin, F. N., Güler, Ö., Pajaujiene, S., Indriuniene, V., et al. (2020). The effect of an enriched sport program on Children’s executive functions: the ESA program. Front. Psychol. 11. doi: 10.3389/fpsyg.2020.00657
Gonzales, C. R., Bowles, R., Geldhof, G. J., Cameron, C. E., Tracy, A., and McClelland, M. M. (2021). The head-toes-knees-shoulders revised (HTKS-R): development and psychometric properties of a revision to reduce floor effects. Early Child. Res. Q. 56, 320–332. doi: 10.1016/j.ecresq.2021.03.008
Gunter, K. B., Nader, P. A., and John, D. H. (2015). Physical activity levels and obesity status of Oregon rural elementary school children. Prev. Med. Rep. 2, 478–482. doi: 10.1016/j.pmedr.2015.04.014
Hackman, D. A., and Farah, M. J. (2009). Socioeconomic status and the developing brain. Trends Cogn. Sci. 13, 65–73. doi: 10.1016/j.tics.2008.11.003
Hoffman, B. (2012). Cognitive efficiency: A conceptual and methodological comparison. Learn. Instr. 22, 133–144. doi: 10.1016/j.learninstruc.2011.09.001
Igazság, B., Demetrovics, Z., and Cserjési, R. (2019). The developmental trajectory of executive functions and their stress sensitivity in adolescence. Psychiatr. Hung. 34, 300–310.
Joens-Matre, R. R., Welk, G. J., Calabro, M. A., Russell, D. W., Nicklay, E., and Hensley, L. D. (2008). Rural–Urban differences in physical activity, physical fitness, and overweight prevalence of children. J. Rural. Health 24, 49–54. doi: 10.1111/J.1748-0361.2008.00136.X
Kenny, S. A., Cameron, C. E., Karing, J. T., Ahmadi, A., Braithwaite, P. N., and McClelland, M. M. (2023). A meta-analysis of the validity of the head-toes-knees-shoulders task in predicting young children’s academic performance. Front. Psychol. 14. doi: 10.3389/fpsyg.2023.1124235
Latzman, R. D., Elkovitch, N., Young, J., and Clark, L. A. (2010). The contribution of executive functioning to academic achievement among male adolescents. J. Clin. Exp. Neuropsychol. 32, 455–462. doi: 10.1080/13803390903164363
Lee, J. S., Boafo, A., Greenham, S., and Longmuir, P. E. (2019). The effect of high-intensity interval training on inhibitory control in adolescents hospitalized for a mental illness. Ment. Health Phys. Act. 17:100298. doi: 10.1016/j.mhpa.2019.100298
Lipina, S. J., and Segretin, M. S. (2019). “Exploraciones neurocientíficas de la pobreza” in International school on mind, brain and education. eds. S. J. Lipina and M. S. Segretin (Erice, Italy: International School on Mind, Brain and Education, Ettore Majorana Foundation and Center for Scientific Culture)
Ludyga, S., Gerber, M., and Kamijo, K. (2022). Exercise types and working memory components during development. Trends Cogn. Sci. 26, 191–203. doi: 10.1016/j.tics.2021.12.004
Luna, B., Garver, K. E., Urban, T. A., Lazar, N. A., and Sweeney, J. A. (2004). Maturation of cognitive processes from late childhood to adulthood. Child Dev. 75, 1357–1372. doi: 10.1111/J.1467-8624.2004.00745.X
McClelland, M. M., Acock, A. C., Piccinin, A., Rhea, S. A., and Stallings, M. C. (2013). Relations between preschool attention span-persistence and age 25 educational outcomes. Early Child. Res. Q. 28, 314–324. doi: 10.1016/j.ecresq.2012.07.008
McClelland, M. M., Cameron, C. E., Duncan, R., Bowles, R. P., Acock, A. C., Miao, A., et al. (2014). Predictors of early growth in academic achievement: the head-toes-knees-shoulders task. Front. Psychol. 5, 1–14. doi: 10.3389/fpsyg.2014.00599
McClelland, M. M., Geldhof, G. J., Cameron, C. E., and Wanless, S. B. (2015). Development and self-regulation. Handb. Child Psychol. Dev. Sci. 1–43. doi: 10.1002/9781118963418.CHILDPSY114
McClelland, M. M., Gonzales, C. R., Cameron, C. E., Geldhof, G. J., Bowles, R. P., Nancarrow, A. F., et al. (2021). The head-toes-knees-shoulders revised: links to academic outcomes and measures of EF in Young children. Front. Psychol. 12. doi: 10.3389/fpsyg.2021.721846
Medeiros, M., Barbosa, R. J., and Carvalhaes, F. (2020). Educational expansion, inequality and poverty reduction in Brazil: A simulation study. Res. Soc. Stratif. Mobil. 66:100458. doi: 10.1016/j.rssm.2019.100458
Miller, S. L., Chelian, S., McBurnett, W., Tsou, W., and Kruse, A. A. (2019). An investigation of computer-based brain training on the cognitive and EEG performance of employees. In 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC) (IEEE), 518–521.
Moffitt, T. E., Arseneault, L., Belsky, D., Dickson, N., Hancox, R. J., Harrington, H., et al. (2011). A gradient of childhood self-control predicts health, wealth, and public safety. Proc. Natl. Acad. Sci. U. S. A. 108, 2693–2698. doi: 10.1073/pnas.1010076108
Nigg, J. T., Wong, M. M., Martel, M. M., Jester, J. M., Puttler, L. I., Glass, J. M., et al. (2006). Poor response inhibition as a predictor of problem drinking and illicit drug use in adolescents at risk for alcoholism and other substance use disorders. J. Am. Acad. Child Adolesc. Psychiatry 45, 468–475. doi: 10.1097/01.chi.0000199028.76452.a9
O’Malley, M., Voight, A., Renshaw, T. L., and Eklund, K. (2015). School climate, family structure, and academic achievement: A study of moderation effects. Sch. Psychol. Q. 30, 142–157. doi: 10.1037/SPQ0000076
Ponitz, C. C., McClelland, M. M., Matthews, J. S., and Morrison, F. J. (2009). A structured observation of behavioral self-regulation and its contribution to kindergarten outcomes. Dev. Psychol. 45, 605–619. doi: 10.1037/a0015365
Raghubar, K. P., Barnes, M. A., and Hecht, S. A. (2010). Working memory and mathematics: A review of developmental, individual difference, and cognitive approaches. Learn. Individ. Differ. 20, 110–122. doi: 10.1016/J.LINDIF.2009.10.005
Reinert, K. R. S., Po’e, E. K., and Barkin, S. L. (2013). The relationship between executive function and obesity in children and adolescents: a systematic literature review. J. Obes. 2013:820956. doi: 10.1155/2013/820956
Ribeiro, S., Mota, N. B., Fernandes Da, V. R., Deslandes, A. C., Brockington, G., and Copelli, M. (2016). Physiology and assessment as low-hanging fruit for education overhaul. Prospects 46, 249–264. doi: 10.1007/s11125-017-9393-x
Robson, D. A., Allen, M. S., and Howard, S. J. (2020). Self-regulation in childhood as a predictor of future outcomes: A Meta-analytic review. Psychol. Bull. 146, 324–354. doi: 10.1037/BUL0000227
Ronan, L., Alexander-Bloch, A., and Fletcher, P. C. (2020). Childhood obesity, cortical structure, and executive function in healthy children. Cereb. Cortex 30, 2519–2528. doi: 10.1093/CERCOR/BHZ257
Rosas, R., Espinoza, V., Porflitt, F., and Ceric, F. (2019). Executive functions can be improved in preschoolers through systematic playing in educational settings: evidence from a longitudinal study. Front. Psychol. 10, 1–12. doi: 10.3389/fpsyg.2019.02024
Rubia, K. (2011). “Cool” inferior frontostriatal dysfunction in attention-deficit/hyperactivity disorder versus “hot” ventromedial orbitofrontal-limbic dysfunction in conduct disorder: A review. Biol. Psychiatry 69, e69–e87. doi: 10.1016/j.biopsych.2010.09.023
Ruediger, M. A. (2017). Education on target: the effects of armed violence on the classrooms. 1st ed., DAPP, FGV: Rio de janeiro.
Samuels, W. E., Tournaki, N., Blackman, S., and Zilinski, C. (2016). Executive functioning predicts academic achievement in middle school: a four-year longitudinal study. J. Educ. Res. 109, 478–490. doi: 10.1080/00220671.2014.979913
Sarsour, K., Sheridan, M., Jutte, D., Nuru-Jeter, A., Hinshaw, S., and Boyce, W. T. (2010). Family socioeconomic status and child executive functions: the roles of language, home environment, and single parenthood. J. Int. Neuropsychol. Soc. 17, 120–132. doi: 10.1017/S1355617710001335
Schafer, J. L., and Graham, J. W. (2002). Missing data: our view of the state of the art. Psychol. Methods 7, 147–177. doi: 10.1037/1082-989X.7.2.147
Spiegel, J. A., and Lonigan, C. J. (2018). A head-to-toes approach to computerized testing of executive functioning in young children. Early Child. Res. Q. 44, 15–23. doi: 10.1016/j.ecresq.2018.01.008
Spiegler, O., and Leyendecker, B. (2017). Balanced cultural identities promote cognitive flexibility among immigrant children. Front. Psychol. 8:1579. doi: 10.3389/FPSYG.2017.01579/BIBTEX
Stein, M., Auerswald, M., and Ebersbach, M. (2017). Relationships between motor and executive functions and the effect of an acute coordinative intervention on executive functions in kindergartners. Front. Psychol. 8. doi: 10.3389/FPSYG.2017.00859/FULL
UNDP (United Nations Development Programme). (2022). Human Development Report 2021–22. United States. UNDP (United Nations Dev Program).
Watts, T. W., Gandhi, J., Ibrahim, D. A., Masucci, M. D., and Raver, C. C. (2018). The Chicago school readiness project: examining the long-term impacts of an early childhood intervention. PLoS One 13:e0200144. doi: 10.1371/journal.pone.0200144
Wende, M. E., Stowe, E. W., Eberth, J. M., McLain, A. C., Liese, A. D., Breneman, C. B., et al. (2021). Spatial clustering patterns and regional variations for food and physical activity environments across the United States. Int. J. Environ. Health Res. 31, 976–990. doi: 10.1080/09603123.2020.1713304
Willoughby, M. T., Blair, C. B., Wirth, R. J., and Greenberg, M. (2012). The measurement of executive function at age 5: psychometric properties and relationship to academic achievement. Psychol. Assess. 24, 226–239. doi: 10.1037/a0025361
Wright, A., and Diamond, A. (2014). An effect of inhibitory load in children while keeping working memory load constant. Front. Psychol. 5, 1–9. doi: 10.3389/fpsyg.2014.00213
Woods, D. L., Kishiyama, M. M., Yund, E. W., Herron, T. J., Edwards, B., Poliva, O., et al. (2011). Improving digit span assessment of short-term verbal memory. J. Clin. Exp. Neuropsychol. 33, 101–111. doi: 10.1080/13803395.2010.493149
Yu, E., and Lippert, A. M. (2016). Neighborhood crime rate, weight-related behaviors, and obesity: A systematic review of the literature. Sociol. Compass 10, 187–207. doi: 10.1111/soc4.12356
Keywords: self-regulation, executive function, cognitive development, measurement, academic achievement
Citation: Fernandes VR, Becker DR, McClelland MM and Deslandes AC (2023) Head-Toes-Knees-Shoulders task and EF in two samples of adolescents in Brazil and United States. Front. Psychol. 14:1149053. doi: 10.3389/fpsyg.2023.1149053
Edited by:
Evely Boruchovitch, State University of Campinas, BrazilReviewed by:
Josie Booth, University of Edinburgh, United KingdomRachel Razza, Syracuse University, United States
Copyright © 2023 Fernandes, Becker, McClelland and Deslandes. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Valter R. Fernandes, Y2Fwb2VpcmFjdXJ1bWltQGhvdG1haWwuY29t