Moses K. Nyongesa1*
Derrick Ssewanyana1,2
Agnes M. Mutua1
Esther Chongwo1
Gaia Scerif3
Charles R. J. C. Newton1,4,5
Amina Abubakar1,4,5,6- 1Neuroassement Group, KEMRI/Wellcome Trust Research Programme, Centre for Geographic Medicine Research (Coast), Kilifi, Kenya
- 2Utrecht Centre for Child and Adolescent Studies, Utrecht University, Utrecht, Netherlands
- 3Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- 4Department of Public Health, Pwani University, Kilifi, Kenya
- 5Department of Psychiatry, University of Oxford, Oxford, United Kingdom
- 6Institute for Human Development, Aga Khan University Hospital, Nairobi, Kenya
Background: There is much research examining adolescents' executive function (EF) but there is little information about tools that measure EF, in particular preference of use, their reliability and validity. This information is important as to help both researchers and practitioners select the most relevant and reliable measure of EF to use with adolescents in their context.
Aims: We conducted a scoping review to: (a) identify the measures of EF that have been used in studies conducted among adolescents in the past 15 years; (b) identify the most frequently used measures of EF; and (c) establish the psychometric robustness of existing EF measures used with adolescents.
Methods: We searched three bibliographic databases (PsycINFO, Ovid Medline, and Web of Science) using key terms “Adolescents,” “Executive Functions,” and “measures”. The search covered research articles published between 1st January 2002 and 31st July 2017.
Results: We identified a total of 338 individual measures of EF from 705 eligible studies. The vast majority of these studies (95%) were conducted in high income countries. Of the identified measures, 10 were the most used frequently, with a cumulative percent frequency accounting for nearly half (44%) the frequency of usage of all reported measures of EF. These are: Digit Span (count = 160), Trail Making Test (count = 158), Behavior Rating Inventory of Executive Function (count = 148), Wisconsin Card Sorting Test (count = 140), Verbal Fluency Tasks (count = 88), Stroop Color-Word Test (count = 78), Classical Stroop Task (count = 63), Color-Word Interference Test from Delis-Kaplan battery (count = 62), Rey-Osterrieth Complex Figure Test (count = 62), and Original Continuous Performance Test (count = 58). In terms of paradigms, tasks from Span (count = 235), Stroop (count = 216), Trails (count = 171), Card sorting (count = 166), Continuous performance (count = 99), and Tower (count = 94) paradigms were frequently used. Only 48 studies out of the included 705 reported the reliability and/or validity of measures of EF used with adolescents, but limited to studies in high income countries.
Conclusion: We conclude that there is a wide array of measures for assessing EF among adolescents. Ten of these measures are frequently used. However, the evidence of psychometric robustness of measures of EF used with adolescents remains limited to support the validity of their usage across different contexts.
Introduction
General Background
Executive function (EF), also known as executive control or cognitive control, is an umbrella term that describes a set of inter-related but distinct cognitive abilities. These cognitive abilities, mediated by the prefrontal cortex (Siddiqui et al., 2008) include, but are not limited to: planning, shifting (i.e., flexibility of thought and action), fluency (i.e., generation of new responses), problem solving, decision making, self-regulation, attentional control, working memory (i.e., concurrent remembering and processing), inhibitory control, and cognitive flexibility (Miyake and Friedman, 2012; Burnett et al., 2013; Costanzo et al., 2013).
Currently, consensus is lacking as to the precise components of EF since it is a multi-faceted construct. Converging research (e.g., Collins and Koechlin, 2012; Lunt et al., 2012; Miyake and Friedman, 2012; Hall and Marteau, 2014; Karbach and Unger, 2014) suggests that EF may be conceptualized best as comprising of three distinct yet related “core” dimensions: working memory, inhibitory control, and cognitive flexibility. Other authors (e.g., Tsermentseli and Poland, 2016; Zimmerman et al., 2016; Poon, 2017) have described EF in terms of “cool” and “hot” components. Cool cognitive skills are elicited under relatively abstract, decontextualized, and non-emotional conditions (Peterson and Welsh, 2014) and require logic and critical analysis (Rubia, 2011). Examples include planning, verbal reasoning, problem-solving, sequencing, cognitive flexibility, working memory, the ability to sustain attention, behavioral monitoring, and inhibition. Hot cognitive skills, in contrast, are elicited in contexts that require personal interpretation where emotions, motivation, or a tension between instant gratification and long-term rewards are generated (Zelazo and Carlson, 2012). Affective cognitive abilities such as social cognition, emotional regulation, affective decision making, and the ability to delay gratification, are posited as aspects of hot EF.
Instead of focusing on individual elements of EF, other investigators in the field have chosen to use theoretical underpinnings on EF for their research purposes. For instance, Burnett et al. (2013), in reviewing the literature on EF and everyday behavior, adopted the Executive Control System conceptual framework (Anderson, 2002; Anderson and Reidy, 2012). This conceptual framework categorizes EF into four broad domains: (i) information processing; (ii) attentional control; (iii) cognitive flexibility; and (iv) goal setting, each consisting of various components tapping into EF. Such a broad categorization, on one hand, overcomes the need of having to focus on components when studying EF. On the other hand, the broadness may lose the precision on the exact construct of EF being studied.
Despite the current lack of consensus about the precise components of EF, it is generally agreed that EF is important for enabling an individual not only to control their emotions and socially interact (Anderson, 2002; Xanthopoulos et al., 2015) but also engage in independent, purposeful, and goal-oriented behavior (Lezak et al., 2004).
Executive Function in Adolescence
EF skills play an important role in shaping an adolescent's behavior and promoting her/his socio-emotional and educational competencies (Riggs et al., 2006; Bierman et al., 2008). An important aspect of EF is the ability to adaptively respond in circumstances that prime inappropriate and/or prepotent responses, which can lead to impetuous acts or errors in judgment (Prencipe et al., 2011). Adolescence, a period of increasing autonomy, may be of particular vulnerability to such errors partly because EF continues to develop throughout this period (Best and Miller, 2010; Taylor et al., 2015). Moreover, transitioning to adolescence is often followed by a new set of challenging responsibilities and self-regulatory demands for example in educational and social spheres (Burnett et al., 2013) requiring greater reliance on the emerging cognitive control.
It is noteworthy that EF in childhood and adolescence is a predictor of adult productivity and future life outcomes (Diamond, 2013). Therefore, the need to monitor, screen and intervene for EF problems early in life cannot be overemphasized.
Tools Used to Assess Executive Function at Adolescence
Studying EF in youth populations has received special attention in the recent years, given that it influences their behavioral, social, emotional, and academic outcomes (Arán Filippetti and Richaud, 2015). A wide range of performance-based measures of EF exists and have been used with adolescent sub-population for years. These include tasks such as the Wisconsin Card Sorting Test (WCST; Heaton, 1981); the Trail Making Test (TMT; Reitan, 1992); and the Stroop Color-Word Test (SCWT; Golden and Freshwater, 1978). To assess aspects of EF comprehensively, some performance-based EF tests are administered as a set, in neuropsychological batteries such as the Delis-Kaplan Executive Function System (D-KEFS; Delis et al., 2001), Cambridge Neuropsychological Test Automated Battery (CANTAB; Cambridge Cognition Ltd, 2006), the Behavioral Assessment of the Dysexecutive Syndrome for Children (BADS-C; Emslie et al., 2003), and a developmental Neuropsychological assessment battery (NEPSY; Korkman et al., 1998).
In the new millennium, researchers have also begun to broaden ways of assessing EF among children and adolescents by including self or informant reported questionnaires designed to index children's everyday EF skills. Examples of such EF rating scales include measures such as the Behavior Rating Inventory of Executive Function (BRIEF; Gioia et al., 2000; Guy et al., 2004), the Dysexecutive Questionnaire for Children (DEX-C; Emslie et al., 2003), Amsterdam Executive Function Inventory (AEFI; Van der Elst et al., 2012) and most recently the Dynamic Occupation Assessment of Executive Function (DOAEF; Chubarov et al., 2015).
Rationale for the Present Study
Despite the existence of a wide array of neuropsychological measures of EF for use with adolescents, little is known about the most preferred (frequently used) measures for this sub-population. Where literature review has been reported, this has been limited to a specific adolescent sub-population such as those living with cerebral palsy (see Pereira et al., 2018). Furthermore, synthesized and summarized information about the psychometric robustness of existing EF measures for use with adolescents i.e., their reliability and ecological validity remains unknown yet this is known among the child (Henry and Bettenay, 2010) and adult population (Pickens et al., 2010). For researchers, neuropsychologists and related practitioners selecting a measure of EF for use in research or clinical purposes, current information on preference, reliability, and validity of EF measures for use with adolescents is essential in helping them make an informed decision.
The aim of this study is therefore to address the above-mentioned knowledge gaps. Specifically, the study examines the following research questions:
1. Which measures are used to assess EF among adolescents within the past 15 years (between January 2002 and July 2017)?
2. Among the identified measures of EF, which are the most preferred or frequently utilized?
3. What is the psychometric robustness of the existing EF measures for use with adolescents?
Methods
A scoping review was undertaken. Scoping reviews are useful for mapping out and summarizing existing literature on a specific topic in order to assist researchers in identifying the extent, range and nature of the current research evidence (Levac et al., 2010). Our focus was on measures of EF used in studies conducted among adolescents and reported within the last one and a half decade, the aim being to capture the most recent evidence in the field. A methodologically rigorous scoping review framework proposed by Arksey and O'Malley (Arksey and O'Malley, 2005) was applied in the current study. This framework involves five key phases: (i) identifying research question(s); (ii) identifying relevant studies; (iii) study selection; (iv) Extracting and charting the data; (v) collating, summarizing, and reporting the results. Our research questions are listed in the introduction section.
Identifying Relevant Studies
A search was conducted in three electronic bibliographic databases, that is, PsycINFO, Ovid MEDLINE, and Web of Science. The search terms comprised of the key words “Adolescents,” “Executive Functions,” and “measures” which were combined using the AND Boolean operator. Synonyms for each of the key terms were combined using the OR Boolean operator (see Appendix 1 in Supplementary material). The search was limited to only peer reviewed articles, articles in English language, published between 1st January 2002 and 31st July 2017. Where a database could allow, we restricted the search to adolescence age group of interest (13–17 years). Dissertations and book chapters were filtered out.
Study Eligibility
Table 1 summarizes the criteria used in the selection of eligible studies for our scoping review. Four authors (MKN, DS, AM, and EC) screened the titles, abstracts, and full-text articles for study eligibility. Where disagreement or doubt arose concerning inclusion or exclusion of an article, the four re-evaluated the article and reached a consensus. Consensus to either include or exclude an article occurred on 25 occasions. Figure 1 illustrates the selection process of the articles.
Table 1. Article inclusion and exclusion criteria.
Figure 1. Selection of articles.
Data Extraction
From each study fulfilling the inclusion criteria, information on: author, year, and journal of publication, country in which study was conducted, the measure of EF used in the study and psychometric properties of the measure (if documented) were extracted and charted into Microsoft office Excel (version 2016). For psychometric data, we extracted Cronbach's alpha, intra-class coefficient of correlation (ICC) or any other correlation coefficient, if reported, when documenting the reliability of a given measure. Where a study explored validity of a given measure of EF, we documented the type of validity examined such as construct, content, criterion, concurrent, divergent, or convergent validity, alongside supporting statistics.
Analysis
We first counted the number of individual measures of EF identified from the review. Then, we computed the frequency of use of each individual measure and summed the frequencies. To describe the most preferred or frequently used measure of EF, we developed a priori working definition, that is, a measure of EF should have accounted for ≥2.5% of the summed frequency of usage of all individual measures of EF (equivalent to a frequency count ≥58 from different included publications). For this analysis, percentage frequencies and cumulative percentage frequencies of measures of EF were computed in Microsoft office Excel (version 2016). We did computations using two approaches. First, we computed frequencies of usage of individual measures of EF (i.e., how frequent an individual measure was used in all the included studies). Second, we grouped measures of EF which we deemed as having a similar underlying principle of measurement (similar underlying latent factor) and computed the frequencies based on this grouping. Data on psychometric properties from the eligible studies were abstracted into a table on Microsoft Office Word (2016). We also coded the countries (and their respective continents) in which the included studies were conducted. These countries were categorized according to World Bank's income ranking (World Bank, 2017). Data were then imported to STATA (version 14.0) statistical software package (StataCorp, 2015) for univariate analyses (frequency and percentage distribution).
Results
Measures of EF Used With Adolescents
This scoping review identified 705 eligible studies as shown in the study selection flowchart (Figure 1). From these studies, we identified a total of 338 individual measures that have been used to assess an aspect of EF among adolescents aged 13–17 years (see Appendix 2 in Supplementary material). Most studies used multiple measures and the total frequency count of all the identified individual measures of EF was 2328. Majority of these measures were performance based, with only 13 out of the 338 identified being self- or informant-reported rating scales of EF. Of the 338 measures of EF identified, 10 were most frequently used (Table 2). The cumulative percent frequency of these 10 frequently used individual measures of EF was 43.7%, nearly half of the total frequency count of usage of all identified measures (see Table 2). Appendix 3 in Supplementary material presents a summary of the administration procedures of these 10 most frequent measures of EF.
Table 2. Frequency count, percentage and cumulative percent frequency of the frequently used individual measures tapping into adolescents' EF.
Of the 338 individual measures identified, we grouped 72 tasks into 12 paradigms of EF namely Cancellation, Card sorting, Continuous Performance Tests (CPT), Go/No-go, Flanker, Hayling & Brixton, Maze, N-back, Span, Stroop, Tower, and Trails tasks (Table 3). These paradigms consisted of individual measures of EF that assess a common underlying latent factor. The frequency count for these paradigms was 1,116 (47.9% of the total frequency count of all identified measures). Tasks from the Span (count = 235), Stroop (count = 216), Trails (count = 171), Card sorting (count = 166), CPT (count = 99), and Tower (count = 94) paradigms were frequently used. Figure 2 shows how the identified measures of EF were categorized.
Table 3. Test variants grouped into paradigms.
Figure 2. Categorization of identified measures of EF.
A breakdown of the results from the analysis of the regional and income ranking of countries where the EF measures were utilized are presented in Appendix 4 in Supplementary material. In summary, majority of the EF assessments among adolescents aged 13–17 years were conducted either in North America (n = 325, 46.1%) or Europe (n = 277, 39.3%). Consequently, included studies were mainly conducted in high income countries (n = 667, 94.6%) compared to a meager 0.3% representation of low income countries.
Psychometric Robustness of Identified Measures of EF
Of the 705 included studies in the scoping review, only 48 reported an aspect of reliability and/or validity of a measure of EF used with the adolescent sub-population in a given study setting. These study settings were all high-income countries except for two studies (Wong et al., 2012; Malek et al., 2013) that were conducted in an upper middle-income economic setting of Cuba and Iran. More than half of these 48 studies (n = 28) reported the psychometric characteristics of the self- or informant-reported rating scales, with n = 22 specifically reporting the psychometrics of the BRIEF (Gioia et al., 2000; Guy et al., 2004; Table 4). Reported internal consistency of the BRIEF ranged from 0.65 to 0.98 in the context of high income countries (both informant- and self-reports; see Table 4 for details). Only one study from United States (Rose and Holmbeck, 2007) reported the inter-rater reliability for the BRIEF (informant report version) which was excellent (0.96 to 0.98). The test-retest reliability for the BRIEF ranged between 0.81 and 0.86. Validity aspects of the BRIEF that were examined across studies included construct, concurrent, and discriminant validity. Presented results, though in the context of high income countries, indicate that BRIEF is a valid measure of EF (see Table 4). It is only in one study from Netherlands (Huizinga and Smidts, 2011), where the Root Mean Square of Error of Approximation (RMSEA) value for the parent-report version of the BRIEF (an estimate of construct validity) was 0.11 which is greater than the recommended value of <0.06 (Thompson, 2004). However, an alternative estimate, Non-Normed Fit Index (NNFI) was excellent at 0.92.
Table 4. Reported psychometric properties of EF measures used with adolescents.
The six additional rating scales with reported psychometrics were: the Dynamic Occupation Assessment of Executive Function (DOAEF; Chubarov et al., 2015); Diabetes Related Executive Functioning Scale (DREFS; Duke et al., 2014); Behavioral Inhibition System and Behavioral Activation System (BIS/BAS; Carver and White, 1994); EpiTrack Junior® (Kadish et al., 2013); Amsterdam Executive Function Inventory (AEFI; Van der Elst et al., 2012); and Ballet Executive Scale (BES; Wong et al., 2012). The reported reliability and validity of each of these six measures of EF is presented in Table 4. In summary, their internal consistency ranged between 0.60 and 0.97 (acceptable to excellent); only one study (Chubarov et al., 2015) reported inter-rater reliability and test-retest reliability (for DOAEF) as 0.97 and 0.91, respectively; only one study (Duke et al., 2014) reported inter-informant reliability (for DREFS) as 0.73. Validity aspects that were examined for some of these six measures included construct, convergent, discriminant, and concurrent validities. Presented results indicated that these measures were valid (see Table 4).
Psychometric characteristics of performance based measures, reported from the remaining 20 studies, are shown in Table 4. Unlike for some EF rating scales e.g., the BRIEF, DOAEF, DREFS, and AEFI, where both reliability and validity aspects were reported (Table 4), studies reporting on psychometric characteristics of performance based measures of EF did report either reliability or validity aspect, not both. The exception was in three studies (Chevignard et al., 2010; Malek et al., 2013; Pesce et al., 2016) where both reliability and validity aspects of the children's cooking task (CCT), SCWT, and the random number generation (RNG) task were reported (Table 4). Also, it was only these 3 studies out of the 20 that reported on validity. The discriminant and concurrent validity of the CCT were established in the study by Chevignard et al. (2010); discriminant validity of the SCWT was established in the study by Malek et al. (2013); whereas construct validity of the RNG task was established in the study by Pesce et al. (2016). The performance based measures that were among the 10 frequently used measures of EF (Table 2) had some psychometric characteristics presented (though not extensive and mostly reliability than validity) except for CPT, and D-KEFS Color Word Interference Test (see Table 4 for details). Poor reliabilities were also reported for some complex executive tasks like SCWT (test-retest as low as 0.37) in the study by Malek et al. (2013) and Tower test (internal consistency of 0.48) in the study by McAuliffe et al. (2008).
Discussion
We carried out a scoping review of measures of EF covering the period between 1st January 2002 and 31st July 2017. We wanted to know three things. First, which measures have been used to assess executive function of adolescents aged 13–17 years. Second, of the identified measures of EF, we were interested in knowing which measures stood out or dominated the field in terms of preference. Lastly, we wanted to establish evidence on the psychometric robustness of measures of EF currently used with the adolescent sub-population.
Preferred Measures of Adolescent EF
We observed that there is a range of individual EF measures (largely performance based) currently in use with young people aged 13–17 years, although 10 measures of these seem to dominate. Besides, there are a range of individual tasks of EF with a similar underlying principle of measurement (similar latent factor) that have been used to assess EF among adolescents. We grouped these into paradigms to get a better understanding of which group of tasks are frequently selected. We found out that tasks from 12 paradigms were often used and that tasks from the card sorting, CPT, Span, Stroop, Tower and Trails paradigms met our criteria of being frequently used, although in different variations, to assess adolescent EF.
From this review, the most measures of EF currently in use with the adolescent sub-population were performance based. Only 13 out of the 338 identified individual measures of EF were rating scales. This observation was also the same when it came to the 10 dominant measures of EF, where the Behavior Rating Inventory of Executive Function (BRIEF; Gioia et al., 2000; Guy et al., 2004) was the only rating scale. The preference for using performance based measures may reflect either the perceived higher reliability or validity of these measures, or the absence of a wide range of informant/self-report based measures. We prefer the latter explanation for various reasons explained from our findings or elsewhere.
Firstly, from our findings, existing evidence on psychometric robustness of such performance based measures remains scanty (see Table 4 and subheading on psychometric robustness of measures below). Relatedly, the extensive use of experimental paradigms for EF assessment has been criticized because they capture mainly performance at either the pathological or impairment level (Whyte et al., 1996). As a result, most end up having limited functional and ecological validity (Chan et al., 2008). Previous research has also found rating scales such as the BRIEF to be sensitive to changes in executive function even in the absence of changes in performance based measures (Cummings et al., 2002).
Secondly, our findings show that the range of EF ratings scales currently available for use with adolescents is limited. Out of 338 identified measures of EF, only 13 were rating scales. These findings suggest a need for further development or adaptation and validation of measures of EF that are informant/self-report based. Availability of many validated EF rating scales will provide researchers or clinicians a range of options to choose from, but most importantly, enhance the functional and ecological validity of their measurement (Chan et al., 2008). Researchers are already working toward this, if recently developed and validated rating scales such as the Amsterdam Executive Function Inventory (AEFI; Van der Elst et al., 2012), Diabetes Related Executive Function Scale (DREFS; Duke et al., 2014), or Dynamic Occupation Assessment of Executive Function (DOAEF; Chubarov et al., 2015) is anything to go by.
Another reason why researchers and practitioners prefer performance based measures of EF is because some assess multiple components of EF. Focusing on the identified dominant measures, most of the performance based measures tap into more than one aspect of EF. As examples, the Trail Making Test (TMT) assesses domains such as psychomotor speed, cognitive flexibility and working memory (Reitan, 1992) while Wisconsin Card Sorting Test (WCST) is believed to examine aspects such as perseveration, abstract reasoning, working memory and cognitive flexibility (Heaton, 1981). The brief administration time of some performance based measures of EF may also attract some researchers and clinicians in the field. For instance, both the digit span (Blackburn and Benton, 1957) and Stroop Color-Word Test (Golden and Freshwater, 1978) take 5 min or less to administer, whereas the TMT takes 5–12 min (Reitan, 1992).
Psychometric Robustness of Measures Currently Being Used to Assess Adolescent EF
Only 48 out of 705 studies included in this scoping review reported the reliability and/or validity of a measure of EF used with adolescents. Almost all of the 48 studies were conducted in high income countries, except for two (Wong et al., 2012; Malek et al., 2013) conducted in upper middle-income countries. This is not surprising because in this scoping review, we observed that most of the studies assessing adolescent EF have been conducted in North America and Europe (see Appendix 4 in Supplementary material). These are the same settings in which the majority, if not all, of the existing measures of EF have been developed and validated. Therefore, the reported psychometrics largely reflects performance of EF measures only within a restricted context. For adolescent EF to be accurately assessed across contexts, validation work should be extended to cover low-to-middle income countries where there is hardly any contextually appropriate measure of EF, yet it is in such settings where a great majority of the world's adolescents live in WHO (2014).
The reported psychometrics characteristics were mainly for EF rating scales (28 out of the 48 studies). These rating scales include the BRIEF (Gioia et al., 2000; Guy et al., 2004), DOAEF (Chubarov et al., 2015), DREFS (Duke et al., 2014), AEFI (Van der Elst et al., 2012), EpiTrack Junior® (Kadish et al., 2013), the Ballet Executive Scale (BES; Wong et al., 2012), and Behavioral Inhibition System and Behavioral Activation System (BIS/BAS; Carver and White, 1994). All, except BIS/BAS have been developed and validated within the last decade. The psychometric characteristics of these rating scales, even though from confined context, are good. A step forward in research will be to see more reports of how these tests perform when adapted and used in low-to-middle income countries.
Being one of the dominant measures used to assess adolescent EF, BRIEF predominated in terms of reported psychometrics. Twenty-two out of the 48 studies in this scoping review reported the psychometric characteristics of the BRIEF. Apart from two studies (Burton et al., 2016; Owens et al., 2016) where the reported sub-scale internal consistency of the BRIEF was below the recommended cut-off standard of 0.70 (Cicchetti, 1994), all the other reported reliabilities (Cronbach alphas, inter-rater and test-retest) ranged from good to excellent. BRIEF appears to be a valid measure of EF in terms of its construct, concurrent, and discriminant validity. Generally, these patterns of results re-affirm the good psychometric properties of the BRIEF as originally reported (Gioia et al., 2000; Guy et al., 2004). More work needs to be done from settings other than high income countries to confidently conclude that BRIEF can be validly used across contexts.
Evidence of psychometric robustness of performance based measures, even in the restricted context of upper middle income and high-income countries, remains scanty. Majority of the studies reporting on psychometrics of performance based measures focused on reliability aspect, though not extensively. Limited psychometrics are also presented for the dominant measures of EF we identified, mostly reliability. For some complex executive tasks like the Stroop Color-Word Test in the study by Malek et al. (2013) and Tower test in the study by McAuliffe et al. (2008), poor reliabilities are presented (see Table 4). Such findings mirror the observation that “complex executive tasks tend to suffer from relatively low internal and/or test–retest reliability” (Miyake et al., 2000). A major weakness is noticeable in terms of validity where only 3 studies (Chevignard et al., 2010; Malek et al., 2013; Pesce et al., 2016) reported on an aspect of validity of performance based measures. Among the dominant performance based measures of EF, only the Stroop Color-Word Test (Golden and Freshwater, 1978) has some evidence of validity for use with adolescents. Despite the lack of concrete evidence about psychometric robustness of performance based measures of EF other than backup by original test developers, these measures continue to be a preferred option overtime because of two potential reasons. First, we think that the relatively long history of development and use of performance based measures of EF, such as the Stroop task (Stroop, 1935), provides some degree of confidence in using them. Second, in reference to findings from this review, it could be that because some performance based measures of EF are used in almost similar contexts as to where they were originally developed and tested, researchers rarely focus on exploring their psychometric stability.
In summary, a notable conclusion from this review is the fact that there is just not enough validity and reliability data to support the use of measures of EF among adolescents across different national and cultural contexts. Similar observations are also noted from a recent review (Pereira et al., 2018). Of concern here, therefore, is the transference and use of these measures of EF in different cultural contexts without adequate adaptation and standardization. This can lead to significant limitations of interpreting findings (Greenfield, 1997), constrict the within-group variance or mask true differences between study groups (Grantham-McGregor, 1993). Assessment bias can also arise because of a lack of familiarity with test demands or content (Baddeley et al., 1995; Vijver, 1997). Findings from assessment of adolescent EF especially in low-to-middle income countries using unstandardized measures may not be a true reflection of their EF performance and can misguide policy or intervention efforts. Given the current scenario, it is essential that researchers adapt and/or develop context-sensitive measures of EF that possess adequate psychometric characteristics for use with adolescents.
Strengths and Limitations of the Review
We chose to conduct a scoping review as it is the most recommended where the nature of research (questions) is anticipated to be large/broader, complex, or highly heterogeneous hence not amenable to a more precise systematic review (Peters et al., 2015). We conducted this scoping review in a systematic and rigorous manner following the recommended scoping review framework (Arksey and O'Malley, 2005). However, the review has some limitations that need to be highlighted. Despite the search strategy being a thorough one, we only searched 3 major electronic databases of MEDLINE, PsycINFO, and Web of Science. Therefore, we cannot be certain that all important data were extracted and consequently reported. We also included and reviewed published articles entirely from journals. We did not search the gray literature. We included articles published in English only because this was the main language that authors are familiar with. We therefore acknowledge that the published literature may not be representative of entirety of work examining EF among adolescents. Because of inconsistency across included studies in reporting the EF domain assessed by tasks/measures of EF we identified, it was difficult to collate information on tasks/measures by EF domain assessed.
Conclusions
There is a range of measures currently in use to evaluate different aspects of executive function among the adolescent sub-population, although 10 measures appear to dominate the field. Unfortunately, there is very limited evidence generated to support the validity and usage of these measures of EF among adolescents across different national and cultural contexts.
Author Contributions
AA, CN, and GS conceptualized the study. MN and AA refined the search strategy and conducted the database search. MN, DS, AM, and EC screened and extracted data from all included articles. MN wrote the first draft of the manuscript. DS, AM, EC, GS, CN, and AA all critically reviewed the first draft and subsequent revisions of the manuscripts. All authors read and approved the final submitted version of the manuscript.
Funding
This work was supported by an MRC award to AA, award number MR/M025454/1. This award is jointly funded by the UK Medical Research Council (MRC) and the UK Department for International Development (DFID) under the MRC/DFID Concordat agreement and is also part of the EDCTP2 programme supported by the European Union. During this work, MKN was supported by the Wellcome Trust Master's Fellowship in Public Health and Tropical Medicine (Grant number 201310/Z/16/Z).
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We would like to thank the Director of Kenya Medical Research Institute (KEMRI) for the permission to publish this work.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2019.00311/full#supplementary-material
References
Anderson, P. (2002). Assessment and development of executive function (EF) during childhood. Child Neuropsychol. 8, 71–82. doi: 10.1076/chin.8.2.71.8724
Anderson, P. J., and Reidy, N. (2012). Assessing executive function in preschoolers. Neuropsychol. Review 22, 345–360. doi: 10.1007/s11065-012-9220-3
Arán Filippetti, V., and Richaud, M. C. (2015). Do executive functions predict written composition? Effects beyond age, verbal intelligence and reading comprehension. Polish Neuropsychol. Soc. 13, 331–349. doi: 10.5604/17307503.1187493
Arksey, H., and O'Malley, L. (2005). Scoping studies: towards a methodological framework. Int. J. Soc. Res. Methodol. 8, 19–32. doi: 10.1080/1364557032000119616
Baddeley, A., Gardner, J. M., and Grantham-McGregor, S. (1995). Cross-cultural cognition: developing tests for developing countries. Appl. Cogn. Psychol. 9, S173–S195. doi: 10.1002/acp.2350090711
Baumgartner, S. E., Weeda, W. D., van der Heijden, L. L., and Huizinga, M. (2014). The relationship between media multitasking and executive function in early adolescents. J. Early Adoles. 34, 1120–1144. doi: 10.1177/0272431614523133
Becker, S. P., and Langberg, J. M. (2014). Attention-deficit/hyperactivity disorder and sluggish cognitive tempo dimensions in relation to executive functioning in adolescents with ADHD. Child Psychiatry Hum. Dev. 45, 1–11. doi: 10.1007/s10578-013-0372-z
Benton, A. L. (1968). Differential behavioral effects in frontal lobe disease. Neuropsychologia 6, 53–60. doi: 10.1016/0028-3932(68)90038-9
Best, J. R., and Miller, P. H. (2010). A developmental perspective on executive function. Child Dev. 81, 1641–1660. doi: 10.1111/j.1467-8624.2010.01499.x
Bierman, K. L., Nix, R. L., Greenberg, M. T., Blair, C., and Domitrovich, C. E. (2008). Executive functions and school readiness intervention: Impact, moderation, and mediation in the head start REDI program. Dev. Psychopathol. 20, 821–843. doi: 10.1017/S0954579408000394
Blackburn, H. L., and Benton, A. L. (1957). Revised administration and scoring of the digit span test. J. Consul. Psychol. 21:139. doi: 10.1037/h0047235
Bombin, I., Mayoral, M., Castro-Fornieles, J., Gonzalez-Pinto, A., De la Serna, E., Rapado-Castro, M., et al. (2013). Neuropsychological evidence for abnormal neurodevelopment associated with early-onset psychoses. Psychol. Med. 43, 757–768. doi: 10.1017/S0033291712001535
Brown, T. M., Ris, M. D., Beebe, D., Ammerman, R. T., Oppenheimer, S. G., Yeates, K. O., et al. (2008). Factors of biological risk and reserve associated with executive behaviors in children and adolescents with spina bifida myelomeningocele. Child Neuropsychol. 14, 118–134. doi: 10.1080/09297040601147605
Burnett, A. C., Scratch, S. E., and Anderson, P. J. (2013). Executive function outcome in preterm adolescents. Early Hum. Dev. 89, 215–220. doi: 10.1016/j.earlhumdev.2013.01.013
Burton, D., Demuynck, S., and Yoder, J. R. (2016). Executive dysfunction predicts delinquency but not characteristics of sexual aggression among adolescent sexual offenders. Sexual Abuse 28, 707–721. doi: 10.1177/1079063214556357
Byerley, A. K., and Donders, J. (2013). Clinical utility of the behavior rating inventory of executive function–Self-report (BRIEF–SR) in adolescents with traumatic brain injury. Rehabili. Psychol. 58:412. doi: 10.1037/a0034228
Cambridge Cognition Ltd (2006). Cambridge Neuropsychological Test Automated Battery (CANTAB). Cambridge: Cambridge Cognition Ltd.
Carney, D. P., Brown, J. H., and Henry, L. A. (2013). Executive function in Williams and Down syndromes. Res. Dev. Disabil. 34, 46–55. doi: 10.1016/j.ridd.2012.07.013
Carver, C. S., and White, T. L. (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. J. Person. Social Psychol. 67:319. doi: 10.1037/0022-3514.67.2.319
Chan, R. C., Shum, D., Toulopoulou, T., and Chen, E. Y. (2008). Assessment of executive functions: review of instruments and identification of critical issues. Arch. Clin. Neuropsychol. 23, 201–216. doi: 10.1016/j.acn.2007.08.010
Chevignard, M. P., Catroppa, C., Galvin, J., and Anderson, V. (2010). Development and evaluation of an ecological task to assess executive functioning post childhood TBI: The Children's Cooking Task. Brain Impairment 11, 125–143. doi: 10.1375/brim.11.2.125
Chubarov, E., Sommerfeld, E., Hermesh, H., Shoval, G., Weiser, M., and Zalsman, G. (2015). Dynamic occupation assessment of executive function in adolescents with schizophrenia spectrum disorders: an initial report. Eur. Psychiatry 30, 894–899. doi: 10.1016/j.eurpsy.2015.09.001
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol. Assess. 6:284. doi: 10.1037/1040-3590.6.4.284
Collins, A., and Koechlin, E. (2012). Reasoning, learning, and creativity: frontal lobe function and human decision-making. PLoS Biol. 10:e1001293. doi: 10.1371/journal.pbio.1001293
Costanzo, F., Varuzza, C., Menghini, D., Addona, F., Gianesini, T., and Vicari, S. (2013). Executive functions in intellectual disabilities: a comparison between Williams syndrome and down syndrome. Res. Dev. Disabil. 34, 1770–1780. doi: 10.1016/j.ridd.2013.01.024
Cummings, D. D., Singer, H. S., Krieger, M., Miller, T. L., and Mahone, E. M. (2002). Neuropsychiatric effects of guanfacine in children with mild Tourette syndrome: a pilot study. Clin. Neuropharmacol. 25, 325–332. doi: 10.1097/00002826-200211000-00009
Delis, D. C., Kaplan, E., and Kramer, J. H. (2001). Delis-Kaplan Executive Function System, Technical Manual (D-KEFS). San Antonio: The Psychological Corporation.
Diamond, A. (2013). Executive functions. Ann. Rev. Psychol. 64:135. doi: 10.1146/annurev-psych-113011-143750
Dougherty, D. M., Mathias, C. W., Marsh, D. M., Greve, K. W., Bjork, J. M., and Moeller, F. G. (2003). Commission error rates on a continuous performance test are related to deficits measured by the Benton Visual Retention Test. Assessment 10, 3–12. doi: 10.1177/1073191102250526
Duke, D. C., Raymond, J. K., and Harris, M. A. (2014). The diabetes related executive functioning scale (DREFS): pilot results. Children's Health Care 43, 327–344. doi: 10.1080/02739615.2013.870040
Effeney, G., Carroll, A., and Bahr, N. (2013). Self-regulated learning and executive function: exploring the relationships in a sample of adolescent males. Edu. Psychol. 33, 773–796. doi: 10.1080/01443410.2013.785054
Emslie, H., Burden, V., Nimmo-Smith, I., Wilson, B. A., and Wilson, C. (2003). Behavioural Assessment of the Dysexecutive Syndrome for Children. London: Thames Valley Test Company.
Fino, E., Melogno, S., Iliceto, P., D'Aliesio, S., Pinto, M. A., Candilera, G., et al. (2014). Executive functions, impulsivity, and inhibitory control in adolescents: a structural equation model. Adv. Cogn. Psychol. 10:32. doi: 10.5709/acp-0154-5
Forssman, L., Eninger, L., Tillman, C. M., Rodriguez, A., and Bohlin, G. (2012). Cognitive functioning and family risk factors in relation to symptom behaviors of ADHD and ODD in adolescents. J. Attent. Disord. 16, 284–294. doi: 10.1177/1087054710385065
Fournet, N., Roulin, J. L., Monnier, C., Atzeni, T., Cosnefroy, O., Le Gall, D., et al. (2015). Multigroup confirmatory factor analysis and structural invariance with age of the behavior rating inventory of executive function (BRIEF)—French version. Child Neuropsychol. 21, 379–398. doi: 10.1080/09297049.2014.906569
Gioia, G. A., Isquith, P. K., Guy, S. C., and Kenworthy, L. (2000). Test review: behavior rating inventory of executive function. Child Neuropsychol. 6, 235–238. doi: 10.1076/chin.6.3.235.3152
Golden, C. J., and Freshwater, S. M., (1978). Stroop color and word test: A manual for clinical and experimental uses. Chicago, IL: Stoelting.
Grantham-McGregor, S. M. (1993). Assessments of the effects of nutrition on mental development and behavior in Jamaican studies. Am. J. Clin. Nutri. 57, 303S−309S. doi: 10.1093/ajcn/57.2.303S
Greenfield, P. M. (1997). You can't take it with you: why ability assessments don't cross cultures. Am. Psychol. 52:1115. doi: 10.1037/0003-066X.52.10.1115
Gutiérrez-Colina, A. M., Eaton, C. K., Lee, J. L., Reed-Knight, B., Loiselle, K., Mee, L. L., et al. (2015). Executive functioning, barriers to adherence, and nonadherence in adolescent and young adult transplant recipients. J. Pediatric Psychol. 41, 759–767. doi: 10.1093/jpepsy/jsv107
Guy, S. C., Isquith, P. K., and Gioia, G. A. (2004). Behavior Rating Inventory of Executive Function–Self-Report Version: Professional Manual.
Hall, P. A., and Marteau, T. M. (2014). Executive function in the context of chronic disease prevention: theory, research and practice. Prev. Med. 68, 44–50. doi: 10.1016/j.ypmed.2014.07.008
Heaton, R. K. (1981). A manual for the Wisconsin card sorting test. Odessa, FL: Western Psycological Services.
Henry, L. A., and Bettenay, C. (2010). The assessment of executive functioning in children. Child Adoles. Mental Health 15, 110–119. doi: 10.1111/j.1475-3588.2010.00557.x
Hinshaw, S. P., Carte, E. T., Fan, C., Jassy, J. S., and Owens, E. B. (2007). Neuropsychological functioning of girls with attention-deficit/hyperactivity disorder followed prospectively into adolescence: evidence for continuing deficits? Neuropsychology 21:263. doi: 10.1037/0894-4105.21.2.263
Hughes, D. M., Turkstra, L. S., and Wulfeck, B. B. (2009). Parent and self-ratings of executive function in adolescents with specific language impairment. Int. J. Language Commun. Disord. 44, 901–916. doi: 10.1080/13682820802425693
Huizinga, M., and Smidts, D. P. (2011). Age-related changes in executive function: a normative study with the dutch version of the behavior rating inventory of executive function (BRIEF). Child Neuropsychol. 17, 51–66. doi: 10.1080/09297049.2010.509715
Jensen, A. R., and Rohwer, W. D. (1966). The Stroop color-word test: a review. Acta Psychologica 25, 36–93. doi: 10.1016/0001-6918(66)90004-7
Kadish, N. E., Baumann, M., Pietz, J., Schubert-Bast, S., and Reuner, G. (2013). Validation of a screening tool for attention and executive functions (EpiTrack Junior®) in children and adolescents with absence epilepsy. Epilepsy Behav. 29, 96–102. doi: 10.1016/j.yebeh.2013.06.004
Karbach, J., and Unger, K. (2014). Executive control training from middle childhood to adolescence. Front. Psychol. 5:390. doi: 10.3389/fpsyg.2014.00390
Kirke-Smith, M., Henry, L. A., and Messer, D. (2016). The effect of maltreatment type on adolescent executive functioning and inner speech. Infant Child Dev. 25, 516–532. doi: 10.1002/icd.1951
Korkman, M., Kirk, U., and Kemp, S. (1998). NEPSY: A Developmental Neuropsychological Assessment Manual. San Antonio: Psychological Corporation.
Levac, D., Colquhoun, H., and O'Brien, K. K. (2010). Scoping studies: advancing the methodology. Implement Sci. 5:69. doi: 10.1186/1748-5908-5-69
Lezak, M. D., Howieson, D. B., Loring, D. W., and Fischer, J. S. (2004). Neuropsychological Assessment. New York, NY: Oxford University Press.
Lunt, L., Bramham, J., Morris, R. G., Bullock, P. R., Selway, R. P., Xenitidis, K., et al. (2012). Prefrontal cortex dysfunction and 'Jumping to Conclusions': bias or deficit? J. Neuropsychol. 6, 65–78. doi: 10.1111/j.1748-6653.2011.02005.x
MacDonald, J., and Duerson, D. (2015). Reliability of a computerized neurocognitive test in baseline concussion testing of high school athletes. Clin. J. Sport Med. 25, 367–372. doi: 10.1097/JSM.0000000000000139
Malek, A., Hekmati, I., Amiri, S., Pirzadeh, J., Gholizadeh, H. (2013). The standardization of Victoria Stroop color-word test among Iranian bilingual adolescents. Arch. Iranian Med. 16:380–384. doi: 10.13167/AIM.004
Martínez-Loredo, V., Fernández-Hermida, J. R., Carballo, J. L., and Fernández-Artamendi, S. (2017). Long-term reliability and stability of behavioral measures among adolescents: the delay discounting and stroop tasks. J. Adoles. 58, 33–39. doi: 10.1016/j.adolescence.2017.05.003
McAuley, T., Crosbie, J., Charach, A., and Schachar, R. (2017). Clinical, sociobiological, and cognitive predictors of ADHD persistence in children followed prospectively over time. J. Abnormal Child Psychol. 45, 765–776. doi: 10.1007/s10802-016-0189-x
McAuliffe, P., Brassard, M. R., and Fallon, B. (2008). Memory and executive functions in adolescents with posttreatment Lyme disease. Appl. Neuropsychol. 15, 208–219. doi: 10.1080/09084280802324473
Miller, M., and Hinshaw, S. P. (2010). Does childhood executive function predict adolescent functional outcomes in girls with ADHD? J. Abnormal Child Psychol. 38, 315–326. doi: 10.1007/s10802-009-9369-2
Minnes, S., Min, M. O., Short, E. J., Wu, M., Lang, A., Yoon, S., et al. (2016). Executive function in children with prenatal cocaine exposure (12–15 years). Neurotoxicol. Teratol. 57, 79–86. doi: 10.1016/j.ntt.2016.07.002
Miyake, A., and Friedman, N. P. (2012). The nature and organization of individual differences in executive functions: four general conclusions. Curr. Direct. Psychol. Sci. 21, 8–14. doi: 10.1177/0963721411429458
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., and Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: a latent variable analysis. Cogn. Psychol. 41, 49–100. doi: 10.1006/cogp.1999.0734
Modi, A. C., Schmidt, M., Smith, A. W., Turnier, L., Glaser, N., and Wade, S. L. (2017). Development of a web-based executive functioning intervention for adolescents with epilepsy: the epilepsy journey. Epilepsy Behav. 72, 114–121. doi: 10.1016/j.yebeh.2017.04.009
Na, K. S., Lee, S. I., Hong, S. D., Kim, J. H., Shim, S. H., Choi, J., et al. (2013). Effect of osmotic-release oral system methylphenidate on learning skills in adolescents with attention-deficit/hyperactivity disorder: an open-label study. Int. Clin. Psychopharmacol. 28, 184–192. doi: 10.1097/YIC.0b013e3283612509
Osterrieth, P. A. (1944). Le test de copie d'une Figure Complexe; Contribution à L'étude de la Perception et de la Mémoire. Arch. Psychol. 30, 206–235.
Owens, J. A., Dearth-Wesley, T., Lewin, D., Gioia, G., and Whitaker, R. C. (2016). Self-regulation and sleep duration, sleepiness, and chronotype in adolescents. Pediatrics 138:e20161406. doi: 10.1542/peds.2016-1406
Park, J. M., Samuels, J. F., Grados, M. A., Riddle, M. A., Bienvenu, O. J., Goes, F. S., et al. (2016). ADHD and executive functioning deficits in OCD youths who hoard. J. Psychiatric Res. 82, 141–148. doi: 10.1016/j.jpsychires.2016.07.024
Pereira, A., Lopes, S., Magalhães, P., Sampaio, A., Chaleta, E., and Rosário, P. (2018). How executive functions are evaluated in children and adolescents with cerebral palsy? a systematic review. Front. Psychol. 9:21. doi: 10.3389/fpsyg.2018.00021
Perez, K. M., Patel, N. J., Lord, J. H., Savin, K. L., Monzon, A. D., Whittemore, R., et al. (2016). Executive function in adolescents with type 1 diabetes: relationship to adherence, glycemic control, and psychosocial outcomes. J. Pediatric Psychol. 42, 636–646. doi: 10.1093/jpepsy/jsw093
Perkins, S. C., Cortina, K. S., Smith-Darden, J. P., and Graham-Bermann, S. A. (2012). The mediating role of self-regulation between intrafamilial violence and mental health adjustment in incarcerated male adolescents. J. Interpersonal Violence 27, 1199–1224. doi: 10.1177/0886260511425240
Pesce, C., Marchetti, R., Forte, R., Crova, C., Scatigna, M., Goudas, M., et al. (2016). Youth life skills training: exploring outcomes and mediating mechanisms of a group-randomized trial in physical education. Sport Exercise and Perform. Psychol. 5:232. doi: 10.1037/spy0000060
Peters, M. D., Godfrey, C. M., Khalil, H., McInerney, P., Parker, D., and Soares, C. B. (2015). Guidance for conducting systematic scoping reviews. Int. J. Evid. Based Healthcare 13, 141–146. doi: 10.1097/XEB.0000000000000050
Peterson, E., and Welsh, M. (2014). “The development of hot and cool executive functions in childhood and adolescence: are we getting warmer?” in Handbook of Executive Functioning, eds S. Goldstein and J. A. Naglieri (New York, NY: Springer), 45–65.
Pickens, S., Ostwald, S. K., Murphy-Pace, K., and Bergstrom, N. (2010). Systematic review of current executive function measures in adults with and without cognitive impairments. Int. J. Evid. Based Healthcare 8, 110–125. doi: 10.1111/j.1744-1609.2010.00170.x
Poon, K. (2017). Hot and cool executive functions in adolescence: development and contributions to important developmental outcomes. Front. Psychol. 8:2311. doi: 10.3389/fpsyg.2017.02311
Pope, C. N., Ross, L. A., and Stavrinos, D. (2016). Association between executive function and problematic adolescent driving. J. Dev. Behav. Pediatrics 37:702. doi: 10.1097/DBP.0000000000000353
Prencipe, A., Kesek, A., Cohen, J., Lamm, C., Lewis, M. D., and Zelazo, P. D. (2011). Development of hot and cool executive function during the transition to adolescence. J. Experim. Child Psychol. 108, 621–637. doi: 10.1016/j.jecp.2010.09.008
Ransom, D. M., Burns, A. R., Youngstrom, E. A., Vaughan, C. G., Sady, M. D., and Gioia, G. A. (2016). Applying an evidence-based assessment model to identify students at risk for perceived academic problems following concussion. J. Int. Neuropsychol. Soc. 22, 1038–1049. doi: 10.1017/S1355617716000916
Reitan, R. M. (1992). Trail Making Test: Manual for Administration and Scoring. Tucson, AZ: Reitan Neuropsychology Laboratory.
Rey, A. (1941). L'examen psychologique dans les cas d'encéphalopathie traumatique.(Les problems.). Arch. Psychol. 28, 215–285.
Riggs, N. R., Jahromi, L. B., Razza, R. P., Dillworth-Bart, J. E., and Mueller, U. (2006). Executive function and the promotion of social–emotional competence. J. Appl. Dev. Psychol. 27, 300–309. doi: 10.1016/j.appdev.2006.04.002
Rose, B. M., and Holmbeck, G. N. (2007). Attention and executive functions in adolescents with spina bifida. J. Pediatric Psychol. 32, 983–994. doi: 10.1093/jpepsy/jsm042
Rosvold, H. E., Mirsky, A. F., Sarason, I., Bransome Jr, E. D., and Beck, L. H. (1956). A continuous performance test of brain damage. J. Consult. Psychol. 20:343. doi: 10.1037/h0043220
Rubia, K. (2011). “Cool” inferior frontostriatal dysfunction in attention-deficit/hyperactivity disorder versus “hot” ventromedial orbitofrontal-limbic dysfunction in conduct disorder: a review. Biol. Psychiatry 69, e69–87. doi: 10.1016/j.biopsych.2010.09.023
Siddiqui, S. V., Chatterjee, U., Kumar, D., Siddiqui, A., and Goyal, N. (2008). Neuropsychology of prefrontal cortex. Indian J. Psychiatry 50, 202–208. doi: 10.4103/0019-5545.43634
Smith, L. B., Kugler, B. B., Lewin, A. B., Duke, D. C., Storch, E. A., and Geffken, G. R. (2014). Executive functioning, parenting stress, and family factors as predictors of diabetes management in pediatric patients with type 1 diabetes using intensive regimens. Children's Health Care 43, 234–252. doi: 10.1080/02739615.2013.839383
Staiano, A. E., Abraham, A. A., and Calvert, S. L. (2012). Competitive versus cooperative exergame play for African American adolescents' executive function skills: Short-term effects in a long-term training intervention. Dev. Psychol. 48:337. doi: 10.1037/a0026938
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. J. Experim. Psychol. 18:643. doi: 10.1037/h0054651
Suchy, Y., Turner, S. L., Queen, T. L., Durracio, K., Wiebe, D. J., Butner, J., et al. (2016). The relation of questionnaire and performance-based measures of executive functioning with Type 1 diabetes outcomes among late adolescents. Health Psychol. 35:661. doi: 10.1037/hea0000326
Taylor, S. J., Barker, L. A., Heavey, L., and McHale, S. (2015). The longitudinal development of social and executive functions in late adolescence and early adulthood. Front. Behav. Neurosci. 9:252. doi: 10.3389/fnbeh.2015.00252
Thompson, B. (2004). Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications. Washington, DC: American Psychological Association. doi: 10.1037/10694-000
Thush, C., Wiers, R. W., Ames, S. L., Grenard, J. L., Sussman, S., and Stacy, A. W. (2008). Interactions between implicit and explicit cognition and working memory capacity in the prediction of alcohol use in at-risk adolescents. Drug Alcohol Depend. 94, 116–124. doi: 10.1016/j.drugalcdep.2007.10.019
Tsermentseli, S., and Poland, S. (2016). Cool versus hot executive function: a new approach to executive function. Encephalos 53, 11–14.
Van de Weijer-Bergsma, E., Formsma, A. R., de Bruin, E. I., and Bögels, S. M. (2012). The effectiveness of mindfulness training on behavioral problems and attentional functioning in adolescents with ADHD. J. Child Family Stud. 21, 775–787. doi: 10.1007/s10826-011-9531-7
Van der Elst, W., Ouwehand, C., van der Werf, G., Kuyper, H., Lee, N., and Jolles, J. (2012). The Amsterdam Executive Function Inventory (AEFI): psychometric properties and demographically corrected normative data for adolescents aged between 15 and 18 years. J. Clin. Experim. Neuropsychol. 34, 160–171. doi: 10.1080/13803395.2011.625353
Vijver, F. V. D. (1997). Meta-analysis of cross-cultural comparisons of cognitive test performance. J. Cross-Cult. Psychol. 28, 678–709. doi: 10.1177/0022022197286003
Weiner, N. W., Toglia, J., and Berg, C. (2012). Weekly Calendar Planning Activity (WCPA): a performance-based assessment of executive function piloted with at-risk adolescents. Am. J. Occupat. Ther. 66, 699–708. doi: 10.5014/ajot.2012.004754
WHO (2014). Health for the World's Adolescents: A Second Chance in the Second Decade: Summary. Geneva: World Health Organization.
Whyte, J., Polansky, M., Cavallucci, C., Fleming, M., Lhulier, J., and Coslett, H. B. (1996). Inattentive behavior after traumatic brain injury. J. Int. Neuropsychol. Soc. 2, 274–281. doi: 10.1017/S1355617700001284
Wong, A., Rodríguez, M., Quevedo, L., Fernandez de Cossio, L., Borges, A., Reyes, A., et al. (2012). Questionnaire of executive function for dancers: an ecological approach. Assessment 19, 383–387. doi: 10.1177/1073191110397483
World Bank (2017). World Bank Country and Lending Groups [Online]. World Bank. Available online at: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups (Accessed June 22, 2018)
Xanthopoulos, M. S., Gallagher, P. R., Berkowitz, R. I., Radcliffe, J., Bradford, R., and Marcus, C. L. (2015). Neurobehavioral functioning in adolescents with and without obesity and obstructive sleep apnea. Sleep 38, 401–410. doi: 10.5665/sleep.4498
Yoran-Hegesh, R., Kertzman, S., Vishne, T., Weizman, A., and Kotler, M. (2009). Neuropsychological mechanisms of digit symbol substitution test impairment in Asperger disorder. Psychiatry Res. 166, 35–45. doi: 10.1016/j.psychres.2007.11.015
Žebec, M. S., Demetriou, A., and Kotrla-Topić, M. (2015). Changing expressions of general intelligence in development: a 2-wave longitudinal study from 7 to 18 years of age. Intelligence 49, 94–109. doi: 10.1016/j.intell.2015.01.004
Zelazo, P. D., and Carlson, S. M. (2012). Hot and cool executive function in childhood and adolescence: development and plasticity. Child Dev. Perspect. 6, 354–360. doi: 10.1111/j.1750-8606.2012.00246.x
Keywords: executive function, assessment, measures, adolescents, scoping review, psychometric properties
Citation: Nyongesa MK, Ssewanyana D, Mutua AM, Chongwo E, Scerif G, Newton CRJC and Abubakar A (2019) Assessing Executive Function in Adolescence: A Scoping Review of Existing Measures and Their Psychometric Robustness. Front. Psychol. 10:311. doi: 10.3389/fpsyg.2019.00311
Received: 05 September 2018; Accepted: 31 January 2019;
Published: 01 March 2019.
Edited by:
Natasha Kirkham, Birkbeck University of London, United KingdomReviewed by:
Rachel M. Flynn, Northwestern University, United StatesJeffrey Coldren, Youngstown State University, United States
Copyright © 2019 Nyongesa, Ssewanyana, Mutua, Chongwo, Scerif, Newton and Abubakar. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Moses K. Nyongesa, TWthY2hhbWFAa2Vtcmktd2VsbGNvbWUub3Jn