Sorting Test, Tower Test, and BRIEF-SR do not predict school performance of healthy adolescents in preuniversity education

Executive functions (EF) such as self-monitoring, planning, and organizing are known to develop through childhood and adolescence. They are of potential importance for learning and school performance. Earlier research into the relation between EF and school performance did not provide clear results possibly because confounding factors such as educational track, boy-girl differences, and parental education were not taken into account. The present study therefore investigated the relation between executive function tests and school performance in a highly controlled sample of 173 healthy adolescents aged 12–18. Only students in the pre-university educational track were used and the performance of boys was compared to that of girls. Results showed that there was no relation between the report marks obtained and the performance on executive function tests, notably the Sorting Test and the Tower Test of the Delis-Kaplan Executive Functions System (D-KEFS). Likewise, no relation was found between the report marks and the scores on the Behavior Rating Inventory of Executive Function—Self-Report Version (BRIEF-SR) after these were controlled for grade, sex, and level of parental education. The findings indicate that executive functioning as measured with widely used instruments such as the BRIEF-SR does not predict school performance of adolescents in preuniversity education any better than a student's grade, sex, and level of parental education.

Introduction

At school, adolescents often get complex assignments and have to do homework for various courses simultaneously. In addition, they have to decide which combinations of courses to follow, which in turn may affect their possibilities for higher education and future careers. Therefore, the adolescent student needs to develop higher cognitive skills, such as self-monitoring, planning and organizing, in order to perform well. It is unclear, however, whether the development of these functions also predicts adolescents' school performance. Insight into the cognitive predictors of school performance is relevant for school (neuro) psychologists and other professionals who work with adolescents. They often have to estimate how scores on intelligence tests and neuropsychological tests are related to task performance in adolescents' daily life, for example to performance at school.

The neuropsychological measures often used for estimating performance in daily life are executive function tests (Gioia and Isquith, 2004; Chan et al., 2008). Executive functions are the functions necessary for goal-directed behavior (e.g., Best and Miller, 2010). A wide range of executive functions have been described in literature, such as inhibition, updating working memory, shifting, planning, organization skills, attentional control, and self-control (Alvarez and Emory, 2006; Best and Miller, 2010; Hofmann et al., 2012). However, concerns have been raised about the ecological validity of executive function tests; that is, how well they predict performance in daily life (Gioia and Isquith, 2004; Chan et al., 2008; Olson et al., 2013). Previous studies that related executive function tests to school performance in adolescents found mixed results (e.g., Gioia and Isquith, 2004; St Clair-Thompson and Gathercole, 2006; Chan et al., 2008; Latzman et al., 2010; Best et al., 2011), which we will address in depth below. These mixed results may have been caused by a lack of control for important confounders (Willoughby et al., 2012). Therefore, the present study set out to investigate whether performance on executive function tests is related to school performance in a highly controlled sample of adolescents.

Neuroscientific research associates executive functions with the functioning of neural networks between several brain areas including, but not restricted to, the prefrontal brain areas (Alvarez and Emory, 2006). These brain areas develop during childhood through adolescence until early adulthood (Gogtay et al., 2004; Giedd, 2008). Neuropsychological studies have confirmed that executive functions develop during this time period, with some functions becoming fully developed earlier than others (Anderson, 2002; Best and Miller, 2010). Considering the nature of executive functions, and the fact that they are still developing during adolescence, it is likely that adolescents' school performance is related to the degree of maturation of relevant executive functions.

As we mentioned earlier, studies investigating the relation between executive functions and school performance have reported mixed results. These studies can be classified by the outcome measures they used to assess school performance. School performance can be measured with various outcome measures, such as report marks or performance on standardized tests. Of these two measures of school performance, report marks have the highest ecological validity, since they are relevant for students' daily lives. Decisions on passing or failing a course or grade are made based on report marks.

Only a few studies have investigated the relation between executive functions and report marks. Most of these studies were conducted with young adolescents, aged 12–13 (Veenman et al., 2005; Checa et al., 2008; Checa and Rueda, 2011). Results showed that executive functions such as executive attention (Checa et al., 2008), and metacognitive (executive) skills (Veenman et al., 2005), partially predicted report marks for mathematics. Attention and effortful control were found to be related not only to performance in mathematics, but also to the average report mark of all subjects at the end of the academic year (Checa and Rueda, 2011). In primary school children in third grade, executive function tests such as the Trail Making Test and the Tower of Hanoi did not relate to report marks. However, performance on a classroom-based planning task and teacher reports on children's time management skills were related to report marks (Cohen et al., 1995).

Most studies that have investigated the relation between executive functions and school performance in adolescents did not consider report marks, but looked at the outcomes of standardized performance tests. Standardized tests are equal for all students, and scores are not dependent on a student's school, class or teacher, as is the case with report marks. Studies on the relation between performance on standardized tests and executive functions showed that girls' performance on mathematics in adolescence and early adulthood was predicted by executive functioning measured in childhood, and especially by the score obtained on the Rey Osterrieth Complex Figure (Miller and Hinshaw, 2010; Miller et al., 2012). Furthermore, in a cross-sectional study, three complex executive function measures from the Cognitive Assessment System were related to school performance on reading and mathematics in children and adolescents aged 5–17 (Best et al., 2011). Other studies found that not all executive function tests contributed equally to various academic skills. Results from one study on adolescent boys aged 11–16 (Latzman et al., 2010) showed that: the Delis-Kaplan Executive Functions System (D-KEFS) composite score for conceptual flexibility was related to performance in reading and science; the monitoring composite was related to reading and social studies; and inhibition was related to mathematics and science. Another study on adolescents aged 11–12 (St Clair-Thompson and Gathercole, 2006) reported that: updating was related to performance in English and mathematics; inhibition was related to English, mathematics, and science; and that shifting was not related to school performance.

As these studies show, performance on some executive function tests appears to be related to school performance. However, the study results vary when it comes to determining which specific executive functions are related to different school subjects, and they are inconclusive about the exact extent of the relationships. Studies that report high correlations between executive functions and school performance (between 0.30 and 0.50 or higher), often used a sample that had diverse socioeconomic backgrounds, or they did not control for sex or intelligence (e.g., St Clair-Thompson and Gathercole, 2006; Best et al., 2011). Studies that did control for these factors generally found lower correlations (around 0.10–0.25) (Latzman et al., 2010; Miller and Hinshaw, 2010). Clearly, in order to investigate the association between executive functions and school performance, it is crucial to carefully control for confounders. In addition, research in younger children shows that the relationship between executive functions and school performance is confounded by unmeasured variables that are constant over time, such as household or care-giver characteristics (Willoughby et al., 2012).

As it is impossible to measure all potential confounders that may affect executive functions, the current study used a homogeneous sample. In that way, we were able to control for many known and unknown variables. Our sample consisted of students in the preuniversity educational track, which is the most advanced track of the Dutch secondary school system; the top 20% of all students in Dutch secondary education are in this track (Ministry of Education, Culture and Science, 2009). Therefore, all our participants were high-performing students. Moreover, we selected students who had never repeated or skipped a grade in school. Studies show that students who have repeated or skipped a grade have different profiles with regard to a range of school related variables compared to students with a regular educational career (Jimerson, 2001; Steenbergen-Hu and Moon, 2011). In addition, given that the former are a year older or younger than their classmates, they are most probably at a different stage of biological and psychological development. Finally, we reduced the effects of medical factors that may influence the relation between executive functions and school performance, such as past brain trauma, a developmental disorder, or medication use, by including only healthy, normally developing adolescents. Because of these selection criteria, our sample was homogeneous with regard to both ability level and developmental history.

The current study investigated one possible confounder in particular, namely sex. Sex is well known for its influence on school performance, as girls and boys excel at different subjects (Machin, 2005; Van Langen et al., 2006; Clark et al., 2008; Driessen and Van Langen, 2010). There is also growing evidence to support the conclusion that the neuropsychological performance of boys and girls differs in the school setting (Martens et al., 2011; Dekker et al., 2013a). In addition, a recent study has reported differences in executive functioning related to school performance in adolescents of different sex (Coenen et al., 2011). Other studies have shown that girls perform better in the school setting because they are better at self-control and self-discipline (Downey et al., 2005; Duckworth and Seligman, 2006; Hyde et al., 2007; Steinmayr and Spinath, 2008), which is interesting as there are indications that executive functions subserve self-control and self-discipline (Hofmann et al., 2012). Finally, evidence is accumulating that boys and girls differ in brain maturation, especially during an extended period in adolescence, with boys lagging behind (Giedd, 2008; Lenroot and Giedd, 2010). This suggests that biological factors pertaining to brain development may underly the complex relation between executive functions and school performance. Other factors may influence the relation between sex and school performance as well. For example, boys are more likely to show work-avoidant motivational strategies than girls in secondary education (Dekker et al., 2013b). It may thus be that a relation between executive function and school performance is visible in girls, but not in boys. This hypothesis is addressed in the present study.

In sum, the main aim of the current study was to investigate the respective relations between three different measures of executive functions and school performance, while keeping close control of confounders. In addition, we investigated whether these relations were moderated by sex. We investigated a homogeneous sample of 173 healthy adolescents, all secondary school students in the pre-university educational track. Two objective, performance-based neuropsychological tests were used to measure categorizing and shifting (Sorting Test from the D-KEFS) and planning skills (Tower Test from the D-KEFS). These tests are suitable for administration to adolescents, and measure executive functions that are still developing at this age range (Delis et al., 2001; Huizinga et al., 2006; Luciana et al., 2009). Furthermore, we also administered the Behavior Rating Inventory of Executive Function—Self-Report Version (BRIEF-SR) (Guy et al., 2004). This questionnaire has been developed to measure a wide range of executive functions based on their appearance in real-world behavior. Therefore, the BRIEF-SR has been claimed to be a more ecologically valid measure of executive functions than objective neuropsychological tests (Gioia and Isquith, 2004; Guy et al., 2004; Olson et al., 2013). We used report marks to measure school performance, since these are most relevant to adolescents themselves. School performance was measured with the end-of-term report marks for Dutch (the native language), English as a foreign language, and mathematics. Based upon the assumption that the BRIEF-SR is more ecologically valid than objective executive function tests, we hypothesized that the BRIEF-SR would predict report marks better than the objective tests. In addition, we hypothesized that the relation between executive functions and school performance would be different for boys and girls.

Methods

Participants

Participants came from seven secondary schools in the south of the Netherlands. They were in grade 7, 9, or 11 of the pre-university educational track. This is the most advanced track in Dutch secondary education; the top 20% of all students in Dutch secondary education are in this track (Ministry of Education, Culture and Science, 2009). Participants had not repeated or skipped a grade. Furthermore, participants had the Dutch nationality, had no learning disorders, psychiatric disorders or developmental disorders, did not use medication that influences cognitive functions and did not have a history of brain damage with a loss of consciousness of more than 30 min. These criteria were measured with a questionnaire that was completed by the parents.

The participants themselves and the parents of under-aged participants had to give permission for participation. Participants received a monetary reward for participation. The Ethical Committee of the Faculty of Psychology of Maastricht University approved the research protocol.

Measures

Executive functions

Objective measures of executive functions were acquired with the Sorting Test and the Tower Test from the Delis-Kaplan Executive Functions System (D-KEFS) (Delis et al., 2001). The Sorting Test is a card sorting test that aims to measure categorization skills and set shifting. No Dutch version existed; therefore we translated the words on the cards, and changed some words to make all original sorts possible. The free sorting condition was used. Outcome measure of the Sorting Test was the number of confirmed correct sorts (range: 0–16). The Tower Test measures planning, and has a strong learning component due to the nature of the items. The raw total achievement score was used as outcome measure (range: 0–30).

As a subjective measure of executive functions, the Behavior Rating Inventory of Executive Function—Self Report Version (BRIEF-SR) (Guy et al., 2004) was used. The BRIEF-SR is an 80-item questionnaire, especially developed for adolescents, in which they have to indicate how often the described behaviors had been a problem in the past six months (never, sometimes or often). The items can be grouped into 8 scales that measure the following executive functions: Inhibit, Shift, Emotional Control, Monitor (together: the Behavioral Regulation Index, BRI), and Working Memory, Plan/Organize, Organization of Materials, and Task Completion (together: the Metacognition Index, MCI). A higher score on the MCI and the BRI indicates more problems with executive functioning. Following the official Dutch translation of the BRIEF Parent Version (by Smidts and Sergeant), the BRIEF-SR was translated into Dutch. Few items of the BRIEF-SR are different from those in the Parent Version. These were translated by a native English-Dutch bilingual psychologist, and reviewed by another psychologist. The internal consistency of this Dutch version of the BRIEF-SR was r = 0.89 for the BRI and r = 0.91 for the MCI.

Report marks

End of term report marks (ranging from 1.0 = very bad to 10.0 = outstanding) for Dutch, English, and mathematics were acquired through the schools' administration. Dutch, English, and mathematics are the first three main goals of secondary education in the Netherlands (Ministry of Education, Culture and Science, 2006) and are valid estimators of school performance (Reed et al., 2010). These report marks are the result of multiple smaller and larger tests and assessments (at least more than 4) that were administered during one school year. The tests and assessments were part of the teaching method or were developed by teachers themselves, and could consist of various assessment methods, e.g., paper-and-pencil tests, essays, presentations. Because the schools in the sample used different grading policies, each school's report marks were transformed into z-scores, based on the school's mean report mark and its standard deviation. In this way, the distribution of scores was similar for each school.

Demographics

Participants reported age and sex. Parents reported both parents' education level. Level of parental education (LPE) was defined as the highest education of the two. LPE was medium when the parents had junior vocational or junior general secondary education and high when they had senior vocational or academic education.

Procedure

Adolescents were recruited through letters that were distributed at the seven schools by the researchers. All students were in grade 7, 9, or 11 at the start of the study. Because the study started at the end of a school year, 50.9% of the adolescents were tested in the new school year, and were therefore in grade 8, 10, or 12 when they participated. A trained psychologist administered tests and questionnaires in a quiet room at school. Administration took approximately 1.5 h. Adolescents participated during school time and therefore missed certain lessons.

Analyses

All analyses were performed with SPSS Statistics 19.0 for Mac. First, to examine relations between all variables of interest, zero-order correlations were calculated. To investigate whether executive functions predicted report marks after correction for grade at the start of the study, sex, and LPE, separate multivariate GLM analyses (MANCOVA) were performed for each executive function score. Dependent variables were standardized report marks for Dutch, English, and mathematics. The following fixed factors and covariates were included: grade, sex, LPE (hereafter called demographic variables) and executive function score. After investigating main effects, we investigated interaction effects. To examine whether results were different for the different grades, we added the interaction term grade * executive function score. To investigate influence of sex, analyses were performed with inclusion of the interaction between sex and executive function score. Finally, we investigated a model with all executive function scores and all demographic variables to investigate their contribution together, and the same model without demographic variables to investigate the amount of variance predicted by executive function scores alone.

Results

A total of 173 adolescents between 12.68 years and 18.05 years participated (age M = 15.22 years; SD = 1.66). Of those, 63.6% had highly educated parents, and the remainder had parents with a medium education level. Table 1 shows outcomes on executive function measures and school performance, per grade and sex. Sex differences were seen on the Tower Test to the advantage of boys. On the other executive function measures, no sex differences were found. On all school report marks, there were differences between grades and between sexes: students from lower grades had higher report marks than students from higher grades, and girls achieved higher report marks than boys.

TABLE 1

Table 1. Descriptive statistics of executive function measures and school performance, per grade and sex.

Relation Between Executive Functions and Report Marks

Table 2 shows correlations between executive function measures and report marks. The BRI and MCI of the BRIEF-SR were the only executive function measures that significantly correlated with report marks. The BRI correlated with Dutch scores only (r = −0.17), while the MCI correlated with report marks in Dutch, English, and mathematics (between r = −0.20 to r = −0.27, p < 0.05). These correlations indicate that the more problems with behavior regulation a student reported, the lower the score for Dutch. In addition, the more problems with metacognition a student reported, the lower the score for Dutch, English, and mathematics.

TABLE 2

Table 2. Zero-order correlations between background variables, executive function measures, and report marks.

Sorting Test

MANCOVA analyses showed no significant main effect of Sorting Test score on report marks, F_{(3, 165)} = 0.27, p = 0.847, partial eta squared = 0.01. Repeating the analyses with interaction effects also showed no significant interaction effect between Sorting Test score and grade, and Sorting Test score and sex on report marks, resp. F_{(6, 326)} = 1.07, p = 0.382, partial eta squared = 0.02, and F_{(3, 162)} = 0.58, p = 0.631, partial eta squared = 0.01.

Tower Test

MANCOVA analyses showed no significant main effect of Tower Test score on report marks, F_{(3, 165)} = 1.98, p = 0.119, partial eta squared = 0.04. Repeating the analyses with the interaction effects also showed no significant interaction between Tower Test score and grade, and Tower Test score and sex on report marks, resp. F_{(6, 326)} = 1.49, p = 0.181, partial eta squared = 0.03 and F_{(3, 162)} = 0.64, p = 0.588, partial eta squared = 0.01.

BRIEF-SR BRI

MANCOVA analyses showed no significant main effect of BRI score on report marks, F_{(3, 165)} = 1.99, p = 0.117, partial eta squared = 0.04. Repeating the analyses with the interaction effects also showed no significant interaction between the score on the BRI and grade, and the BRI and sex on report marks, resp. F_{(6, 326)} = 0.60, p = 0.729, partial eta squared = 0.01, and F_{(3, 162)} = 1.99, p = 0.118, partial eta squared = 0.04.

BRIEF-SR MCI

MANCOVA analyses showed no significant main effect of MCI score on report marks, F_{(3, 165)} = 2.12, p = 0.100, partial eta squared = 0.04. Repeating the analyses with the interaction effects also showed no significant interaction between the score on the MCI and grade, and the MCI and sex on report marks, resp. F_{(6, 326)} = 0.57, p = 0.751, partial eta squared = 0.01, and F_{(3, 162)} = 1.03, p = 0.382, partial eta squared = 0.02.

Model with All Executive Function Scores

MANCOVA analyses with all executive function scores in one model showed no significant effects of any of the executive function scores [Sorting Test: F_{(3, 162)} = 0.17, p = 0.918, partial eta squared = 0.00; Tower Test: F_{(3, 162)} = 2.36, p = 0.074, partial eta squared = 0.042; BRIEF-SR BRI: F_{(3, 162)} = 0.66, p = 0.577, partial eta squared = 0.01; BRIEF-SR MCI: F_{(3, 162)} = 1.34, p = 0.262, partial eta squared = 0.02]. Investigating the three demographic variables showed that grade and sex were significant [grade: F_{(6, 326)} = 8.35, p < 0.001, partial eta squared = 0.13; sex: F_{(3, 162)} = 16.12, p < 0.001, partial eta squared = 0.23], and LPE approached significance [F_{(3, 162)} = 2.54, p = 0.058, partial eta squared = 0.05]. In all analyses performed in this article, these three demographic variables were included and their effects were as described in this analysis. A model without demographic variables, but with all executive function scores, showed a significant effect for the BRIEF-SR MCI [F_{(3, 166)} = 4.21, p = 0.007, partial eta squared = 0.07]. This effect is smaller than the variance explained by demographic variables in the previous analysis.

Discussion

The current study investigated whether executive functions predicted report marks in healthy adolescents aged 12–18 who were secondary school students in the pre-university educational track. Results showed that performance on the Sorting Test and the Tower Test did not predict report marks for Dutch, English, and mathematics. There was a zero-order correlation between scores on the BRIEF-SR and report marks (r = 0.17–0.27). Such correlations are often reported in studies on the relation of executive function tests to school performance (e.g., St Clair-Thompson and Gathercole, 2006; Best et al., 2011). However, after correcting for grade, sex, and LPE, the BRIEF-SR did not predict report marks anymore. Moreover, sex did not influence the relation between executive functions and report marks, since the sex* executive function score interaction term was not significant in any of our models.

The magnitude of the correlation between the BRIEF-SR and report marks in the current study was comparable to that found in studies that controlled for intelligence (Latzman et al., 2010; Miller and Hinshaw, 2010). In the current study, all participants were in the pre-university educational track, the level at which the top 20% of Dutch students is studying (Ministry of Education, Culture and Science, 2009). By selecting this high-performing sample, we used a group that is relatively homogeneous with respect to potential and intellectual ability [estimated intelligence quotient (IQ) higher than 90; (Van den Bos et al., 2012)]. Students in the pre-university educational track are overrepresented in higher socioeconomic status (SES) groups and there is some evidence that these students are generally more mature with regard to neuropsychological development (Hackman and Farah, 2009). The advantage of our study is that it enabled some control over possible confounders related to SES, and over slow (neuro) psychological development due to lack of environmental support. The findings can be considered as strong for the upper segment of the educational system (pre-university education). On the other hand, our design has the drawback that the results cannot be generalized to all adolescents and that we were not able to take IQ-scores into account. Forthcoming studies will address executive functions in relation to boy-girl difference in students in the “vocational” educational track; given the fact that this group is characterized by broader variance in SES, intelligence, learning motivation, and study results, findings are anticipated to be different than reported in the present paper. This would be relevant in terms of applicability of the findings in educational practice, notably in councelling of students and their parents and teachers with regard of the development of executive functions.

The fact that the BRIEF-SR did relate to report marks, while the objective neuropsychological tests (Sorting Test and Tower Test) did not, may illustrate the higher ecological validity of the BRIEF-SR. Yet, controlling for grade, sex, and LPE removed the relation between the BRIEF-SR and report marks. Thus, if one knows whether a student in a preuniversity educational track is a boy or a girl, which grade the student is in, and the educational level of the parents, one can predict report marks as well as with the score on the BRIEF-SR. This indicates that the BRIEF-SR measured aspects of performance in school that could be explained by grade, sex, and LPE.

Why could the executive function tests in the present study not predict school performance any better than grade, sex, and LPE? One explanation has to do with the fact that our study used a sample which was homogeneous with respect to educational track: only students from pre-university level were investigated, i.e., 20% of all students in secondary education. This choice will have reduced the influence of the background variables on the relation between executive functions and school performance. Our findings are in line with those of Willoughby et al. (2012), a study that resembles the present study by taking into account more than only measured confounders. Moreover, other studies that did not find any effects between executive functions and school performance may have remained unpublished. Another explanation is that students who are elected for the preuniversity track in secondary education are more mature in executive functioning in comparison to students in other educational tracks. This is of importance since—usually—, primary schools advise each student which educational level in secondary education is appropriate for them. This advice takes into account not only cognitive performance, but also expected development, motivation for school, and study approach (Driessen, 2005). It could be that, unknowingly, executive functions are taken into account as well. Students with good executive skills would then be advised to go to preuniversity education, while students with poorer executive skills would be advised to go to general secondary education or prevocational education. Future research should investigate the relation between executive functions and school performance in general secondary education and prevocational education as well. Possibly, effects will be found at these other educational levels.

However, also within preuniversity education, students themselves report that they differ with respect to their executive function skills (Coenen et al., 2011). Moreover, sex differences in self-control, which is closely linked to executive functions, appear to contribute to sex differences in school performance (Downey et al., 2005; Duckworth and Seligman, 2006; Hyde et al., 2007; Steinmayr and Spinath, 2008). This may indicate that the executive function tests used in this study were not sensitive enough to measure differences in high-performing healthy adolescents. Executive function tests used in clinical practice are often not sensitive enough to distinguish executive function difficulties in clinical groups (Chan et al., 2008), let alone in healthy subjects. Since each executive function test also measures other (not executive) functions, so called task-impurity, this may trouble the accurate measurement of executive functions (Miyake and Friedman, 2012). To accurately measure differences in executive functions between healthy high-performing adolescents, other tests, a combination of tests, or statistical methods such as the latent variable approach may be needed (Miyake and Friedman, 2012).

A strong point of the current study is that it used report marks to estimate school performance. Most studies measure school performance with standardized tests. An advantage of standardized tests is that these tests are similar for all participants in the study (OECD, 2007). A disadvantage of standardized tests is the lack of ecological validity, because standardized tests are not the outcomes on which students are being assessed in school (Cohen et al., 1995; Wolfson and Carskadon, 2003). Students are reliant on report marks for their school success, as report marks indicate whether a student may pass to the next grade or enter a certain school or educational track. Report marks may also have higher reliability than standardized tests because they involve multiple measurements and more closely measure learning that takes place at school (Wolfson and Carskadon, 2003). Thus, report marks may give a better estimation of real life outcomes than standardized tests.

The present study shows that school performance in healthy, high-performing adolescents could not be predicted by scores on the Sorting Test, Tower Test, and BRIEF-SR. It raises the question whether and to what extent school performance in this sample depends on executive functions. In this sample of healthy, high-performing adolescents, school performance may be affected more strongly by other cognitive factors, for example, content knowledge of the school subjects, or psychological factors such as motivation or personality. Moreover, the study illustrates that controlling for confounders is very important in research on the effect of executive functions on school performance (Willoughby et al., 2012). Future research may investigate whether these results also hold for other executive function tests and other samples. For instance, are similar results seen in adolescents who study at other educational levels? And in adolescents who repeated a grade or have a developmental disorder? Based on the current study, we can conclude that the executive functions measured with the Sorting Test, Tower Test, and BRIEF-SR do not play a major role in report marks obtained by healthy, high-performing adolescents.

Author Contributions

All authors contributed to the design and execution of the study. All authors drafted and revised the study, and approved the final version. All authors agreed to be accountable for all aspects of the work.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Alvarez, J. A., and Emory, E. (2006). Executive function and the frontal lobes: a meta-analytic review. Neuropsychol. Rev. 16, 17–42. doi: 10.1007/s11065-006-9002-x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Anderson, P. (2002). Assessment and development of executive function (EF) during childhood. Child Neuropsychol. 8, 71–82. doi: 10.1076/chin.8.2.71.8724

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Best, J. R., and Miller, P. H. (2010). A developmental perspective on executive function. Child Dev. 81, 1641–1660. doi: 10.1111/j.1467-8624.2010.01499.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Best, J. R., Miller, P. H., and Naglieri, J. A. (2011). Relations between executive function and academic achievement from ages 5 to 17 in a large, representative national sample. Learn. Individ. Differ. 21, 327–336. doi: 10.1016/j.lindif.2011.01.007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Chan, R. C. K., Shum, D., Toulopoulou, T., and Chen, E. Y. H. (2008). Assessment of executive functions: review of instruments and identification of critical issues. Arch. Clin. Neuropsychol. 23, 201–216. doi: 10.1016/j.acn.2007.08.010

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Checa, P., Rodriguez-Bailon, R., and Rueda, M. (2008). Neurocognitive and temperamental systems of self-regulation and early adolescents' social and academic outcomes. Mind Brain Educ. 2, 177–187. doi: 10.1111/j.1751-228X.2008.00052.x

CrossRef Full Text

Checa, P., and Rueda, M. R. (2011). Behavioral and brain measures of executive attention and school competence in late childhood. Dev. Neuropsychol. 36, 1018–1032. doi: 10.1080/87565641.2011.591857

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Clark, M. A., Thompson, P., and Vialle, W. (2008). Examining the gender gap in educational outcomes in public education: involving pre-service school counsellors and teachers in cross-cultural and interdisciplinary research. Int. J. Adv. Couns. 30, 52–66. doi: 10.1007/s10447-007-9044-7

CrossRef Full Text

Coenen, J., Meng, C., and Van Der Velden, R. (2011). Schoolsucces van Jongens en Meisjes in het HAVO en VWO: Waarom Meisjes Het Beter Doen (No. ROA-R-2011/2). nieuws.wegenerweb.nl. Maastricht: Researchcentrum voor Onderwijs en Arbeidsmarkt (ROA).

Cohen, G., Bronson, M., and Casey, M. (1995). Planning as a factor in school achievement. J. Appl. Dev. Psychol. 16, 405–428. doi: 10.1016/0193-3973(95)90027-6

CrossRef Full Text

Dekker, S., Krabbendam, L., Aben, A., De Groot, R., and Jolles, J. (2013a). Coding task performance in early adolescence: a large-scale controlled study into boy-girl differences. Front. Psychol. 4:550. doi: 10.3389/fpsyg.2013.00550

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Dekker, S., Krabbendam, L., Lee, N. C., Boschloo, A. M., De Groot, R., and Jolles, J. (2013b). Sex differences in goal orientation in adolescents aged 10-19: the older boys adopt work-avoidant goals twice as often as girls. Learn. Individ. Differ. 26, 196–200. doi: 10.1016/j.lindif.2012.07.011

CrossRef Full Text

Delis, D. C., Kaplan, E., and Kramer, J. H. (2001). Delis-Kaplan Executive Function System, Technical Manual. San Antonio, TX: Psychological Corporation.

Downey, D., Yuan, V., and Anastasia, S. (2005). Sex differences in school performance during high school: puzzling patterns and possible explanations. Soc. Q. 46, 299–321. doi: 10.1111/j.1533-8525.2005.00014.x

CrossRef Full Text

Driessen, G. (2005). De totstandkoming van de adviezen voortgezet onderwijs: invloeden van thuis en school. [The coming about of the recommendation for secondary school]. Pedagogiek 25, 279–298.

Driessen, G., and Van Langen, A. (2010). De Onderwijsachterstand van Jongens: Omvang, Oorzaken en Interventies [The Educational Disadvantage of Boys: Magnitude, Causes and Interventions]. Nijmegen: ITS: Radboud Universiteit Nijmegen.

Duckworth, A., and Seligman, M. (2006). Self-discipline gives girls the edge: gender in self-discipline, grades, and achievement test scores. J. Educ. Psychol. 98, 198–208 doi: 10.1037/0022-0663.98.1.198

CrossRef Full Text

Giedd, J. N. (2008). The teen brain: insights from neuroimaging. J. Adolesc. Health 42, 335–343. doi: 10.1016/j.jadohealth.2008.01.007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gioia, G. A., and Isquith, P. K. (2004). Ecological assessment of executive function in traumatic brain injury. Dev. Neuropsychol. 25, 135–158. doi: 10.1080/87565641.2004.9651925

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gogtay, N., Giedd, J. N., Lusk, L., Hayashi, K. M., Greenstein, D. K., Vaituzis, A. C., et al. (2004). Dynamic mapping of human cortical development during childhood through early adulthood. Proc. Natl. Acad. Sci. U.S.A. 101, 8174–8179. doi: 10.1073/pnas.0402680101

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Guy, S. C., Isquith, P. K., and Gioia, G. A. (2004). BRIEF-SR: Behavior Rating Inventory of Executive Function-Self-Report Version: Professional Manual. Lutz, FL: Psychological Assessment Resources, Inc.

Hackman, D. A., and Farah, M. J. (2009). Socioeconomic status and the developing brain. Trends. Cogn. Sci. 13, 65–73. doi: 10.1016/j.tics.2008.11.003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hofmann, W., Schmeichel, B. J., and Baddeley, A. D. (2012). Executive functions and self-regulation. Trends Cogn. Sci. 16, 174–180. doi: 10.1016/j.tics.2012.01.006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Huizinga, M., Dolan, C. V., and Van der Molen, M. W. (2006). Age-related change in executive function: developmental trends and a latent variable analysis. Neuropsychologia 44, 2017–2036. doi: 10.1016/j.neuropsychologia.2006.01.010

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hyde, J., Lindberg, S., and Wilson, A. (2007). “Facts and assumptions about the nature of gender differences and the implications for gender equity,” in Handbook for Achieving Gender Equity Through Education, eds S. S. Klein, B. Richardson, D. A. Grayson, L. H. Fox, C. Kramarae, D. S. Pollard, and C. A. Dwyer (Mahwah, NJ: Lawrence Erlbaum Associates, Inc), 19–32.

Jimerson, S. (2001). Meta-analysis of grade retention research: Implications for practice in the 21st century. Sch. Psychol. Rev. 30, 420–437.

Latzman, R. D., Elkovitch, N., Young, J., and Clark, L. A. (2010). The contribution of executive functioning to academic achievement among male adolescents. J. Clin. Exp. Neuropsycol. 32, 455–462. doi: 10.1080/13803390903164363

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lenroot, R. K., and Giedd, J. N. (2010). Sex differences in the adolescent brain. Brain Cogn. 72, 46–55. doi: 10.1016/j.bandc.2009.10.008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Luciana, M., Collins, P. F., Olson, E. A., and Schissel, A. M. (2009). Tower of London performance in healthy adolescents: the development of planning skills and associations with self-reported inattention and impulsivity. Dev. Neuropsychol. 34, 461–475. doi: 10.1080/87565640902964540

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Machin, S. (2005). Gender and student achievement in english schools. Oxford Rev. Econ. Pol. 21, 357–372. doi: 10.1093/oxrep/gri021

CrossRef Full Text

Martens, R., Hurks, P. P. M., Meijs, C., Wassenberg, R., and Jolles, J. (2011). Sex differences in arithmetical performance scores: central tendency and variability. Learn. Individ. Differ. 21, 549–554. doi: 10.1016/j.lindif.2011.06.003

CrossRef Full Text

Miller, M., and Hinshaw, S. P. (2010). Does childhood executive function predict adolescent functional outcomes in girls with ADHD? J. Abnorm. Child Psychol. 38, 315–326. doi: 10.1007/s10802-009-9369-2

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Miller, M., Nevado-Montenegro, A. J., and Hinshaw, S. P. (2012). Childhood executive function continues to predict outcomes in young adult females with and without childhood-diagnosed ADHD. J. Abnorm. Child Psychol. 40, 657–658. doi: 10.1007/s10802-011-9599-y

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ministry of Education, Culture and Science. (2006). Besluit Vaststelling Kerndoelen Onderbouw VO en Aanpassing INRICHTINGSBESLUIT WVO [Decision Establishing core Objectives for the Lower School in Secondary Education and Amending the Decision Implementing the Law on Secondary Education] (Kamerstuk WJZ/2006/24505(3805)). Available online at: http://www.rijksoverheid.nl/bestanden/documenten-en-publicaties/kamerstukken/2006/06/16/besluit-vaststelling-kerndoelen-onderbouw-vo-en-aanpassing-inrichtingsbesluit-wvo/24506a.pdf (Accessed: May/20/2010).

Ministry of Education, Culture and Science. (2009). Kerncijfers 2004-2008: Onderwijs, Cultuur en Wetenschap [Core figures 2004-2008: Education, Culture and Science]. (OCW39.008/1.000/08BK2009B003). Den Haag: Ministry of Education, Culture and Science.

Miyake, A., and Friedman, N. P. (2012). The nature and organization of individual differences in executive functions: four general conclusions. Curr. Dir. Psychol. 21, 8–14. doi: 10.1177/0963721411429458

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

OECD. (2007). PISA 2006. Vol. 1: Analysis. Paris: OECD. doi: 10.1787/9789264040014-en

CrossRef Full Text

Olson, K., Jacobson, K. K., and Van Oot, P. (2013). Ecological validity of pediatric neuropsychological measures: current state and future directions. Appl. Neuropsychol. Child 2, 17–23. doi: 10.1080/21622965.2012.686330

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Reed, H., Ouwehand, C., Van der Elst, W., Boschloo, A., and Jolles, J. (2010). Measures of school performance for use in educational neuropsychology. Front. Neurosci. Conference Abstract EARLI SIG22 - Neuroscience and Education. doi: 10.3389/conf.fnins.2010.11.00058

CrossRef Full Text

St Clair-Thompson, H. L., and Gathercole, S. E. (2006). Executive functions and achievements in school: shifting, updating, inhibition, and working memory. Q. J. Exp. Psychol. 59, 745–759. doi: 10.1080/17470210500162854

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Steenbergen-Hu, S., and Moon, S. M. (2011). The effects of acceleration on high-ability learners: a meta-analysis. Gifted Child Q. 55, 39–53. doi: 10.1177/0016986210383155

CrossRef Full Text

Steinmayr, R., and Spinath, B. (2008). Sex differences in school achievement: what are the roles of personality and achievement motivation? Eur. J. Personality 22, 185–209. doi: 10.1002/per.676

CrossRef Full Text

Van den Bos, W., Crone, E. A., and Güroğlu, B. (2012). Brain function during probabilistic learning in relation to IQ and level of education. Dev. Cogn. Neurosci. 2s, s78–s89. doi: 10.1016/j.dcn.2011.09.007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Van Langen, A., Bosker, R., and Dekkers, H. (2006). Exploring cross-national differences in gender gaps in education. Educ. Res. Eval. 12, 155–177. doi: 10.1080/13803610600587016

CrossRef Full Text

Veenman, M. V. J., Kok, R., and Blöte, A. W. (2005). The relation between intellectual and metacognitive skills in early adolescence. Instr. Sci. 33, 193–211. doi: 10.1007/s11251-004-2274-8

CrossRef Full Text

Willoughby, M. T., Kupersmidt, J. B., and Voegler-Lee, M. E. (2012). Is preschool executive function causally related to academic achievement? Child Neuropsychol. 18, 79–91. doi: 10.1080/09297049.2011.578572

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wolfson, A. R., and Carskadon, M. A. (2003). Understanding adolescents' sleep patterns and school performance: a critical appraisal. Sleep Med. Rev. 7, 491–506. doi: 10.1016/S1087-0792(03)90003-7

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text