The Use of a Serious Game to Assess Inhibition Mechanisms in Children

The design and implementation of a serious game (SG) concerning inhibition skills in children are presented. The SG consists of a set of activities, each eliciting the tendency to respond in an immediate and inappropriate (wrong) way. The SG is based on the Dual Pathway model of attention-deficit/hyperactivity disorder (ADHD) proposed by Sonuga-Barke and on the Unity/Diversity model of executive functions proposed by Miyake. In the SG, children must block impulsive tendencies, reflect upon the situation, inhibit irrelevant thoughts, and find the non-immediate solution. A study was carried out by testing the SG on typically developing primary school children (30 children, 16 boys; age, M = 9.30 years, SD = 0.87) to verify that it measures the same variables addressed by tests usually employed to assess attention ability in children and to diagnose ADHD. Three standardized tasks belonging to the Italian Battery for ADHD were administered, as well as an ad hoc questionnaire devised to check the acceptability, usability, and comprehensibility of the SG. Positive correlations between impulsiveness as measured by standard tests and impulsiveness scores in the SG emerged. These findings support the notion that skills associated with the control of impulsivity are involved in the SG. Furthermore, self-report ratings in the questionnaire showed that the SG is easy to be understood, is engaging, and elicits positive reactions in children.

Introduction

Attention-deficit/hyperactivity disorder (ADHD) is a neurodevelopmental disorder included in the Diagnostic and Statistical Manual of Mental Disorders (DSM-V; American Psychiatric Association, 2013). ADHD is characterized by inattention and hyperactivity. Such symptoms involve the core of executive functions (EFs) (Miyake et al., 2000; Diamond, 2013), namely:

• Inhibition is defined as the deliberate overriding of a dominant or assertive response (Miyake and Friedman, 2012). Inhibition is a multicomponential construct itself and comprehends different abilities such as managing impulses and interferences (Nigg et al., 2004; Diamond, 2013).

• Impulsivity, which occurs in rapid actions that are taken without reflection, resulting from a desire for immediate reward or an inability to delay gratification (Martinez et al., 2016). Impulsivity is not negative at all, but it becomes disabling when it interferes with decision making and doing actions (Kahneman and Tversky, 1974). Impulsivity is often linked with a multitude of behaviors or responses that are poorly conceived, premature, or inappropriate and that frequently result in unwanted or deleterious outcomes (Daruna and Barnes, 1993).

• Cognitive flexibility or shifting, namely, the capacity to change perspective over a task. To do so, people need to deactivate previous perspectives and to retrieve or devise a new one. This involves the ability to adjust and change demands or priorities, as well as to take advantage of unexpected opportunities (Diamond, 2013).

• Working memory (WM) refers to the ability to hold in mind and manipulate useful information available for a short time while completing a task when the information is no longer present (Baddeley and Hitch, 1994; Smith and Jonides, 1999). Diamond (2013) theorized a bidirectional relation between inhibition and WM, by describing them as separate dimensions yet strongly related.

The features of ADHD can be described also from a pathophysiological point of view (Cortese and Castellanos, 2012; Sonuga-Barke and Taylor, 2015). The currently dominant neuro-cognitive model of ADHD stresses the role of dysfunctions underpinned by disturbances in the frontodorsal striatal circuit and associated dopaminergic branches (Sonuga-Barke, 2005). In contrast, motivationally based accounts focus on altered reward processes and implicate frontoventral striatal reward circuits and those mesolimbic branches that terminate in the ventral striatum, especially the nucleus accumbens (Sonuga-Barke, 2003, 2005).

ADHD is meant as a motivational style characterized by attempts to escape or avoid delay of rewards, which arises from fundamental disturbances in reward centers (Sonuga-Barke, 2003). While traditionally regarded as competing, these models have been presented as complementary accounts of two psychopathophysiological subtypes of ADHD with different developmental pathways, underpinned by different corticostriatal circuits and modulated by different branches of the dopamine system (Sonuga-Barke, 2005). In the Dual Pathway model of ADHD, the first pathway (the cognitive one) concerns EFs, including WM and inhibition, while the motivational path is linked to delay sensitivity and aversion (i.e., the tendency to choose a smaller immediate reward rather than waiting for a larger delayed reward) (Sjöwall et al., 2013). In this perspective, ADHD is considered as depending on hypersensitivity to reward-related delay and characterized by an abnormal sensitivity to reinforcement (reward, punishment, and response cost) (Sonuga-Barke, 1996).

The cognitive path of the Dual Pathway model shows some overlaps with the Unity/Diversity model of EFs proposed by Miyake et al. (2000), and Miyake and Friedman (2012), which focuses on three aspects of EFs: updating WM, shifting, and inhibition. The model by Miyake states that the key requirement for response inhibition is the ability to be captured by common EFs, whereas stopping itself may be relatively automatic (Chatham et al., 2012). Hence, inhibition deficits should merit special attention in the assessment and treatment of ADHD.

Considering the neurocognitive aspect of ADHD is fundamental in order to plan functional and useful long-term effect training (Chacko et al., 2018).

Technological Tools Addressed to ADHD

Technology represents a potentially useful instrument to help children with ADHD to manage and monitor behavior (Simons et al., 2016; Powell et al., 2019). The literature presents some applications and games supporting parents in managing and monitoring children (e.g., by suggesting daily routines, such as getting dressed and going to bed); other apps, instead, aim at enabling children to monitor their own behavior (Powell et al., 2017). With respect to gaming solutions, Kulman (2012) claimed that playing a computer game involves EFs and metacognitive skills. In fact, the problem-solving skills needed to accomplish a task within the game imply to identify the relevant strategies to be applied (Antonietti et al., 2000), an ability that is poorly developed in people with ADHD. Furthermore, since children with ADHD get bored fast, it is important to constantly catch their attention. Serious games (SGs) can meet these requirements because the player has to stay focused all the time. Moreover, all the senses of the players are stimulated, and this makes the activities more interesting (Wronska et al., 2015). Finally, children with ADHD prefer immediate reward, and they need to maintain the experience at a “minimum waiting level” (Sonuga-Barke, 1996). All these aspects can be found in technological solutions such as SGs, which, therefore, represent valuable tools to engage children with ADHD.

Several studies demonstrate the usefulness of SGs for enhancing some positive attitudes (Beale et al., 2007) in children with ADHD (Rassin et al., 2004) by increasing problem-solving strategies (Coyle et al., 2007) and by modifying abnormal behaviors (Walshe et al., 2003). An example is the Harvest Challenge BCI Videogame (Muñoz et al., 2015), which consists of three mini-games that work with the use of neurofeedback. Plan-It Commander (Bul et al., 2015) is an online computer game structured in two parts: the mission game, formed of three isolated mini-games with embedded learning goals, and a closed social community interacting through predefined messages. Braingame Brian is an SG aimed at improving WM in children with ADHD aged 8–12 years (Van der Oord et al., 2014). PlayMancer (Fernández-Aranda et al., 2012) has been proven able to change underlying attitudinal, behavioral, and emotional processes of patients with impulse-related disorders, which are similar to ADHD. The Virtual Classroom (Rizzo et al., 2002; Coleman et al., 2019) was developed for the study, assessment, and rehabilitation of cognitive processes in patients with different forms of central nervous system dysfunction. It offers an assessment in a real-world dynamic simulation with distractors that mimic situations typically occurring in a classroom (Parsons et al., 2019).

In addition to SGs, recent studies applied Augmented Reality to rehabilitation of children with ADHD. As an example, the project Beyond the tReatment of the Attention deficit hyperactiVity disOrder (BRAVO) aims at implementing an immersive therapeutic game context to improve the relationships between young patients and therapies. The solution, based on an ICT system, combines the use of SGs, wearable equipment, and Virtual and Augmented Reality devices (Barba et al., 2019).

SGs as Assessment Tools for Children With ADHD

Even though the assessment was one of the aims of the SGs mentioned before, they were mainly designed as training tools. Therefore, data supporting their use as reliable tests for assessment are lacking. Furthermore, most of them are not grounded on a specific model of the mechanisms underlying ADHD. Technology, which should be reliably helpful, must meet some quality standards; however, SGs are often developed rapidly and are based on little evidence (Powell et al., 2017). Hence, the need for an SG based on updated and well-reputed theories and properly tested as an assessment tool is still unmet. The goal of the SG described in this paper is to address such a need.

Being inhibition involved in the cognitive pathway of ADHD, according to the development of the Unity/Diversity model, the two main goals of the SG are to assess the control of impulsivity and to prevent non-adaptive behaviors and irrelevant thoughts through inhibition. The purpose is to provide a technological tool able to assess to what extent children with ADHD handle impulsivity by implementing functional inhibition strategies.

Symptoms of ADHD are basically assessed with clinical interviews, behavioral observation, standardized tests or questionnaires, and information gathered from parents, teachers, and clinicians (Mühlberger et al., 2020). Recent studies show that SGs concerning EFs are useful for exploring the differences between children with ADHD and controls, as well as among ADHD subtypes (Areces et al., 2019). In this paper, we describe the structure of an SG aimed to assess inhibition mechanisms in children. We verified whether the abilities the SG involves correspond to those measured by some of the standard tests usually employed to diagnose lack of inhibition in ADHD, in order to be sure that the SG in the future could be used to assess ADHD.

The SG is addressed to children aged 8–11 because this is the period when children begin to master efficient self-management strategies and are able to generalize them in different everyday-life contexts (Welsh et al., 2006; Prencipe et al., 2011).

Antonyms: A SG for Children With ADHD

The SG, Antonyms, was designed as a single storyline that includes different scenarios. In this way, the child has an overview of the activities he/she will have to perform, which should help him/her to carry out every single adventure. Many existing SGs are made up of mini-games. This approach allows the child to focus on one aspect at a time. Antonyms consist of a series of activities, each eliciting the tendency to respond in an immediate, impulsive way (see the flow chart in Appendix 1). In the SG, children must block such a tendency, reflect upon the situation, and find the non-intuitive solution.

The SG can run on a tablet, a smartphone, or a touchscreen laptop. The player interacts with the environments by tapping on the screen or by dragging and dropping objects. If no touch screen is available, he/she can interact through the keyboard using the “Space bar” and Return key. Antonyms has been developed in Unity3D, a game development platform, based on C# programming language. A more detailed description of the SG architecture is reported by Crepaldi et al. (2017). Scores computed in each part of the game can be exported in an Excel database, and all the results are saved immediately for each player.

The SG is characterized by the presence of a narrative frame that involves players in all parts of the story. The player personifies a superhero (called Atansyon), who is asked to save a realm on the opposite side of the Earth (Antonyms). Atansyon will face different steps to free the planet from enemies. The tasks that require to inhibit responses are contextualized. As a consequence, the activation of inhibition mechanisms has a specific meaning, which is not arbitrary (as it happens in many SGs).

The complete version of the SG consists of three different scenarios. In the present study, only two scenarios are taken into account: Training School and Central Building. At the beginning of the game, the player can choose the main settings of the game: screen resolution, graphic quality, and monitor. After the main settings, the player is required to enter his/her name, surname, age, and gender to create a new user profile. For each scenario of the game, there is a user menu with the options: “Start,” “Settings,” “Instructions,” “Audio,” “Back,” and “Quit” (Figure 1).

FIGURE 1

Figure 1. Settings in antonyms.

Each mini-game has different levels of difficulty so that the activities become more challenging. The transition from a level to the next one is constrained to the number of errors made by children. It is worth mentioning the multimodal, detailed, and immediate feedback provided by the SG in the form of both visual and auditory messages, which have been proven to be highly beneficial to people with ADHD (Fabio and Antonietti, 2012). Moreover, it is possible to examine the player's behavior during the SG by saving the performance in the form of different types of errors (e.g., errors in waiting, wrong answer, etc.) and time spent.

The activities of the SG were inspired by classical neuropsychological tasks addressing several subcomponents of inhibition and attention, such as (1) the Flanker task (Eriksen and Eriksen, 1974), in which the target is flanked by non-target stimuli, which correspond either to the same direction response as the target, to the opposite response, or to neither (respectively: congruent, incongruent, and neutral flankers); (2) the Stroop test (Stroop, 1935; Dalrymple-Alford, 1972), assessing the ability to inhibit an automatic, incorrect response and to produce a non-automatic, correct response; and (3) the Stop signal task (Lappin and Eriksen, 1966; Logan, 1994) in which the subject must respond to an arrow stimulus by selecting one of two options, depending on the direction in which the arrow points. If an audio tone is present, the subject must withhold the production of that response.

The mechanisms underlying these tasks were adapted to create a new assessment tool with a more ecological background, addressing inhibitory control abilities. In a different investigation, Antonyms was administered to a small group of children with ADHD (Crepaldi et al., 2020) who showed significantly lower performances than a matched group of typically developing children, thus suggesting the SG sensitivity.

Method

SG Concurrent Validity: A Pilot Study

Teachers and children, belonging to schools different than those involved in the study, have been involved in the first part of the project, when they were shown examples of the tasks included in the SG and could play a prototype (pilot application) of the SG. A series of features of the SG were changed, according to the feedbacks received about the pilot application. This phase of the project, even though not based on a systematic assessment of the usability and ergonomics of the game but rather on qualitative feedbacks, was nevertheless useful to check the acceptability of the SG. The new version of the game better matched the initial idea to create an SG that is realistic and able to engage children.

Design of the SG: Scenarios and Activities

Two activities from the Antonyms SG were tested in the present study:

• Training School (TS) (Figure 2) is a same-different task addressing visual selective attention and inhibitory control. In the center of the screen, a target stimulus (an object useful to the mission) appears, and the player must compare it with the object that appears on a shelf: if they are the same, the player takes the object from the shelf and puts it in the backpack; otherwise, he/she has to put it in the trash in the shortest possible time. The activities are divided into two different levels: a warm-up (level 1) and the actual game (level 2). The objects from the shelf are dragged and dropped using touch; they should be the same as the given target, but slight differences can hamper the success. A total of 30 items appear in each level of the task, 15 of which are “different” trials, and 15 are “same” trials. The order of appearance of the objects and their position on the shelf are random with the only constraint that the two correct objects from two consecutive groups of stimuli cannot appear in the same position. Errors are classified as Trash Errors (putting the correct object in the trash) and Bag Errors (putting a wrong object in the bag), by referring to the signal detection theory (Swets, 1964), which categorizes, in same-different tasks, incorrect responses in either false alarms or miss responses. In the Training School task, Trash Errors are considered false alarms, since the player incorrectly detected a difference between identical stimuli, whereas Bag Errors are considered miss responses, since the player failed to detect the difference between stimuli. The correct responses and errors in the second level of the game (since the first level was a warm-up task, the actual performance was recorded during the most challenging level) were used to compute a sensitivity score as a measure of discriminability (Macmillan and Creelman, 2004).

• Central Building (CB) (Figure 3) addresses sustained attention and the ability to hold automatic responses. The player walks through the corridors to arrive at the central room. Movements are driven by colored lights appearing on the floor that the player has to select either touching directly the screen or by pressing the Space bar on the keyboard. He/she must walk only when the path is highlighted with a proper light (green light) according to a randomized sequence whose timing is referred to the Conners (1994) Continuous Performance Test, the most used instrument to measure ADHD impairment in impulsivity and inhibition (Mühlberger et al., 2020), which requires respondents to select a response to a particular stimulus that appears at regular intervals (Fang and Dai Han, 2019). The interstimulus intervals are 1, 2, and 4 s with display time of 250 ms. When a different light (blue light) appears or there is no light on the screen, the player must stop. Scores are assigned to the child based on the speed of the response (touch the screen/press the spacebar): five points for an answer in <1 s and 10 points for an answer from 1 to 3 s. Two paths have been created differing in direction and predictability. Both paths can be completed in about 5 min in case of no errors. Starting from these two paths, we designed two levels of difficulty for each route, and, as in the previous scenarios, the transition from one level to another depends on the number of errors. Errors are classified as follows: Anticipation Errors = errors while waiting for the appearance of the light (the player clicks but no light has appeared yet); Position Errors = failure to select the position; impulsivity errors = in level 2, the player selects a blue light; Omission Errors = non-selection. In the cases of Position or Omission Errors, the player retreats by an amount of distance equal to a quarter of the displacement expected to reach the light just missed, for a maximum of four times (i.e., the player has moved backward to the previous position). The instructions of the SG translated in English are available in Appendix 2.

FIGURE 2

Figure 2. Training school.

FIGURE 3

Figure 3. Central building.

Participants

The SG was tested in two primary schools near Milan (Italy), which have had the access to the Service of Learning and Educational Psychology of the Catholic University of the Sacred Heart of Milan in previous circumstances when they took part in research projects. We obtained the permission, through written informed consent, from parents to involve 70 8–11 years old typically developing children, excluding a priori those who had a diagnosis of cognitive impairment, ADHD, or other neurodevelopmental disorders. They were administered the Scala per il Disturbo di Attenzione e Iperattività (Scale for the Attention Deficit and Hyperactivity) (SDAI) (Marzocchi and Cornoldi, 2000) included in the Italian Battery for ADHD (BIA: Marzocchi et al., 2010). SDAI is the most commonly used screening tool for ADHD in Italy. Its standard use is based on the frequency with which each of the behaviors reported in the questionnaire is manifested. The 18-item scale is completed by the teacher, who must evaluate, for each of the behaviors listed, the frequency with which it appears. It is composed of two subscales: attention (odd items) and hyperactivity/impulsivity (even items). Responses are on a Likert scale (0–3) for each item (0 if the child never shows that behavior; 1 if the child sometimes presents that behavior; 2 if the child presents it often; 3 if the child presents it very often). According to Marzocchi and Cornoldi (2000), the cutoff score to suspect the presence of ADHD is >14 in one of the two subscales. Since we were interested in impulsivity, only the score of the relevant subscale was taken into account. In the sample, no child obtained a score of 14 or higher in the hyperactivity/impulsivity subscale. From the initial sample of 70 children, 30 children−15 from the lowest range (0–6) and 15 from the highest range (7–13)—were randomly extracted by researchers for the next step so to have a reasonable (in term of time needed for each subject) number of participants who could studied individually and a heterogeneous sample (but still constituted by typically developing children). Children were distributed evenly in each grade (third, fourth, and fifth). Gender was equally distributed in all grades [${\chi }_{(2)}^2$ = 0.268; p = 0.875] and in the two SDAI level groups [${\chi }_{(1)}^2$ = 0.536; p = 0.464].

Assessment Instruments

The standard tests chosen from the Italian Battery for ADHD (BIA, Marzocchi et al., 2010) were the following:

• Ranette (small frogs), which is a mirror of the “Walk/Don't walk” task. The test involves sustained attention (the test lasts about 10 min), selective attention (children must detect the target sound), and motor inhibition (children must stop the impulsive response to go ahead). In this task, the errors represent a failed inhibition of the answer to “GO” to the right time (difficulty in controlling inhibitory, typical of children with ADHD). The total score consists of the number of correct responses in the 20 items presented. In the analyses we carried out, we considered the complementary score (i.e., the number of errors) as a measure of sustained attention, to facilitate the interpretation of the relations between different measures.

• Number Stroop task, which measures interference control and the ability to stop an automatic, but incorrect, response. The task consists of two parts: a baseline task, in which individuals are asked to count how many asterisks are present in each box, and a Stroop task, in which the request is to count how many elements (numbers) are present in each cell. Two kinds of errors may occur: identity errors (when the written number is read instead of the number of stimuli presented; interference of stimuli occurs) and counting errors (when there are errors in counting the stimuli). For the sake of the analyses, the number of interference errors was considered as a measure of inhibition.

• MF20, an adaptation of the Matching Familiar Figure Test (MFFT), which consists of 20 items. The task measures visual selective attention and inhibitory control of impulsive responses. Each item is made up of two pages: on the first page, a target figure is shown; on the second page, six figures are shown; they are similar to the target but only one is just the same as the target. The task is to choose the figure that is identical to the target. Then, the following variables are computed: Time of the First answer and Number of Errors. We decided to include in the analyses the number of errors score only, namely, the most discriminant measure, as reported by previous literature on the use of the MF task for measuring impulsivity (Homatidis and Konstantareas, 1981; Brown and Wynne, 1984; Milich and Kramer, 1984; Marzocchi and Cornoldi, 1998).

• Ad hoc questionnaire. Children were asked how they felt playing the game (Likert scale); if it was enjoyable or boring (Likert scale); if there was something that they would add in the scenarios (open questions); if the story was simple or difficult (Likert scale); and to summarize the storyboard (open question). The questionnaire is reported in Appendix 3.

Procedure

The protocol included three individual sessions for each child. Children carried out the tasks at school in a quiet room in the morning with the presence of a psychologist. All children played all the levels but in a slightly different time, depending on errors and execution speed. The overall time, however, varied in a not very broad range, for a total of 45 min for each section. In the first session, each child completed the Ranette test, the Stroop test, the first level of the Training School (in Antonyms), and the first path of the Central Building (in Antonyms). In the second session, MF20 was proposed together with the second level of the Training School and the remaining paths of the Central Building. In the last session, the ad hoc questionnaire was proposed. Each child performed the same tests in the same order and the same activities of the SG. Neither reinforcements nor credits were given to the participants.

Statistical Methods

Statistical analyses were performed with Jamovi R-based software, version 1.0.8.0 (2019), The Jamovi project (2020), IBM Corp (2016), and SPSS, version 24 (2016). Given the count nature of error scores for both the standardized tests and the SG activities, assumptions of normality (all Shapiro–Wilk ps < 0.01, with the only exception of the MF20 error score), homogeneity of error variance, and linearity of the associations were not met. Therefore, non-parametric analyses have been performed.

As for the Training School task, the proportion of hits (i.e., items correctly put in the trash) and false alarm errors (i.e., items incorrectly put in the trash) was used to compute a sensitivity score as a measure of discriminability {d′ = [z(Hit) – z(FA)]} (Macmillan and Creelman, 2004). A criterion score, representing the bias toward the same (i.e., items put in the bag) response type (C > 0) or different (i.e., items put in the trash) response type (C < 0), has been computed as well {C = –[z(Hit) + z(FA)]/2} and considered in the analyses.

First, descriptive analyses have been carried out, and gender, school-grade, and SDAI's level differences for all measures have been tested. The p-value was adjusted to compensate for multiple comparisons, using the Bonferroni correction (alpha = 0.05/9 = 0.005). Afterwards, we computed correlations (Spearman's rho) between the measures of the BIA (Ranette, Number Stroop, and MF20). The rationale of these analyses was to check that these tests, usually employed for diagnostic aims, were adequate to the purpose of the present study: Coherent patterns of correlation, in the expected directions, between them would support their relevance. Then, correlations between the BIA tests and the performance in Antonyms (scores in Training School and Central Building) were computed to verify, according to the main goal of the paper, if the SG can be considered a valid instrument to measure impulsivity tendencies, and therefore testing its concurrent validity. When school-grade differences have been found, partial Spearman's correlations controlling by school grade have been computed. Finally, responses to the ad hoc questionnaire were analyzed to draw conclusions about the subjective experience with the SG.

Results

Neither SG performance scores nor the other standardized tasks from the BIA battery (i.e., Ranette and Number Stroop) differed between gender groups (all ps > 0.46) (Table 1).

TABLE 1

Table 1. Gender differences in SG activities (TS, Training School; CB, Central Building) and in the standardized tests (BIA battery).

Similar performances were measured in different school grades (i.e., third, fourth, and fifth) in all tasks (all ps > 0.01, Table 2). The SDAI level (medium-low vs. medium-high) was not found to discriminate children's performance in either the SG tasks, or the standardized tests, except for the Ranette subtest (Table 3), although in the normal range, a lower SDAI level was found to be associated with a lower level of sustained attention, as measured by the Ranette task.

TABLE 2

Table 2. School grade differences in SG activities (TS, Training School; CB, Central Building) and in the standardized tests (BIA battery).

TABLE 3

Table 3. SDAI level differences in SG activities (TS: Training School; CB: Central Building) and in the standardized tests (BIA battery).

Within BIA tests, we found that scores in the Ranette test (number of errors in the auditory sustained attention test) were positively correlated to the interference errors in the Number Stroop test (the score indicates the errors that children make in pronouncing the stimuli; rho = 0.37; p < 0.05, one-tailed). High Ranette scores (right actions) were associated to less impulsive behaviors and more attention in the tasks. There was a negative correlation between errors in MF (when children choose the wrong figure) and time of the performance (more time spent in the performance is linked with a greater accuracy of the answer and fewer errors; rho = 0.62; p < 0.001, one-tailed). This means that the more the child was impulsive, the less time he/she took to complete the task and the more mistakes he/she did. These patterns of correlations were expected and are reported just to support the validity of the assessment tools we used in the kind of population the sample belongs to.

Concerning the relationships between the standardized tests and performances in the SG activities, errors in MF20 performance correlated negatively with the sensitivity score in the Training School (Table 4). This association suggests that a lower performance in the SG task (lower sensitivity scores) is indicative of a failed inhibition of the automatic response (higher number of errors in MF20) and, therefore, emphasizes that the two tasks measure the same construct involving the similar basic principle. Conversely, the bias score in the Training School task was not correlated with any standardized measure. Scores in the Ranette test (number of errors in the sustained attention task) were positively correlated to anticipation errors in the Central Building scenario (measure of the difficulty for children to stop their irrelevant actions) (Table 4). Errors of interference in the Stroop Test (when children say the number written in the box and not the correct number of stimuli inside the box) correlated with errors in anticipation and omission in the Central Building (failure in touching the light) but not with sensitivity in Training School. This may be because the performance request in the Training School is more ecological and helps children in considering details and differences; the Stroop test, instead, is more specific, and there is more conflict between the two codes considered. To sum up, there was evidence of correlations between standard tests, like the tasks from BIA and performances in the SG.

TABLE 4

Table 4. Spearman's correlation matrix between SG activities (TS, Training School; CB, Central Building) and standardized tests (BIA battery).

Concerning the ad hoc questionnaire, instructions were considered easy, and the only difficulties that children underlined were linked to some words [three third-grade children and one fourth-grade child marked as difficult words: perfido (perfidious), impulsivo (impulsive)]. Twenty-eight children reported that the story is easy, and only two claimed that the story is difficult to follow. In general, all players understood the game and remembered that Antonyms is the planet of “contrary thoughts and actions,” that is, it is a “slowly” planet and that there were enemies who captured the inhabitants. All participants accomplished the different tasks without the help of an adult.

The main difficulties described in the Training School were that some objects differed because of small details. This may depend on the fact that the background color of the shelf is sometimes misleading with respect to the color of the objects that appear. The main difficulties reported in Central Building were that the player must wait for different time lapses before the light appears, and he/she must remember not to click on the blue light. In the second level, when the lights were smaller, it was difficult for children to properly select the light. We suggest that such difficulty can be explained by the higher level of required sustained attention necessary to detect a small visual cue, therefore addressing one of the main ADHD-related symptoms, and not by any issues in the perception of a small visual target.

Given the question “Was Antonyms enjoyable or boring?,” 29 children answered “enjoyable” and only one fifth-grade child found it “boring.” Twenty-five participants liked the game, and only five liked quite the game; none answered that he/she did not like it.

When requested to provide suggestions to improve the SG, three children would add the avatar of Antonyms to modify the hero and make him/her similar to themselves. According to the responses given in the questionnaire, we can conclude that the general aims of the SG were understood by children. They reported to have fun in playing the games and appeared to be fully immersed in the task. All these elements support the notion that Antonyms reached a satisfactory level of acceptability.

Discussion and Conclusions

Chacko et al. (2018) pointed out that, although the main interventions now available for children with ADHD consist of a combination of pharmacological treatments and behavioral training [which usually have short-term effects: Daley et al. (2014), DuPaul et al. (2012), and Evans et al. (2014)], it is important to focus attention also on neurocognitive aspects since they, if properly stimulated, have long-term effects. Precisely for this reason, it is important to have engaging tools aimed at strengthening cognitive functions and having a neuropsychological basis, such as the SG presented here.

The optimal intervention should be based on psychosocial skills and take into consideration the neurocognitive mechanisms and processes involved in a specific functional ability together with targeted training to improve and enhance the mechanisms and processes on which these skills are based (Chacko et al., 2018).

The SG was designed, referring to the Dual Pathway model by Sonuga Barke because the basis of the game is the cognitive path that concerns EF aspects involved in ADHD. In particular, inhibition (in the SG, the most immediate action is the wrong one), shifting (cognitive flexibility is requested in order to remember changes in the rules of the game), and WM (remembering the storyboard is needed during the activities) are considered. Considering the two main objectives of the proposed SG (assessing impulsive control and preventing non-adaptive behaviors and irrelevant thoughts through inhibition), the performance and data obtained seem to show a positive result in this direction.

Results suggest that the activities embedded in the SG appear to be associated with the performance in standard tests usually employed to assess inattention, impulsivity, and hyperactivity levels in children. Few significant correlations emerged, but it is interesting to observe that these concerned only those aspects of the SG that closely match the specific skills measured by those tests. No significant differences emerged in the SG scores between SDAI levels (i.e., medium-low vs. medium-high), which reflects the trend obtained in standardized tests. These results are not surprising, since the sample was composed only by typically developing children. In the study by Crepaldi et al. (2020) on ADHD children, Antonyms scores were found to be significantly different between a group of children with a diagnosis of ADHD and a paired subgroup of typically developing children (which was extracted from the larger group recruited for the present study). Since in Antonyms there are no significant differences between different levels of the normal SDAI score range, but instead differences emerged when compared to a clinical sample (SDAI score in the clinical range, >14) (Crepaldi et al., 2020), this leads us to suppose that Antonyms could be a useful tool to discriminate between ADHD and healthy children and not between different levels within a non-clinical range.

When designing the SG, we chose to propose activities with a reference to everyday life. The generalizability of tasks to everyday life remains a limit of many existing computer activities, which we tried to overcome in Antonyms. For example, the path in the Central building may be compared to the attention that children must pay when they are walking on a street: in this situation, children must be careful where they put their feet, stop when there is a danger, and proceed only when it is allowed. The activities in the Training School can mirror preparing the backpack for school, which is an activity in which children with ADHD often struggle.

The SG has been ultimately designed to assess impulsivity and inhibition in children with ADHD and ADHD-related characteristics. However, since the abilities that are lacking in ADHD are not all/nothing variables, a continuum in performance can be assumed. Hence, the SG can be employed also to check if inhibitory skills are poorly expressed in typically developing children. Another advantage is that scoring is automatized and, therefore, more accurate and quicker. Furthermore, only a laptop is needed to administer the SG, whereas traditional tests require a series of different materials (sheets, cards, audio-recorder, pencils), which are continuously presented and removed from the child, thus by distracting him/her.

In the future, some critical ergonomic points stressed by the participants might be improved, and the ergonomic aspects of the SG might be tested in a systematic way. We also plan to implement more mini-games with other activities to test the efficacy of Antonyms in enhancing the ability to keep attention in daily life situations and to propose the game to a large sample of children diagnosed with ADHD.

Furthermore, it will be advisable to verify the influence and role of the greater engagement of the SG compared to the pencil paper tasks, but it seems that, in comparison to standardized test, the SG is more motivating for children. If, as it seems, this is verified, it could prevent evaluators to struggle to attract and keep attention of the patient, a problem which is rather common in the assessment of ADHD.

We expect that children who will be engaged in this SG could improve attention and learn strategies to manage impulsivity so to inhibit irrelevant thoughts and thus an enlarged version of Antonyms could be proposed as a rehabilitation tool.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by the ethical committee of the Catholic University of Milan. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.

Author Contributions

MC: project, game design, tasks administration to children, data analysis, and writing. AA: project, game design supervision, data analysis, and contacts with schools. MS, SM, VC, and DB: game design and implementation and supervision. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomp.2020.00034/full#supplementary-material

References

American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders: DSM 5, 5th Edn. Arlington, VA: American Psychiatric Publishing. doi: 10.1176/appi.books.9780890425596

CrossRef Full Text

Antonietti, A., Ignazi, S., and Perego, P. (2000). Metacognitive knowledge about problem-solving methods. Br. J. Educ. Psychol. 70, 1–16. doi: 10.1348/000709900157921

PubMed Abstract | CrossRef Full Text | Google Scholar

Areces, D., García, T., Cueli, M., and Rodríguez, C. (2019). Is a virtual reality test able to predict current and retrospective ADHD symptoms in adulthood and adolescence? Brain Sci. 9:274. doi: 10.3390/brainsci9100274

PubMed Abstract | CrossRef Full Text | Google Scholar

Baddeley, A. D., and Hitch, G. J. (1994). Developments in the concept of working memory. Neuropsychology 8, 485–493. doi: 10.1037/0894-4105.8.4.485

CrossRef Full Text | Google Scholar

Barba, M. C., Covino, A., De Luca, V., De Paolis, L. T., D'Errico, G., Di Bitonto, P., et al. (2019). “BRAVO: a gaming environment for the treatment of ADHD,” in International Conference on Augmented Reality, Virtual Reality and Computer Graphics (Cham: Springer), 394–407. doi: 10.1007/978-3-030-25965-5_30

CrossRef Full Text | Google Scholar

Beale, I. L., Kato, P. M., Marin-Bowling, V. M., Guthrie, N., and Cole, S. W. (2007). Improvement in cancer-related knowledge following use of a psychoeducational video game for adolescents and young adults with cancer. J. Adolesc. Health. 41, 263–270. doi: 10.1016/j.jadohealth.2007.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, R. T., and Wynne, M. E. (1984). Attentional characteristics and teachers ratings in hyperactive, reading disabled, and normal boys. J. Clin. Child Adolesc. Psychol. 13, 38–43. doi: 10.1207/s15374424jccp1301_6

CrossRef Full Text | Google Scholar

Bul, K. C. M., Franken, I. H. A., Van der Oord, S., Kato, P. M., Danckaerts, M., Vreeke, L. J., et al. (2015). Development and user satisfaction of “plan-it commander,” a serious game for children with ADHD. Games Health J. 4, 502–512. doi: 10.1089/g4h.2015.0021

PubMed Abstract | CrossRef Full Text | Google Scholar

Chacko, A., Bedard, A. C., Marks, D., Gopalan, G., Feirsen, N., Uderman, J., et al. (2018). Sequenced neurocognitive and behavioral parent training for the treatment of ADHD in school-age children. Child Neuropsychol. 24, 427–450. doi: 10.1080/09297049.2017.1282450

PubMed Abstract | CrossRef Full Text | Google Scholar

Chatham, C. H., Claus, E. D., Kim, A., Curran, T., Banich, M. T., et al. (2012). Cognitive control reflects context monitoring, not motoric stopping, in response inhibition. PLoS ONE 7:e31546. doi: 10.1371/journal.pone.0031546

PubMed Abstract | CrossRef Full Text | Google Scholar

Coleman, B., Marion, S., Rizzo, A., Turnbull, J., and Nolty, A. (2019). Virtual reality assessment of classroom-related attention: an ecologically relevant approach to evaluating the effectiveness of working memory training. Front. Psychol. 10:1851. doi: 10.3389/fpsyg.2019.01851

PubMed Abstract | CrossRef Full Text | Google Scholar

Conners, C. K. (1994). The Conners Continuous Performance Test. Toronto, ON: Multi-Health Systems.

Google Scholar

Cortese, S., and Castellanos, F. X. (2012). Neuroimaging of attention-deficit/hyperactivity disorder: current neuroscience-informed perspectives for clinicians. Curr. Psychiatry Rep. 14, 568–578. doi: 10.1007/s11920-012-0310-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Coyle, D., Doherty, G., Matthews, M., and Sharry, J. (2007). Computers in talk-based mental health interventions. Interact. Comput. 19, 545–562. doi: 10.1016/j.intcom.2007.02.001

CrossRef Full Text | Google Scholar

Crepaldi, M., Colombo, V., Baldassini, D., Mottura, S., and Antonietti, A. (2017). “Supporting rehabilitation of ADHD children with serious games and enhancement of inhibition mechanisms,” in International Conference on Virtual Reality and Augmented Reality (Cham: Springer), 167–181. doi: 10.1007/978-3-319-72323-5_11

CrossRef Full Text | Google Scholar

Crepaldi, M., Colombo, V., Mottura, S., Baldassini, D., Sacco, M., Cancer, A., et al. (2020). Antonyms: a computer game to improve inhibitory control of impulsivity in children with attention deficit/hyperactivity disorder (ADHD). Information 11:230. doi: 10.3390/info11040230

CrossRef Full Text | Google Scholar

Daley, D., Van der Oord, S., Ferrin, M., Danckaerts, M., Doepfner, M., Cortese, S., et al. (2014). Behavioral interventions in attention-deficit/hyperactivity disorder: a meta-analysis of randomized controlled trials across multiple outcome domains. J. Am. Acad. Child Adolesc. Psychiatry 53, 835–847. doi: 10.1016/j.jaac.2014.05.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Dalrymple-Alford, E. C. (1972). Associative facilitation and interference in the stroop color-word task. Percept. Psychophys. 11, 274–276. doi: 10.3758/BF03210377

CrossRef Full Text | Google Scholar

Daruna, J. H., and Barnes, P. A. (1993). “A neurodevelopmental view of impulsivity,” in The Impulsive Client: Theory, Research and Treatment, eds W. G. McCown, J. L. Johnson, and M. B. Shure (Washington, DC: American Psychological Association), 23–37. doi: 10.1037/10500-002

CrossRef Full Text | Google Scholar

Diamond, A. (2013). Executive functions. Annu. Rev. Psychol. 64, 135–168. doi: 10.1146/annurev-psych-113011-143750

CrossRef Full Text | Google Scholar

DuPaul, G. J., Eckert, T. L., and Vilardo, B. (2012). The effects of school-based interventions for attention deficit hyperactivity disorder: a meta-analysis 1996–2010. School Psychol. Rev. 41, 387–412.

Google Scholar

Eriksen, B. A., and Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a non-search task. Percept. Psychophys. 16, 143–149. doi: 10.3758/BF03203267

CrossRef Full Text | Google Scholar

Evans, S. W., Owens, J. S., and Bunford, N. (2014). Evidence-based psychosocial treatments for children and adolescents with attention-deficit/hyperactivity disorder. J. Clin. Child Adolesc. Psychol. 43, 527–551. doi: 10.1080/15374416.2013.850700

PubMed Abstract | CrossRef Full Text | Google Scholar

Fabio, R. A., and Antonietti, A. (2012). Effects of hypermedia instruction on declarative, conditional and procedural learning in ADHD students. Res. Dev. Disabil. 33, 2028–2039. doi: 10.1016/j.ridd.2012.04.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Fang, Y., and Dai Han, H. L. (2019). A virtual reality application for assessment for attention deficit hyperactivity disorder in school-aged children. Neuropsychiatr. Dis. Treat. 15, 1517–1523. doi: 10.2147/NDT.S206742

PubMed Abstract | CrossRef Full Text | Google Scholar

Fernández-Aranda, F., Jiménez-Murcia, S., Santamaría, J. J., Gunnard, K., Soto, A., Kalapanidas, E., et al. (2012). Video games as a complementary therapy tool in mental disorders: playMancer, a European multicentre study. J. Ment. Health 21, 364–374. doi: 10.3109/09638237.2012.664302

PubMed Abstract | CrossRef Full Text | Google Scholar

Homatidis, S., and Konstantareas, M. M. (1981). Assessment of hyperactivity: isolating measures of high discriminant ability. J. Consult. Clin. Psychol. 49:533. doi: 10.1037/0022-006X.49.4.533

PubMed Abstract | CrossRef Full Text | Google Scholar

IBM Corp (2016). IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp.

Kahneman, D., and Tversky, A. (1974). Judgment under uncertainty: heuristics and biases. Science 185, 1124–1131. doi: 10.1126/science.185.4157.1124

PubMed Abstract | CrossRef Full Text | Google Scholar

Kulman, R. (2012). “Generalization of game-based learning for children with ADHD,” in Proceedings of The Annual American Psychological Association Convention (Orlando, FL), 2–5. doi: 10.1037/e636962012-001

CrossRef Full Text

Lappin, J. S., and Eriksen, C. W. (1966). Use of a delayed signal to stop a visual reaction-time response. J. Exp. Psychol. 72, 805–811. doi: 10.1037/h0021266

CrossRef Full Text | Google Scholar

Logan, G. D. (1994). “On the ability to inhibit thought and action: a users' guide to the stop signal paradigm,” in Inhibitory Processes in Attention, Memory, and Language, eds D. Dagenbach and T. H. Carr (New York, NY: Academic Press), 189–239.

PubMed Abstract | Google Scholar

Macmillan, N. A., and Creelman, C. D. (2004). Detection Theory: A User's Guide. Washington, DC: Psychology Press. doi: 10.4324/9781410611147

CrossRef Full Text | Google Scholar

Martinez, L., Prada, E., Satler, C., Tavare, M. C. H., and Tomaz, C. (2016). Executive dysfunctions: the role in attention deficit hyperactivity and post-traumatic stress neuropsychiatric disorders. Front. Psychol. 7:1230. doi: 10.3389/fpsyg.2016.01230

PubMed Abstract | CrossRef Full Text | Google Scholar

Marzocchi, G. M., and Cornoldi, C. (1998). Disturbi di impulsività e ricerca visiva in bambini con deficit di attenzione/iperattività. Psichiatr. Dell'infanz. Dell'adolesc. 65, 173–186.

Google Scholar

Marzocchi, G. M., and Cornoldi, C. (2000). Una scala di facile uso per la rilevazione dei comportamenti problematici dei bambini con deficit di attenzione e iperattività. Psicol. Clin. Dello Sviluppo 4, 43–64. doi: 10.1449/584

CrossRef Full Text | Google Scholar

Marzocchi, G. M., Re, A. M., and Cornoldi, C. (2010). BIA-Batteria Italiana Per l'ADHD. Batteria Per La Valutazione Dei Bambini Con Deficit Di Attenzione/Iperattività. Trento: Edizioni Centro Studi Erickson.

Google Scholar

Milich, R., and Kramer, J. (1984). Reflections on impulsivity: an empirical investigation of impulsivity as a construct. Adv. Learn. Behav. Disabil. 3, 57–94.

Google Scholar

Miyake, A., and Friedman, N. P. (2012). The nature and organization of individual differences in executive functions: four general conclusions. Curr. Dir. Psychol. Sci. 21, 8–14. doi: 10.1177/0963721411429458

PubMed Abstract | CrossRef Full Text | Google Scholar

Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., and Howerter, A. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: a latent variable analysis. Adv. Cogn. Psychol. 41, 49–100. doi: 10.1006/cogp.1999.0734

PubMed Abstract | CrossRef Full Text | Google Scholar

Mühlberger, A., Jekel, K., Probst, T., Schecklmann, M., Conzelmann, A., Andreatta, M., et al. (2020). The influence of methylphenidate on hyperactivity and attention deficits in children with ADHD: a virtual classroom test. J. Atten. Disord. 24, 277–289. doi: 10.1177/1087054716647480

PubMed Abstract | CrossRef Full Text | Google Scholar

Muñoz, J. E., Lopez, D. S., Lopez, J. F., and Lopez, A. (2015). “Design and creation of a BCI videogame to train sustained attention in children with ADHD,” in 2015 10th Computing Colombian Conference (10CCC) (Bogota: IEEE), 194–199. doi: 10.1109/ColumbianCC.2015.7333431

CrossRef Full Text | Google Scholar

Nigg, J. T., Goldsmith, H. H., and Sechek, J. (2004). Temperament and attention deficit hyperactivity disorder: the development of a multiple pathway model. J. Clin. Child Adolesc. Psychol. 33, 42–53. doi: 10.1207/S15374424JCCP3301_5

PubMed Abstract | CrossRef Full Text | Google Scholar

Parsons, T. D., Duffield, T., and Asbee, J. (2019). A comparison of virtual reality classroom continuous performance tests to traditional continuous performance tests in delineating ADHD: a meta-analysis. Neuropsychol. Rev. 29, 338–356. doi: 10.1007/s11065-019-09407-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Powell, L., Parker, J., and Harpin, V. (2017). ADHD: is there an app for that? A suitability assessment of apps for the parents of children and young people with ADHD. JMIR MHealth UHealth 5:e149. doi: 10.2196/mhealth.7941

CrossRef Full Text | Google Scholar

Powell, L., Parker, J., Harpin, V., and Mawson, S. (2019). Guideline development for technological interventions for children and young people to self-manage attention deficit hyperactivity disorder: realist evaluation. J. Med. Internet Res. 21:e12831. doi: 10.2196/12831

PubMed Abstract | CrossRef Full Text | Google Scholar

Prencipe, A., Kesek, A., Cohen, J., Lamm, C., Lewis, M. D., and Zelazo, P. D. (2011). Development of hot and cool executive function during the transition to adolescence. J. Exp. Child Psychol. 108, 621–637. doi: 10.1016/j.jecp.2010.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Rassin, M., Gutman, Y., and Silner, D. (2004). Developing a computer game to prepare children for surgery. AORN J. 80, 1095–1102. doi: 10.1016/S0001-2092(06)60689-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Rizzo, A. A., Bowerly, T., Buckwalter, J. G., Schultheis, M. T., Matheis, R., Shahabi, C., et al. (2002). “Virtual environment for the assessment of attention and memory processes: the virtual classroom and office,” in Proceeding of the 4th International Conference on Disability, Virtual Reality and Associated Technology (Hungary: University of Reading, Vresprem), 3–11.

Google Scholar

Rodríguez, C., García, T., and Areces, D. (2017). New and future challenges concerning the use of virtual reality tools for assessing ADHD. Curr. Dev. Disord. Rep. 4, 8–10. doi: 10.1007/s40474-017-0103-4

CrossRef Full Text | Google Scholar

Simons, L., Valentine, A. Z., Falconer, C. J., Groom, M., Daley, D., and Craven, M. P. (2016). Developing mHealth remote monitoring technology for attention deficit hyperactivity disorder: a qualitative study eliciting user priorities and needs. JMIR MHealth UHealth 23:e31. doi: 10.2196/mhealth.5009

CrossRef Full Text | Google Scholar

Sjöwall, D., Roth, L., Lindqvist, S., and Thorell, L. B. (2013). Multiple deficits in ADHD: executive dysfunction, delay aversion, reaction time variability, and emotional deficits. J. Child Psychol. Psychiatry 54, 619–627. doi: 10.1111/jcpp.12006

PubMed Abstract | CrossRef Full Text | Google Scholar

Smith, E. E., and Jonides, J. (1999). Storage and executive processes in the frontal lobes. Science 283, 1657–1661. doi: 10.1126/science.283.5408.1657

PubMed Abstract | CrossRef Full Text | Google Scholar

Sonuga-Barke, E. J. (1996). When “impulsiveness is delay aversion”: a reply to Schweitzer and Sulzer-Azaroff (1995). J. Child Psychol. Psychiatry 37, 1023–1025. doi: 10.1111/j.1469-7610.1996.tb01499.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Sonuga-Barke, E. J. (2003). The dual pathway model of AD/HD: an elaboration of neurodevelopmental characteristics. Neurosci. Biobehav. Rev. 27, 593–604. doi: 10.1016/j.neubiorev.2003.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Sonuga-Barke, E. J. (2005). Causal models of attention-deficit/hyperactivity disorder: from common simple deficits to multiple developmental pathways. Biol. Psychiatry 57, 1231–1238. doi: 10.1016/j.biopsych.2004.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Sonuga-Barke, E. J., and Taylor, E. (2015). “Disorders of attention activity,” in Rutter's Child and Adolescent Psychiatry, 6th Edn, ed A. Thapar (Oxford: Blackwell), 738–756. doi: 10.1002/9781118381953.ch55

CrossRef Full Text

Stroop, J. R. (1935). Studies of interference in serial verbal reactions. J. Exp. Psychol. 18, 643–662. doi: 10.1037/h0054651

CrossRef Full Text | Google Scholar

Swets, J. A. (1964). Signal Detection and Recognition by Human Observers. New York, NY: Wiley. doi: 10.1037/e444572004-001

CrossRef Full Text | Google Scholar

The jamovi project (2020). jamovi (Version 1.2) [Computer Software]. Retrieved from: https://www.jamovi.org (accessed June 30, 2020).

Van der Oord, S., Ponsioen, A. J. G. B., Geurts, H. M., Ten Brink, E. L., and Prins, P. J. M. (2014). A pilot study of the efficacy of a computerized executive functioning remediation training with game elements for children with ADHD in an outpatient setting: outcome on parent- and teacher-rated executive functioning and ADHD behavior. J. Atten. Disord. 18, 699–712. doi: 10.1177/1087054712453167

PubMed Abstract | CrossRef Full Text | Google Scholar

Walshe, D. G., Lewis, E. J., Kim, S. I., O'Sullivan, K., and Wiederhold, B. K. (2003). Exploring the use of computer games and virtual reality in exposure therapy for fear of driving following a motor vehicle accident. CyberPsychol. Behav. 6, 329–334. doi: 10.1089/109493103322011641

PubMed Abstract | CrossRef Full Text | Google Scholar

Welsh, M. C., Friedman, S. L., and Spieker, S. J. (2006). “Executive functions in developing children: current conceptualizations and questions for the future,” in: Blackwell Handbook of Early Childhood Development, eds K. McCartney and D. Phillips (Blackwell Publishing), 167–187. doi: 10.1002/9780470757703.ch9

CrossRef Full Text | Google Scholar

Wronska, N., Garcia-Zapirain, B., and Mendez-Zorrilla, A. (2015). An iPad-based tool for improving the skills of children with attention deficit disorder. Int. J. Environ. Res. Public Health 12, 6261–6280. doi: 10.3390/ijerph120606261

PubMed Abstract | CrossRef Full Text | Google Scholar