A Systematic Review of Commercial Cognitive Training Devices: Implications for Use in Sport

Background: Cognitive training (CT) aims to develop a range of skills, like attention and decision-making, through targeted training of core cognitive functions. While CT can target context specific skills, like movement anticipation, much CT is domain general, focusing on core abilities (e.g., selective attention) for transfer to a range of real-world tasks, such as spotting opponents. Commercial CT (CCT) devices are highly appealing for athletes and coaches due to their ease of use and eye-catching marketing claims. The extent to which this training transfers to performance in the sporting arena is, however, unclear. Therefore, this paper sought to provide a systematic review of evidence for beneficial training effects of CCT devices and evaluate their application to sport.

Methods: An extensive search of electronic databases (PubMed, PsychInfo, GoogleScholar, and SportDiscus) was conducted to identify peer-reviewed evidence of training interventions with commercially available CT devices. Forty-three studies met the inclusion criteria and were retained for quality assessment and synthesis of results. Seventeen studies assessed transfer effects beyond laboratory cognitive tests, but only 1 directly assessed transfer to a sporting task.

Results: The review of evidence showed limited support for far transfer benefits from CCT devices to sporting tasks, mainly because studies did not target the sporting environment. Additionally, a number of methodological issues with the CCT literature were identified, including small sample sizes, lack of retention tests, and limited replication of findings by researchers independent of the commercial product. Therefore, evidence for sporting benefits is currently limited by the paucity of representative transfer tests and a focus on populations with health conditions.

Conclusions: Currently there is little direct evidence that the use of CCT devices can transfer to benefits for sporting performance. This conclusion, however, stems more from a lack of experimental studies in the sporting field and a lack of experimental rigor, rather than convincing null effects. Subsequently, there is an opportunity for researchers to develop more reliable findings in this area through systematic assessment in athletic populations and major methodological improvements.

Introduction

Rationale

Over the past 10 years, cognitive training (also known as brain training, perceptual training, attention training, or mind training) has boomed, both as a research topic, and as a commercial product. The overall cognitive training (CT) and assessment market is currently worth $1.98 billion (US) and set to rise to over $8 billion by 2021 (marketsandmarkets.com, 2017). Currently, however, it is unclear to what extent device popularity and marketing claims align with scientific evidence. While many commercial CT (CCT) programmes are based on well researched cognitive tasks that have shown trainability (Shipstead et al., 2012; Harrison et al., 2013; Melby-Lervåg and Hulme, 2013), marketing claims suggest more extensive benefits for boosting general brain power and aiding daily mental function (Simons et al., 2016). Additionally, companies cite scientific evidence for their products, which often relates to the basic cognitive tasks rather than direct testing of their device.

CCT devices that allow the user to download an application or log on to a website and immediately begin training can be referred to as “off-the-shelf” devices. They require no instruction or expertise to use, and can often be run on just a mobile phone or computer. Such devices are highly appealing for sport, as they claim to enhance a range of skills, such as attention, speed of processing, decision-making and problem solving, and can be practiced at the athlete's convenience. Given the recent proliferation of these devices and controversy in the academic literature regarding their efficacy, we aim to provide a systematic appraisal of the peer-reviewed evidence for CCT devices. As thesedevices hold particular interest for developing the cognitive skills of athletes, we will also evaluate the evidence for transfer to the sporting domain.

CT consists of systematic practice on tasks intended to develop abilities such as working memory and attention, for transfer to other tasks and settings (Simons et al., 2016). Domain-general CT, which seeks to develop core functions applicable to a multitude of tasks, can be distinguished from context-specific CT such as training perceptual-cognitive abilities using the expert performance approach (Ericsson, 2003), which targets cognitive skills in a specific task (e.g., tennis serve anticipation). Further, the aforementioned commercial devices are distinct from the, often bespoke, methods used exclusively for research (e.g., Jaeggi et al., 2011; Ducrocq et al., 2016, 2017). Here, we are primarily concerned with commercially available methods that aim to enhance domain general abilities. The scientific rationale for CT largely stems from the concept of “neuroplasticity,” which claims that the brain, much like a muscle, can change and adapt to challenges, and that targeted conditioning of a specific region will cause a sustained development in size and/or functional capacity (Draganski et al., 2004). Such adaptation, evident in both young and old (Mahncke et al., 2006; Schlaug et al., 2009), could facilitate a wide range of benefits that are supposedly harnessed through CT, including memory, attention, processing speed, fluid intelligence, problem-solving, and learning abilities (Simons et al., 2016) (see Table 1 for descriptions of cognitive functions). The end goal of CT is to achieve (1) improvements in the cognitive function that was trained (near transfer); (2) improvements in other associated or “overlapping” cognitive functions (e.g., after training working memory, are improvements in attentional control achieved?); and finally, (3) improvements in the performance of tasks in the real world that utilize those cognitive functions (far transfer) (Simons et al., 2016). As such, context general CT relies heavily on the proposition of domain generality; that is, the belief that training-related improvements in domain-specific abilities will transfer onto more general cognitions and skills (Baddeley, 1986; Dahlin et al., 2008).

TABLE 1

Table 1. Description of cognitive functions targeted by CT training devices included in the systematic review.

In order to evaluate the efficacy of current commercially available devices, it is necessary to outline the criteria through which existing research will be appraised. In order to determine causal effects, only studies in which training interventions are used will be considered. Of these, randomized, double-blind clinical trials provide the gold standard. A recent review of CT by Simons et al. (2016) outlines five key questions for assessing the evidence for a training device:

1. Has the training demonstrated transfer of training to other laboratory tasks that measure the same cognitive function as the training?

2. Has the training demonstrated transfer to relevant real-world tasks?

3. Has the training been evaluated using an active control group whose members have the same expectations of training benefits as the members of the experimental group?

4. How long are the trained skills retained?

5. Have the purported benefits of the training been replicated by research groups other than those selling the product?

These questions will be central to our assessment of the current literature on commercial devices. Firstly, the device must demonstrate robust evidence that it does indeed enhance the cognitive function it purports to train, through near transfer to similar tests. If not, subsequent considerations are immaterial. Secondly, and crucially for applications to sport, it must show evidence of transfer to real-world tasks. Thirdly, good experimental design requires the use of active control groups where participants expect a training benefit. Simons and colleagues identify the poor use of control groups in much CT research, where the use of passive controls means that training effects may be due to the expectations of the training group. Fourthly, if CT makes use of “neuroplasticity,” changes in cognitive function in response to training can lead to long term neural changes, which should be retained over time (Park and Bischof, 2013). Finally, much research on commercial devices has been conducted by researchers linked to the companies selling the products. Therefore, in order for research to be considered reliable, the findings should be replicable by researchers independent of the company. These critical questions will be used to identify the strength of evidence for each training device.

CT is typically adopted in the following contexts: (1) compensatory—to overcome or circumnavigate cognitive deficits (Rapport et al., 2013); (2) restorative—to rediscover or restore lost cognitive functions; or (3) additive—to enhance or build upon existing cognitive functions (Ward et al., 2008). Benefits for sport fall into the third context. Currently, however, commercial devices have received little direct testing in athletes or other healthy populations, but considerable testing in older adults and populations with health conditions, where the device aims to overcome deficits in cognitive function. As such, most of the existing findings relate to compensatory or restorative rather than additive ergogenic effects. These findings remain imperative for evaluating the general effectiveness of CT devices, but generalizing to athletes is more difficult. Therefore, reviewed studies will be divided based on the use of young and healthy (additive) versus aged and non-healthy (compensatory/restorative) samples. In doing this, we aim to answer two questions; (1) Is there reliable evidence for any far transfer benefits (all adult populations), following training with CCT devices? (2) Is there reliable evidence for transfer to sporting tasks, following training with CCT devices?

Performing optimally in sport requires a range of cognitive skills, like selective attention (Abernethy, 1987), divided attention (Memmert, 2009) and working memory (Furley and Memmert, 2010), particularly when under pressure (Eysenck and Wilson, 2016). Recent findings suggest that training these functions may transfer to sport, as Ducrocq et al. (2016) demonstrated that training on a bespoke attentional task targeting the inhibition function of working memory improved pressurized volley performance in recreational tennis players. Perceptual-cognitive training, a form of CT that aims to train perceptual and sensory functions responsible for decision-making and anticipatory skills, has also shown cognitive benefits. Typically, life-sized video is used to replicate key situations from the performance environment, enabling trainees to develop the cognitive functions that are utilized in the real world (Williams et al., 2002). This approach has demonstrated benefits for skills like anticipation (see Broadbent et al., 2015 for review). Alternatively, vision training, such as Quiet Eye Training, uses videos of eye movements to teach expert-like gaze strategies to novices. This approach has shown substantial benefits in perceptual-motor as well as perceptual-cognitive tasks (see Vine et al., 2014 for review). Consequently, there is robust evidence for enhancing sporting performance through other methods of cognitive enhancement. The fundamental question is whether these benefits can also be achieved by CCT devices that purport to train domain general abilities (as Jaeggi et al., 2011; Ducrocq et al., 2016), rather than task specific perceptual or attentional abilities?

Research Question

CT can take several forms, based on the purpose of the device and method of training. In particular, commercial devices, like smart phone based braining training games, can be distinguished from non-commercial devices, such as bespoke methods for research (e.g., Jaeggi et al., 2011; Ducrocq et al., 2016, 2017). Additionally, CT can be either truly domain general, or context-specific, such as training of sport specific perceptual-cognitive abilities (Broadbent et al., 2015) and task-specific visual training (Vine et al., 2014). While these methods hold promise for sport, they are highly specialized and often require expert instruction, limiting potential for general usage. Therefore, we aim to review devices that are commercially available for use by a range of sportspeople, and target domain-general skills. CCT devices have the potential to provide an affordable and convenient way of regularly training cognitive skills. This ease of use, in combination with the far-reaching marketing claims, means that CCT devices can be attractive to coaches and athletes. It is currently unclear, however, if these devices can provide reliable transfer to sporting skills. Therefore, we aim to systematically review existing evidence for the use of these devices. Specifically, we firstly assess evidence for performance enhancement across a range of adult populations, and secondly evidence for potential benefits in the sporting arena. We also aim to evaluate study quality to inform future research in this area.

Methods

Search Strategy

The methodology employed for the systematic review was based on the guidelines described by Khan et al. (2003). The aim of the review was to summarize and synthesize peer-reviewed research relating to the effectiveness of CCT devices in adult populations, firstly relating to compensatory/restorative¹ effects, and secondly with regards to potential transfer to sport. Only devices claiming to directly train domain general cognitive function were reviewed. For instance, there is evidence for the beneficial effects of exercise and mindfulness training for cognitive function (Cassilhas et al., 2007; Howells et al., 2016), but our search was restricted to devices specifically designed for CCT. Additionally the search was restricted to studies investigating performance enhancement in adult populations. To this end, an electronic search of PubMed, PsycInfo, GoogleScholar, and SPORTDiscus databases was conducted, for research relating to CCT devices, up to, and including, September 2017. The initial search was performed in PubMed and adapted to the other databases. Key search terms were cognitive, brain, working memory, or attention, combined with training, and excluded titles containing children. Research sections of websites for CCT devices identified in the database search were an additional source of papers. These included the websites for Neurotracker, Cogmed, Cognifit, Lumosity, Posit Science, and Dynavision. Further studies were identified through searching reference lists. The retrieved results were initially assessed for relevance based on their title and abstract, with studies that were ineligible, irrelevant, or duplicates removed. Next the remaining results were screened based on the full-text article, with further ineligible or irrelevant results removed.

Selection of Studies

Included studies were required to meet the following criteria: (1) test a commercially available device, (2) be in a peer-reviewed, English language journal, (3) use adult participants (18+ years of age), (4) use a training intervention (i.e., assigned groups to device practice for any time duration), (5) assess either near or far transfer², and (6) accurately represent the commercial device (i.e., when a device employs multiple subtasks, all tasks were used and training groups did not use more than one CT device).

The identification and selection of papers was guided by the four-phase flow diagram of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA: Figure 1).

FIGURE 1

Figure 1. Four-phase PRISMA flow diagram.

Data Extraction and Quality Assessment

After all articles fitting the search criteria were obtained, they were assessed for quality and key data was extracted for the summary table (Tables 3A,B). Data extraction summarized the following information from each paper: authors; participant population; name of the training device; was an active control group used (if yes, what); was a near transfer test included (yes/no); was a far transfer test included (yes/no); was a retention test used (yes/no); were researchers independent of the company marketing the device (yes/no); which cognitive functions were assessed (Table 1 for descriptions); summary of findings. For consistency, discussion and crosschecking of included studies was carried out amongst the authors.

Study quality was determined by evaluating the internal and external validity of the selected studies. Items for assessing study quality (Appendix 2 in Supplementary Material) were taken from the Quality Index (Downs and Black, 1998), the Epidemiological Appraisal Instrument (Genaidy et al., 2007) and Durant's (1994) checklist for the evaluation of research articles. The five critical questions relating specifically to CT research taken from Simons et al. (2016) review were also included. This formed a 22 item checklist that was scored 1 when a criterion was met and 0 when it was not (or was unknown). This gives a maximum score of 22 for the highest possible quality. The quality assessment was primarily conducted by one author, with queries discussed among the remaining authors (Appendix 1 in Supplementary Material).

Results

Search Results

The initial database searches returned 125,867 papers which, after screening for relevance and matching to inclusion criteria (Figure 1), resulted in 43 papers to be reviewed against the quality assessment criteria (Appendix 2 in Supplementary Material).

Characteristics of Included Studies

The included studies resulted in seven devices for review, these were (with number of studies); Cogmed (15), Lumosity (9), Insight and Brain Fitness by Posit Science (6), Cognifit (4), Neurotracker (4), Nintendo Big Brain Academy and Brain Age (4), and Dynavision (1). The participant samples included populations that were healthy and those with health conditions, 27 with participants from healthy young (<60 years) or old (>60 years) adult groups, four focusing on ADHD, nine on brain injury and cognitive impairments, two on cancer survivors and one on participants living with depression. There was one study with participants from the armed forces and only one in a sporting population.

Twenty-one studies assessed far transfer³, that is, to a measure other than a cognitive test, such as driving ability or soccer passing. Fourteen of these, however, were self-report measures such as quality of life, perceived cognitive function and health condition symptoms. The non-self-report transfer measures were expert ratings of motor skill and safety to drive, ability to perceive human motion, sleep quality, soccer passing ability, and two direct neural measures. Only two studies did not assess near transfer, both of which were studies assessing Neurotracker focusing on far transfer.

Summary of Evidence for CT Devices

An overview of each CCT device is provided in Table 2, and a summary of findings from each study is included in Table 3A (compensatory/restorative effects) and Table 3B (additive effects). Here we give an overview of the evidence for each device, in relation to the five critical questions.

TABLE 2

Table 2. Summary of devices identified in the systematic review.

TABLE 3A

Table 3A. Summary of compensatory and restorative studies (older adults and populations with health conditions).

TABLE 3B

Table 3B. Summary of studies using healthy and elite populations (additive studies).

Cogmed

Cogmed was found to have the most extensive research base with 15 studies matching the criteria, many of which recruited populations with cognitive impairments. Several showed good evidence for near transfer effects, for instance, Åkerlund et al. (2013), Björkdahl et al. (2013), and Dunning and Holmes (2014) all found greater improvement on working memory tasks in the training group than controls. There were, however, null findings regarding working memory improvements in the studies of Gropper et al. (2014), Liu et al. (2016, 2017), and Mawjee et al. (2015). Additionally there were few findings showing improvements in related areas, such as executive function (but cf. Hellgren et al., 2015). With regards to testing far transfer, the Cogmed studies used almost exclusively self-report outcomes, such as quality of life and health condition symptoms. The one exception was Metzler-Baddeley et al. (2016) who found changes in cortical thickness as a result of training. Some of the Cogmed studies provided the best examples of an active control group (Brehmer et al., 2012; Dunning and Holmes, 2014; Metzler-Baddeley et al., 2016), with participants given the same (but non-adaptive) tasks as the trainees. Additionally, two studies assessed skill retention (Brehmer et al., 2012; Gropper et al., 2014) and several of the positive findings came from independent research groups.

Overall, there is good evidence, albeit with some null findings, for near transfer effects following Cogmed training. Some studies also found this to extend to self-rated improvements in everyday life, but there were no studies extending the observed working memory benefits to tasks representative of daily life or sporting activities.

Lumosity

Like Cogmed, several of the nine included Lumosity studies used populations with health conditions (Finn and McDonald, 2011; Charvet et al., 2015; Wentink et al., 2016), but the device has also been tested in healthy populations more relevant to sport. In particular, a large trial of 4,715 participants ranging from 18 to 80 years (Hardy et al., 2015) provides a more generalizable sample. In this study, the training group showed greater improvements than active controls (crossword puzzles) in a range of cognitive tests assessing working memory, executive function, and attention. Across the studies there was good support for the benefits of Lumosity training for near transfer in several cognitive functions, such as speed of processing (Ballesteros et al., 2014), working memory (Hardy et al., 2011), and executive function (Kesler et al., 2013). There were also some null findings for near transfer, but in a small sample (Finn and McDonald, 2011). Regarding far transfer effects, Charvet et al. (2015) found improved motor skill in multiple sclerosis patients, but other studies found no change in self-reported wellbeing (Ballesteros et al., 2014) or mood (Finn and McDonald, 2011).

The study of Hardy et al. (2015) provided easily the largest cohort of the studies in this review, but as participants were already Lumosity users, who were compensated with Lumosity membership, these findings should be viewed with caution. Nevertheless, many of the findings were independent of those manufacturing the product. Overall, despite support for near transfer, there was no evidence of retention, and there has been limited assessment of real-world transfer or additive benefits, the key criteria for generalizing to sport. Findings of improved motor function suggest potential benefits, however this was observed in a population living with multiple sclerosis. Overall there is little evidence that Lumosity training can transfer to tasks beyond the lab.

Posit Science

All six studies meeting the review criteria reported positive effects of Posit Science training for near transfer, mainly in older adults. Improvements in processing speed (Edwards et al., 2013a,b), working memory (Smith et al., 2009; Leung et al., 2015) and short-term memory (Von Ah et al., 2012) were found, predominantly in older adults. Tests of transfer were confined to self-report measures with no real-world tasks relevant to sport, and only weak benefits were found. While Von Ah et al. (2012) found a marginal benefit for perceived cognitive function, Edwards et al. (2013a,b) observed null effects. Several studies used active controls, such as educational training (Smith et al., 2009; Leung et al., 2015), and positive findings have been replicated by independent researchers (Strenziok et al., 2014; Leung et al., 2015). As was the case for most devices, no retention of skills was assessed. Overall, studies supported near transfer effects for compensatory/restorative training, but no evidence for additive effects.

Cognifit

Four studies were identified that directly assessed Cognifit training, across healthy older adults, adults living with intellectual disabilities, adults living with insomnia sufferers and participants living with depression. Three of the four studies found evidence for near transfer benefits (Peretz et al., 2011; Haimov and Shatil, 2013; Siberski et al., 2015), but there were several null effects across these studies and one other showing exclusively null effects (Preiss et al., 2013). The only test of far transfer was self-rated improvement in depressive symptoms (Preiss et al., 2013), which did indicate benefits. These studies generally used appropriate active control groups, principally other computer games (Peretz et al., 2011). Unfortunately there was no test of retention and all studies were conducted by researchers with ties to Cognifit. Overall, the evidence for near transfer effects was relatively weak, and there was no evidence of transfer to tasks representative of sport.

Neurotracker

Four studies investigating Neurotracker were included in the review, although the website lists further studies indicating that Neurotracker ability correlates with sporting (Faubert, 2013; Mangine et al., 2014), driving (Michaels et al., 2017), and surgical (Harenberg et al., 2016) performance. The research base for Neurotracker differs somewhat from those of Cogmed, Lumosity, and Posit Science, which have focused almost exclusively on near transfer effects. Only two of the Neurotracker studies actually tested near transfer effects; Parsons et al. (2016) found improvements in sustained attention, inhibition and working memory following training, while Vartanian et al. (2016) similarly found improvements in several measures of working memory. There is, conversely, more evidence for far transfer effects, and greater use of young and healthy populations, in comparison to other devices.

Firstly, Parsons et al. (2016) found training effects to be accompanied by changes in resting state brain function, primarily decreased theta, alpha, and delta EEG bands in the frontal cortex following 10 training sessions. Secondly, among older adults with impairments in perceiving biological motion, Legault and Faubert (2012) found significant improvements in identifying point light walkers (coordinated moving dots that simulate human motion) at a distance relevant for collision avoidance. Of most relevance for current purposes, is a study by Romeas et al. (2016) which provided the only study in this review to directly test transfer to a sporting task. Romeas et al. (2016) found significant improvements in coach ratings of passing accuracy following Neurotracker training, however, the small sample size (<10 per group) and the null effects for dribbling and shooting should, however, be taken into account. Three of the four studies used appropriate active controls, such as a working memory task, but there was no testing of retention.

Overall, the evidence for far transfer effects and sporting benefits in particular is more promising than most devices. Transfer effects have been found for perception of motion and soccer passing, with EEG suggesting measurable changes in neural activity. Nonetheless the evidence for near transfer is weaker than other devices, and studies have, for the most part, used small samples and been conducted by researchers connected to the company.

Nintendo's Brain Age

Four studies included in the review assessed Nintendo's Brain Age and Big Brain Academy, which provided mixed findings for near transfer effects. Two studies, conducted by Nintendo's researchers, found improvements in executive function, processing speed, and working memory following training, relative to computer game controls (Nouchi et al., 2012, 2013). Conversely, Ackerman et al. (2010) found no benefit to the training group above controls, and McDougall and House (2012) found null effects across most sub-measures of the Wechsler Adult Intelligence Scale. Therefore the evidence for even near transfer effects is weak. Additionally, there are no studies testing far transfer effects of Nintendo's products or retention of abilities. Hence, there was little support for this device and no evidence for sporting transfer.

Dynavision

One study, conducted by independent researchers, was identified that employed a Dynavision training intervention. There is currently little evidence regarding the cognitive functions that are directly targeted by Dynavision as the one included paper inferred improvements in processing speed from the trained task, and did not employ other cognitive measures (Klavora et al., 1995), so there is no evidence of near transfer. There is, however, initial evidence for far transfer, as Klavora et al. (1995) found 10 participants assessed as unsafe to drive following a stroke, to show significantly improved driving ability following training. Unfortunately, this study did not use an active control group, or assess retention of the improvement in driving. Overall the evidence base for this device is weak, as even near transfer to other cognitive tasks is yet to be established and there has been no test of sporting transfer.

Quality Assessment

Scores ranged from 40.9 to 81.8%, with a mean of 62.2% (Appendix 1 in Supplementary Material). Overall, studies scored highly in items relating to the tasks used, basic design, making clear hypotheses, reporting the main findings, assessment tasks, and measuring near transfer. The lowest scoring item was inclusion of a transfer task representative of real-world performance, which was only achieved in four studies. Additionally, only seven studies included justification of sample size, and only eight assessed retention of trained skills. Other issues that were poorly addressed were consistent reporting of effect sizes and the generalizability of findings, due to many studies using niche or non-healthy populations. Eighteen of the 43 studies were carried out by researchers with known connections to the companies.

Discussion

The aim of this systematic review was firstly to evaluate the evidence that currently exists for the effectiveness of CCT devices, and secondly the evidence for transfer to sporting performance. In principle, regular training of key cognitive abilities may hold great value for sporting scenarios, which place high demand on attentional and processing resources, requiring decisions to be made under pressure (Ducrocq et al., 2016). Currently, however, there is a gulf between scientific findings and marketing claims. Therefore, we aimed to provide a rigorous overview of the peer-reviewed evidence for these devices. With regards to our stated aims, the CCT devices showed limited evidence for far transfer effects in general, and evidence of additive effects relevant to sport was particularly scarce, mainly because only one study directly assessed transfer to a sporting task.

Summary of Evidence

The premise of CT is that training of core cognitive abilities will transfer to other tasks and environments. As such, while there was good evidence for near transfer effects in many devices (as has been found in other reviews; Melby-Lervåg and Hulme, 2013), this is not sufficient to conclude overall device effectiveness. Within the compensatory/restorative studies there was limited evidence for far transfer effects beyond the trained tasks, and where transfer tests were used, they often consisted of self-reporting of symptoms. This is a particular problem given the sporadic use of active control groups. Overall, evidence is currently weak for real world benefits from CCT devices, even in deficit populations where we might expect the largest effects. With regards to the narrower focus on potential sporting benefits, the evidence reviewed provides little indication that CCT devices can transfer to the sports field. Firstly, the number of studies using tasks and populations that can be generalized to sport was almost null, with only one study directly using a sporting transfer task. Secondly, the lack of transfer across all populations is not encouraging for athletes who are seeking additive effects. The underwhelming quality of the studies assessed means that positive effects cannot yet be ruled out, but there is little current evidence for them.

Based on the results of the review, the findings relating to Cogmed, Lumosity, Cognifit, and Posit Science⁴ could be grouped together due to similarity of training method and published evidence. These devices use online or app-based games, which closely mimic traditional cognitive tasks, such as memory span and dual load tasks. Their evidence base for near transfer effects is fairly strong, and these devices likely enhance working memory, processing speed, executive function, and attention in laboratory based tasks (Melby-Lervåg and Hulme, 2013). There was, however, very little testing of far transfer effects or retention of trained skills. Whether far transfer tasks have been employed, but remain in the “file drawer” due to null effects, cannot be known. Therefore, these devices hold little promise for benefiting sporting performance.

Outside of this group, Neurotracker provided a training option that included a greater perceptual element and aimed to be more representative of sporting skills. In comparison to other devices, there was relatively little direct testing of near transfer effects, but findings are rather more promising for transfer to real-world tasks. Studies provided initial evidence for enhancing human motion perception (Legault and Faubert, 2012) and soccer passing (Romeas et al., 2016); an indication of far transfer that was absent from the first group of devices. Studies with this device are yet to assess retention effects, following a period without device use. As such further study is required to understand whether beneficial effects rely on persistent use, or can be achieved from a single intervention. In addition, Dynavision training, which similarly included a perceptual element, has been linked to improvements in driving ability (Klavora et al., 1995), but here the evidence was relatively weak. Consequently, while these findings certainly warrant further consideration, firm conclusions cannot yet be drawn as these studies suffer from the same methodological issues discussed previously. In summary, adopting any of the reviewed devices for training athletes would be based on a belief in the principles of domain generality and neuroplasticity rather any conclusive evidence of transfer effects. While these devices may benefit performance in similar, laboratory-based tasks, there is currently weak evidence of their value for sport.

Quality Assessment

Quality assessment scores (Appendix 1 in Supplementary Material) suggest that, overall, the studies in this area display several methodological issues. Some particular concerns include basic experimental design issues like calculation of sample size. A number of the papers reviewed (13) had small samples (<15 per group) with no power analysis as justification. As a result, many of the studies in this area are likely underpowered, meaning the positive findings that do exist have an increased chance of being erroneous (Button et al., 2013). Additionally, many studies included batteries of cognitive tests, which created a multiple testing issue that was, in general, ignored. Preregistration⁵ of planned analyses would be a major step forward in avoiding an ad hoc approach to assessing training effects in this area (Simons et al., 2016).

Methodological choices of the included studies have also limited the conclusions that can be drawn about transfer to sport. In particular the lack of representative real-world tasks and assessment of retention mean that extending findings to sporting scenarios is problematic. Similarly, participant populations often had cognitive deficits, limiting generalizability to healthy populations, where effect sizes may well be smaller. For CCT devices to provide convincing evidence for sporting benefits, these questions must be addressed in future studies.

Future Directions

Future work in this area should focus on the devices that hold the greatest promise for sporting transfer, namely those with a perceptual-cognitive element, more representative of the demands of sport. More studies are required that use athlete populations (rather than cognitively impaired) and test transfer to more representative tasks. Studies must, however, take note of the methodological issues that are prevalent in this area (Simons et al., 2016). As this literature is particularly prone to selective reporting of tests and results, preregistration of accurately powered trials is imperative. The use of adequate active control groups must also be improved, to allow a fair comparison of training effects. CT is an area where much research to date could claim to be “exploratory,” but in order to move toward any kind of reliable evidence, a more systematic approach, which rectifies many of the methodological issues, is required.

Limitations

As with any systematic review, the conclusions must be taken within the context of the search criteria. Other methods of training cognitive function are available, such as transcranial direct stimulation, mindfulness, and exercise interventions. Additionally, amalgamations of interventions were not included, hence the efficacy of combined training strategies cannot be ruled out. There are also a large number of excluded studies which use non-commercially available devices. These studies may report more convincing methods or effects, indeed much working memory training research is more rigorous (see Melby-Lervåg and Hulme, 2013; Ducrocq et al., 2016). We suggest, however, that a focus on commercial devices was warranted given their growing popularity, easy access, endorsements, and the confusion about their effectiveness in the sporting community.

Conclusions

In this systematic review we aimed to evaluate the evidence currently available for CCT devices. Through assessing study quality and synthesizing the available results, it is apparent that there is limited evidence that improvements found in lab-based cognitive tasks transfer to real world benefits. In particular, the very limited use of populations and tasks representative of sport means inferences about CCT effectiveness for athletes are unreliable. Additionally, we identified a series of methodological issues within the CCT literature, such as use of appropriate controls, small sample sizes, lack of retention tests and limited replication of findings by independent researchers. Companies promoting CCT products must address these issues in order to make scientifically valid claims about device effectiveness, while those in the sporting community looking to adopt the use of these products should seek to verify device claims with a healthy degree of skepticism.

Author Contributions

All authors contributed to the review design, search criteria, and writing of the paper. DH and SV conducted the paper search and assessment.

Funding

This research was supported by a grant awarded to SV and MW (TIN 3.215) from the Defence Human Capability Science and Technology Centre, UK.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00709/full#supplementary-material

Footnotes

1. ^Older adult samples were generally classified as compensatory/restorative when participants were over 60 years of age, but this was somewhat guided by the intentions of individual studies.

2. ^We have classified far transfer as any test that was not a laboratory cognitive test. In several cases this included self-report of daily functioning and symptoms of health conditions. Any self-report tests are identified in the results.

3. ^Some papers used “far transfer” to refer to enhancement of a cognitive function that was not directly trained, but here, due to the overlapping nature of concepts like working memory, executive function, and fluid intelligence, we restrict the term to real-world tasks or benefits.

4. ^Nintendo's device could also be included here, but its evidence base is somewhat weaker.

5. ^Recording intended methods and analyses prior to data collection.

References

Abernethy, B. (1987). Selective attention in fast ball sports. II: expert novice differences. Aust. J. Sci. Med. Sport 19, 7–16.

Google Scholar

Ackerman, P. L., Kanfer, R., and Calderwood, C. (2010). Use it or Lose it? Wii brain exercise practice and reading for domain knowledge. Psychol. Aging 25, 753–766. doi: 10.1037/a0019277

PubMed Abstract | CrossRef Full Text | Google Scholar

Åkerlund, E., Esbjörnsson, E., Sunnerhagen, K. S., and Björkdahl, A. (2013). Can computerized working memory training improve impaired working memory, cognition and psychological health? Brain Inj. 27, 1649–1657. doi: 10.3109/02699052.2013.830195

PubMed Abstract | CrossRef Full Text | Google Scholar

Baddeley, A. (1986). Working Memory. Oxford: Oxford University Press.

Google Scholar

Ball, K., Berch, D. B., Helmers, K. F., Jobe, J. B., Leveck, M. D., Marsiske, M., et al. (2002). Effects of cognitive training interventions with older adults: a randomized controlled trial. JAMA 288, 2271–2281. doi: 10.1001/jama.288.18.2271

PubMed Abstract | CrossRef Full Text | Google Scholar

Ballesteros, S., Prieto, A., Mayas, J., Toril, P., Pita, C., Ponce de León, L., et al. (2014). Brain training with non-action video games enhances aspects of cognition in older adults: a randomized controlled trial. Front. Aging Neurosci. 6:277. doi: 10.3389/fnagi.2014.00277

PubMed Abstract | CrossRef Full Text | Google Scholar

Björkdahl, A., Åkerlund, E., Svensson, S., and Esbjörnsson, E. (2013). A randomized study of computerized working memory training and effects on functioning in everyday life for patients with brain injury. Brain Inj. 27, 1658–1665. doi: 10.3109/02699052.2013.830196

PubMed Abstract | CrossRef Full Text | Google Scholar

Brehmer, Y., Rieckmann, A., Bellander, M., Westerberg, H., Fischer, H., and Bäckman, L. (2011). Neural correlates of training-related working-memory gains in old age. Neuroimage 58, 1110–1120. doi: 10.1016/j.neuroimage.2011.06.079

PubMed Abstract | CrossRef Full Text | Google Scholar

Brehmer, Y., Westerberg, H., and Bäckman, L. (2012). Working-memory training in younger and older adults: training gains, transfer, and maintenance. Front. Hum. Neurosci. 6:63. doi: 10.3389/fnhum.2012.00063

PubMed Abstract | CrossRef Full Text | Google Scholar

Broadbent, D. P., Causer, J., Williams, A. M., and Ford, P. R. (2015). Perceptual-cognitive skill training and its transfer to expert performance in the field: Future research directions. Eur. J. Sport Sci. 15, 322–331. doi: 10.1080/17461391.2014.957727

PubMed Abstract | CrossRef Full Text | Google Scholar

Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., et al. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365–376. doi: 10.1038/nrn3475

PubMed Abstract | CrossRef Full Text | Google Scholar

Cassilhas, R. C., Viana, V. A. R., Grassmann, V., Santos, R. T., Santos, R. F., Tufik, S., et al. (2007). The impact of resistance exercise on the cognitive function of the elderly. Med. Sci. Sports Exerc. 39, 1401–1407. doi: 10.1249/mss.0b013e318060111f

PubMed Abstract | CrossRef Full Text | Google Scholar

Charvet, L., Shaw, M., Haider, L., Melville, P., and Krupp, L. (2015). Remotely-delivered cognitive remediation in multiple sclerosis (MS): protocol and results from a pilot study. Mult. Scler. J. Exp. Transl. Clin. 1:2055217315609629. doi: 10.1177/2055217315609629

PubMed Abstract | CrossRef Full Text | Google Scholar

Dahlin, E., Neely, A. S., Larsson, A., Backman, L., and Nyberg, L. (2008). Transfer of learning after updating training mediated by the striatum. Science 320, 1510–1512. doi: 10.1126/science.1155466

PubMed Abstract | CrossRef Full Text | Google Scholar

Downs, S. H., and Black, N. (1998). The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J. Epidemiol. Commun. Health 52, 377–384.

PubMed Abstract | Google Scholar

Draganski, B., Gaser, C., Busch, V., Schuierer, G., Bogdahn, U., and May, A. (2004). Neuroplasticity: changes in grey matter induced by training. Nature 427, 311–312. doi: 10.1038/427311a

PubMed Abstract | CrossRef Full Text | Google Scholar

Ducrocq, E., Wilson, M., Smith, T. J., and Derakshan, N. (2017). Adaptive working memory training reduces the negative impact of anxiety on competitive motor performance. J. Sport Exerc. Psychol. 39, 412–422. doi: 10.1123/jsep.2017-0217

PubMed Abstract | CrossRef Full Text | Google Scholar

Ducrocq, E., Wilson, M., Vine, S., and Derakshan, N. (2016). Training attentional control improves cognitive and motor task performance. J. Sport Exerc. Psychol. 38, 521–533. doi: 10.1123/jsep.2016-0052

PubMed Abstract | CrossRef Full Text | Google Scholar

Dunning, D. L., and Holmes, J. (2014). Does working memory training promote the use of strategies on untrained working memory tasks? Mem. Cogn. 42, 854–862. doi: 10.3758/s13421-014-0410-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Dunning, D. L., Holmes, J., and Gathercole, S. E. (2013). Does working memory training lead to generalized improvements in children with low working memory? A randomized controlled trial. Dev. Sci. 16, 915–925. doi: 10.1111/desc.12068

PubMed Abstract | CrossRef Full Text | Google Scholar

Durant, R. H. (1994). Checklist for the evaluation of research articles. J. Adolesc. Health 15, 4–8.

PubMed Abstract | Google Scholar

Edwards, J. D., Hauser, R. A., O'Connor, M. L., Valdés, E. G., Zesiewicz, T. A., and Uc, E. Y. (2013a). Randomized trial of cognitive speed of processing training in Parkinson disease. Neurology 81, 1284–1290. doi: 10.1212/WNL.0b013e3182a823ba

PubMed Abstract | CrossRef Full Text | Google Scholar

Edwards, J. D., Valdés, E. G., Peronto, C., Castora-Binkley, M., Alwerdt, J., Andel, R., et al. (2013b). The Efficacy of InSight cognitive training to improve useful field of view performance: a brief report. J. Gerontol. Ser. B 70, 417–422. doi: 10.1093/geronb/gbt113

PubMed Abstract | CrossRef Full Text | Google Scholar

Ericsson, K. A. (2003). “How the expert performance approach differs from traditional approaches to expertise in sport: in search of a shared theoretical framework for studying expert performance,” in Expert Performance in Sports: Advances in Research on Sport Expertise, eds J. L. Starkes and K. A. Ericsson (Champaign: Human Kinetics), 371–402.

Google Scholar

Eysenck, M. W., and Wilson, M. R. (2016). “Sporting performance, pressure and cognition: introducing attentional control theory: sport,” in An Introduction to Applied Cognitive Psychology, eds D. Groome and M. Eysenck (London: Routledge), 329–350.

Faubert, J. (2013). Professional athletes have extraordinary skills for rapidly learning complex and neutral dynamic visual scenes. Sci. Rep. 3:1154. doi: 10.1038/srep01154

PubMed Abstract | CrossRef Full Text | Google Scholar

Finn, M., and McDonald, S. (2011). Computerised cognitive training for older persons with mild cognitive impairment: a pilot study using a randomised controlled trial design. Brain Impair. 12, 187–199. doi: 10.1375/brim.12.3.187

CrossRef Full Text | Google Scholar

Furley, P. A., and Memmert, D. (2010). The role of working memory in sport. Int. Rev. Sport Exerc. Psychol. 3, 171–194. doi: 10.1080/1750984X.2010.526238

CrossRef Full Text | Google Scholar

Genaidy, A. M., LeMasters, G. K., Lockey, J., Succop, P., Deddens, J., Sobeih, T., et al. (2007). An epidemiological appraisal instrument – a tool for evaluation of epidemiological studies. Ergonomics 50, 920–960. doi: 10.1080/00140130701237667

PubMed Abstract | CrossRef Full Text | Google Scholar

Gibson, B. S., Gondoli, D. M., Kronenberger, W. G., Johnson, A. C., Steeger, C. M., and Morrissey, R. A. (2013). Exploration of an adaptive training regimen that can target the secondary memory component of working memory capacity. Mem. Cognit. 41, 726–737. doi: 10.3758/s13421-013-0295-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Gropper, R. J., Gotlieb, H., Kronitz, R., and Tannock, R. (2014). Working memory training in college students with ADHD or LD. J. Atten. Disord. 18, 331–345. doi: 10.1177/1087054713516490

CrossRef Full Text | Google Scholar

Haimov, I., and Shatil, E. (2013). Cognitive training improves sleep quality and cognitive function among older adults with insomnia. PLoS ONE 8:e61390. doi: 10.1371/journal.pone.0061390

PubMed Abstract | CrossRef Full Text | Google Scholar

Hardy, J. L., Drescher, D., Sarkar, K., Kellett, G., and Scanlon, M. (2011). Enhancing visual attention and working memory with a web-based cognitive training program. Mensa Res. J. 42, 13–20.

Hardy, J. L., Nelson, R. A., Thomason, M. E., Sternberg, D. A., Katovich, K., Farzin, F., et al. (2015). Enhancing cognitive abilities with comprehensive training: a large, online, randomized, active-controlled trial. PLoS ONE 10:e0134467. doi: 10.1371/journal.pone.0134467

PubMed Abstract | CrossRef Full Text | Google Scholar

Harenberg, S., McCaffrey, R., Butz, M., Post, D., Howlett, J., Dorsch, K. D., et al. (2016). Can multiple object tracking predict laparoscopic surgical skills? J. Surg. Educ. 73, 386–390. doi: 10.1016/j.jsurg.2015.11.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Harrison, T. L., Shipstead, Z., Hicks, K. L., Hambrick, D. Z., Redick, T. S., and Engle, R. W. (2013). Working memory training may increase working memory capacity but not fluid intelligence. Psychol. Sci. 24, 2409–2419. doi: 10.1177/0956797613492984

CrossRef Full Text | Google Scholar

Hellgren, L., Samuelsson, K., Lundqvist, A., and Börsbo, B. (2015). Computerized training of working memory for patients with acquired brain injury. Open J. Ther. Rehabil. 3, 46–55. doi: 10.4236/ojtr.2015.32007

CrossRef Full Text

Howells, A., Ivtzan, I., and Eiroa-Orosa, F. J. (2016). Putting the ‘app’ in happiness: a randomised controlled trial of a smartphone-based mindfulness intervention to enhance wellbeing. J. Happiness Stud. 17, 163–185. doi: 10.1007/s10902-014-9589-1

CrossRef Full Text | Google Scholar

Hyer, L., Scott, C., Atkinson, M. M., Mullen, C. M., Lee, A., Johnson, A., et al. (2016). Cognitive training program to improve working memory in older adults with MCI. Clin. Gerontol. 39, 410–427. doi: 10.1080/07317115.2015.1120257

PubMed Abstract | CrossRef Full Text | Google Scholar

Jaeggi, S. M., Buschkuehl, M., Jonides, J., and Shah, P. (2011). Short-and long-term benefits of cognitive training. Proc. Natl. Acad. Sci. U.S.A. 108, 10081–10086. doi: 10.1073/pnas.1103228108

PubMed Abstract | CrossRef Full Text | Google Scholar

Kesler, S., Hadi Hosseini, S. M., Heckler, C., Janelsins, M., Palesh, O., Mustian, K., et al. (2013). Cognitive training for improving executive function in chemotherapy-treated breast cancer survivors. Clin. Breast Cancer 13, 299–306. doi: 10.1016/j.clbc.2013.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Khan, K. S., Kunz, R., Kleijnen, J., and Antes, G. (2003). Five steps to conducting a systematic review. J. R. Soc. Med. 96, 118–121. doi: 10.1258/jrsm.96.3.118

PubMed Abstract | CrossRef Full Text | Google Scholar

Klavora, P., Gaskovski, P., Martin, K., Forsyth, R. D., Heslegrave, R. J., Young, M., et al. (1995). The effects of Dynavision rehabilitation on behind-the-wheel driving ability and selected psychomotor abilities of persons after stroke. Am. J. Occup. Ther. 49, 534–542.

PubMed Abstract | Google Scholar

Legault, I., and Faubert, J. (2012). Perceptual-cognitive training improves biological motion perception: evidence for transferability of training in healthy aging. Neuroreport 23, 469–473. doi: 10.1097/WNR.0b013e328353e48a

PubMed Abstract | CrossRef Full Text | Google Scholar

Leung, N. T., Tam, H. M., Chu, L. W., Kwok, T. C., Chan, F., Lam, L. C. W., et al. (2015). Neural plastic effects of cognitive training on aging brain. Neural Plast. 2015:535618. doi: 10.1155/2015/535618

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, Z.-X., Glizer, D., Tannock, R., and Woltering, S. (2016). EEG alpha power during maintenance of information in working memory in adults with ADHD and its plasticity due to working memory training: a randomized controlled trial. Clin. Neurophysiol. 127, 1307–1320. doi: 10.1016/j.clinph.2015.10.032

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, Z.-X., Lishak, V., Tannock, R., and Woltering, S. (2017). Effects of working memory training on neural correlates of Go/Nogo response control in adults with ADHD: a randomized controlled trial. Neuropsychologia 95, 54–72. doi: 10.1016/j.neuropsychologia.2016.11.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Lundqvist, A., Grundström, K., Samuelsson, K., and Rönnberg, J. (2010). Computerized training of working memory in a group of patients suffering from acquired brain injury. Brain Inj. 24, 1173–1183. doi: 10.3109/02699052.2010.498007

PubMed Abstract | CrossRef Full Text | Google Scholar

Mahncke, H. W., Connor, B. B., Appelman, J., Ahsanuddin, O. N., Hardy, J. L., Wood, R. A., et al. (2006). Memory enhancement in healthy older adults using a brain plasticity-based training program: a randomized, controlled study. Proc. Natl. Acad. Sci. U.S.A. 103, 12523–12528. doi: 10.1073/pnas.0605194103

PubMed Abstract | CrossRef Full Text | Google Scholar

Mangine, G. T., Hoffman, J. R., Wells, A. J., Gonzalez, A. M., Rogowski, J. P., Townsend, J. R., et al. (2014). Visual tracking speed is related to basketball-specific measures of performance in NBA Players. J. Strength Cond. Res. 28, 2406–2414.

Google Scholar

marketsandmarkets.com, (2017). Cognitive Assessment and Training Market Worth 8.06 Billion USD by 2021. [ONLINE] Available online at: http://www.marketsandmarkets.com/PressReleases/cognitive-assessment.asp (Accessed September 12, 2017).

Mawjee, K., Woltering, S., and Tannock, R. (2015). Working memory training in post- secondary students with ADHD: a randomized controlled study. PLoS ONE 10:e0137173. doi: 10.1371/journal.pone.0137173

PubMed Abstract | CrossRef Full Text | Google Scholar

Mayas, J., Parmentier, F. B. R., Andrés, P., and Ballesteros, S. (2014). Plasticity of attentional functions in older adults after non-action video game training: a randomized controlled trial. PLoS ONE 9:e92269. doi: 10.1371/journal.pone.0092269

PubMed Abstract | CrossRef Full Text | Google Scholar

McDougall, S., and House, B. (2012). Brain training in older adults: evidence of transfer to memory span performance and pseudo-Matthew effects. Aging Neuropsychol. Cogn. 19, 195–221. doi: 10.1080/13825585.2011.640656

PubMed Abstract | CrossRef Full Text | Google Scholar

McNab, F., Varrone, A., Farde, L., Jucaite, A., Bystritsky, P., Forssberg, H., et al. (2009). Changes in cortical dopamine D1 receptor binding associated with cognitive training. Science 323, 800–802. doi: 10.1126/science.1166102

PubMed Abstract | CrossRef Full Text | Google Scholar

Melby-Lervåg, M., and Hulme, C. (2013). Is working memory training effective? A meta- analytic review. Dev. Psychol. 49, 270–291. doi: 10.1037/a0028228

PubMed Abstract | CrossRef Full Text | Google Scholar

Memmert, D. (2009). Pay attention! A review of visual attentional expertise in sport. Int. Rev. Sport Exerc. Psychol. 2, 119–138. doi: 10.1080/17509840802641372

CrossRef Full Text | Google Scholar

Metzler-Baddeley, C., Caeyenberghs, K., Foley, S., and Jones, D. K. (2016). Task complexity and location specific changes of cortical thickness in executive and salience networks after working memory training. Neuroimage 130, 48–62. doi: 10.1016/j.neuroimage.2016.01.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Michaels, J., Chaumillon, R., Nguyen-Tri, D., Watanabe, D., Hirsch, P., Bellavance, F., et al. (2017). Driving simulator scenarios and measures to faithfully evaluate risky driving behavior: a comparative study of different driver age groups. PLoS ONE 12:e0185909. doi: 10.1371/journal.pone.0185909

PubMed Abstract | CrossRef Full Text | Google Scholar

Nouchi, R., Taki, Y., Takeuchi, H., Hashizume, H., Akitsuki, Y., Shigemune, Y., et al. (2012). Brain training game improves executive functions and processing speed in the elderly: a randomized controlled trial. PLoS ONE 7:e29676. doi: 10.1371/journal.pone.0029676

PubMed Abstract | CrossRef Full Text | Google Scholar

Nouchi, R., Taki, Y., Takeuchi, H., Hashizume, H., Nozawa, T., Kambara, T., et al. (2013). Brain training game boosts executive functions, working memory and processing speed in young adults: a randomized controlled trial. PLoS ONE 8:e55518. doi: 10.1371/journal.pone.0055518

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, D. C., and Bischof, G. N. (2013). The aging mind: neuroplasticity in response to cognitive training. Dialogues Clin. Neurosci. 15, 109–119.

PubMed Abstract | Google Scholar

Parsons, B., Magill, T., Boucher, A., Zhang, M., Zogbo, K., Bérubé, S., et al. (2016). Enhancing cognitive function using perceptual-cognitive training. Clin. EEG Neurosci. 47, 37–47. doi: 10.1177/1550059414563746

PubMed Abstract | CrossRef Full Text | Google Scholar

Peretz, C., Korczyn, A. D., Shatil, E., Aharonson, V., Birnboim, S., and Giladi, N. (2011). Computer-based, personalized cognitive training versus classical computer games: a randomized double-blind prospective trial of cognitive stimulation. Neuroepidemiology 36, 91–99. doi: 10.1159/000323950

PubMed Abstract | CrossRef Full Text | Google Scholar

Preiss, M., Shatil, E., Cermáková, R., Cimermanová, D., and Ram, I. (2013). Personalized cognitive training in unipolar and bipolar disorder: a study of cognitive functioning. Front. Hum. Neurosci. 7:108. doi: 10.3389/fnhum.2013.00108

PubMed Abstract | CrossRef Full Text | Google Scholar

Pylyshyn, Z. W., and Storm, R. W. (1988). Tracking multiple independent targets: evidence for a parallel tracking mechanism. Spat. Vis. 3, 179–197.

PubMed Abstract | Google Scholar

Rapport, M. D., Orban, S. A., Kofler, M. J., and Friedman, L. M. (2013). Do programs designed to train working memory, other executive functions, and attention benefit children with ADHD? A meta-analytic review of cognitive, academic, and behavioral outcomes. Clin. Psychol. Rev. 33, 1237–1252. doi: 10.1016/j.cpr.2013.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Rass, O., Schacht, R. L., Buckheit, K., Johnson, M. W., Strain, E. C., and Mintzer, M. Z. (2015). A randomized controlled trial of the effects of working memory training in methadone maintenance patients. Drug Alcohol Depend. 156, 38–46. doi: 10.1016/j.drugalcdep.2015.08.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Romeas, T., Guldner, A., and Faubert, J. (2016). 3D-Multiple Object Tracking training task improves passing decision-making accuracy in soccer players. Psychol. Sport Exerc. 22, 1–9. doi: 10.1016/j.psychsport.2015.06.002

CrossRef Full Text | Google Scholar

Schlaug, G., Forgeard, M., Zhu, L., Norton, A., Norton, A., and Winner, E. (2009). Training- induced neuroplasticity in young children. Ann. N.Y. Acad. Sci. 1169, 205–208. doi: 10.1111/j.1749-6632.2009.04842.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Shipstead, Z., Redick, T. S., and Engle, R. W. (2012). Is working memory training effective? Psychol. Bull. 138, 628–654. doi: 10.1037/a0027473

PubMed Abstract | CrossRef Full Text | Google Scholar

Siberski, J., Shatil, E., Siberski, C., Eckroth-Bucher, M., French, A., Horton, S., et al. (2015). Computer-based cognitive training for individuals with intellectual and developmental disabilities: pilot study. Am. J. Alzheimers Dis. Dementiasr. 30, 41–48. doi: 10.1177/1533317514539376

PubMed Abstract | CrossRef Full Text | Google Scholar

Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., et al. (2016). Do “brain-training” programs work? Psychol. Sci. Public Interest 17, 103–186. doi: 10.1177/1529100616661983

PubMed Abstract | CrossRef Full Text | Google Scholar

Smith, G. E., Housen, P., Yaffe, K., Ruff, R., Kennison, R. F., Mahncke, H. W., et al. (2009). A cognitive training program based on principles of brain plasticity: results from the improvement in memory with plasticity-based adaptive cognitive training (IMPACT) study. J. Am. Geriatr. Soc. 57, 594–603. doi: 10.1111/j.1532-5415.2008.02167.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Strenziok, M., Parasuraman, R., Clarke, E., Cisler, D. S., Thompson, J. C., and Greenwood, P. M. (2014). Neurocognitive enhancement in older adults: comparison of three cognitive training tasks to test a hypothesis of training transfer in brain connectivity. Neuroimage 85, 1027–1039. doi: 10.1016/j.neuroimage.2013.07.069

PubMed Abstract | CrossRef Full Text | Google Scholar

Vartanian, O., Coady, L., and Blackler, K. (2016). 3d multiple object tracking boosts working memory span: Implications for cognitive training in military populations. Mil. Psychol. 28, 353–360. doi: 10.1037/mil0000125

CrossRef Full Text | Google Scholar

Vine, S. J., Moore, L. J., and Wilson, M. R. (2014). Quiet eye training: The acquisition, refinement and resilient performance of targeting skills. Eur. J. Sport Sci. 14, S235–S242. doi: 10.1080/17461391.2012.683815

PubMed Abstract | CrossRef Full Text | Google Scholar

Von Ah, D., Carpenter, J. S., Saykin, A., Monahan, P., Wu, J., Yu, M., et al. (2012). Advanced cognitive training for breast cancer survivors: a randomized controlled trial. Breast Cancer Res. Treat. 135, 799–809. doi: 10.1007/s10549-012-2210-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Ward, P., Farrow, D., Harris, K. R., Williams, M. A., Eccles, D. W., and Ericsson, A. K. (2008). Training perceptual-cognitive skills: can sport psychology research inform military decision training? Mil. Psychol. 20, 71–102. doi: 10.1080/08995600701804814

CrossRef Full Text | Google Scholar

Wentink, M. M., Berger, M. A. M., Kloet, A. J., de Meesters, J., Band, G. P. H., Wolterbeek, R., et al. (2016). The effects of an 8-week computer- based brain training programme on cognitive functioning, QoL and self-efficacy after stroke. Neuropsychol. Rehabil. 26, 847–865. doi: 10.1080/09602011.2016.1162175

PubMed Abstract | CrossRef Full Text | Google Scholar

Williams, A. M., Ward, P., Knowles, J. M., and Smeeton, N. J. (2002). Anticipation skill in a real- world task: measurement, training, and transfer in tennis. J. Exp. Psychol. Appl. 8, 259–270. doi: 10.1037//1076-898X.8.4.259

PubMed Abstract | CrossRef Full Text | Google Scholar