Metacognitive Treatment in Acquired Brain Injury and Its Applicability to Aphasia: A Systematic Review

Purpose: The purpose of this systematic review is to identify the utility of metacognitive therapeutic intervention for persons with acquired brain injury (ABI), with a focus on persons with aphasia.

Methods: A search of six databases resulted in two hundred and sixty-six unique manuscripts relating to the explicit use of metacognitive treatment for people with ABI. Two independent reviewers rated abstracts for inclusion or exclusion of the study given predetermined criteria. Twenty-nine articles, five of which included people with aphasia, were selected for inclusion in this systematic review. SCED+ and PEDro+ rating scales were used to rate the methodological quality of each study.

Results: Methodological quality of the 29 studies that met inclusion criteria ranged from weak to high quality studies. Three -hundred and sixty-nine individuals with ABI took part in the 29 studies. Varying treatment methods were employed. Outcome measures were inconsistent. Metacognitive treatment has been applied to people with aphasia with positive results, but efficacy of the treatment cannot yet be determined.

Conclusions: Metacognitive therapeutic intervention tends to be effective for persons with acquired brain injury (ABI) despite variability between intervention designs and treatment outcomes across studies. Due to so few studies with participants with aphasia, we were unable to draw conclusions regarding the efficacy of metacognitive treatment for people with aphasia. Further research on the efficacy of metacognitive treatment for this population is warranted.

Introduction

Metacognition is self-regulated insight into one's own thinking. It enables analysis and adjustments to be made in response to active behavioral performance as well as to changes in internal states. Metacognitive skills comprise two dynamic facets: metacognitive knowledge and online awareness (1). The former refers to judgement and understanding of one's ability to complete a task, whereas the latter is active engagement when carrying out a task (1). Metacognitive deficits are common following acquired brain injury (ABI), altering behavioral performance and negatively influencing safe engagement in independent activties of daily-living (IADLs) (2, 3).

Anosognosia, an unawareness of deficit (4), is often researched separately from metacognition. Though similar, the theoretical construct of the two were developed independently from one another (5). Sunderaraman and Cosentino (4) posit that metacognition explains more of the cognitive construct of unawareness, where anosognosia describes the clinical construct. If we were to fit anosognosia into the model of metacognition, it would be synonymous with the metacognitive knowledge facet (6). In this paper, we focus our research on the more all-ecompassing metacognition rather than anosognosia. The purpose of this systematic review is to identify the utility and effectiveness of metacognitive therapeutic interventions with individuals with ABIs including persons with aphasia.

In regards to typical aging, there is evidence that the dorsal and ventral white matter tracts atrophy, affecting both language and cognitive processing (7). There is evidence that the effect is more prominent in fluid cognition (ie. executive function or working memory) than crystallized cognitive abilities [recalling stored knowledge or past experiences; (8–10)]. That is, one will see less of an effect on tasks that rely on existent knowledge as compared to new tasks that rely on learning (10). Individuals with ABI rely on new learning; this finding implies increased difficulty in rehabilitation of older individuals with ABI. Metacognitive skills in older adults, which may be affected as a result of aging, can help account for the cognitive declines observed with age (11). Metacognitive training used to increase new learning and overall cognitive functioning in typical older adults has proven successful, which provides evidence that metacognitive rehabilitation for individuals with ABI may be successful as well.

A brain injury often times leads to cognitive deficits beyond aging in those affected. Following the brain injury, individuals remember themselves prior to the brain injury and may not grasp changed status in cognition. Metacognitive deficits following injury, in part, reflect a failure to update this knowledge in response to injury (i.e., recognition of current level of functioning) (1). Reduced ability to regulate these processes results in reduced success with completing tasks and can trigger implementation of maladaptive strategies. A failure to update knowledge results in overestimating performance abilty, which can lead to a sense of loss of control, depression, and isolation (1). In order to increase awareness of deficits, metacognitive knowledge, and online processing must be rehabilitated with individuals receiving care responsonible for identifying errors as they occur (1).

Metacognitive deficits are a common sequelae of ABI and reflect altered executive processes. Common areas of impairment include: initiation, flexibility, problem solving, self-monitoring, and self-regulation (12–14). Executive dysfunction can lead to significant life challenges including an inability to identify goals, pursue goals, apply learned strategies to different situations, and function independently within daily environment (12, 15, 16). Furthermore, deficits within these areas are also observed in language processing such as an inability to plan what to say, decreased success in delivering the message, and reduced ability with respect to inhibiting unwanted responses (17–19).

Language

Aphasia, a common secondary result of ABI, is a multimodal language disorder in which the manipulation, comprehension and formulation of linguistic symbols and elements present as the prominent deficit in individuals affected (20–22). Though the primary deficit in aphasia is language, researchers have also identified concomitant impairments in working memory, self-regulation, attention, and executive function (23–25). Attentional skills are integrated in different stages of word production tasks including phonological encoding and lexical retrieval, and attentional skills required for these tasks can be affected in persons with aphasia (26, 27).

Types of aphasia can be broadly categorized into fluent and nonfluent aphasia; though metacognitive skills may be disrupted differently in each, lack of awareness may be present. Levelt et al. (28) proposed the perceptual loop hypothesis, stating that language output was monitored by one's comprehension of language. Therefore, if one does not comprehend errors, they would not be able to recognize and correct errors. The theory accounts for decreased metacognitive skills in those with fluent aphasia. Contrast, metacognitive awareness is postulated to be a conscious experience relying on both attention and executive function (29). Since these cognitive skills rely on the integrity and connections within the frontal cortex (17, 24), awareness in those with nonfluent aphasia may be affected. Evidence shows cognitive control and monitoring are important for word selection tasks and may be interrupted due to infarcts linked with aphasia (30). Moreover, performance on measures of awareness do correlate with severity of language impairment (6). Enhancing metacognitive skills is therefore likely to aid general cognitive functioning, subsequently bolstering linguistic performance.

Rehabilitation

Metacognitive Strategy Instruction (MSI) involves training individuals to increase self-awareness of their strengths and weaknesses, thereby increasing their independence in completing everyday tasks. When describing treatment protocols that promote self-awareness, (31) state that self awareness retraining, “promote(s) internalization of self-regulation strategies through self-instruction and self-monitoring as a practice option” (p. 1688). In theory, once one is aware of strengths and weakness, they will be able to allocate resources where necessary (be it language or underlying cognitive deficits), thus increasing overall functioning. Another avenue for metacognitive training is a focus of error awareness throughout strategy training. The focus of this type of metacognitive training is the ability to recognize errors throughout completion of a task (14, 32). Once one has a heightened sense of awareness, they will be able to self-correct errors during language output.

Metacognitive therapy is used with individuals with traumatic brain injury (TBI) to increase self-awareness, self-reliance, and overall independence (33, 34). In this population metacognitive strategies commonly consist of breaking down goals into manageable steps, learning to change behavior to reach desired goal, and carrying out the change in behavior (35). Kennedy et al. (35) completed a meta-analysis evaluating therapeutic interventions for problem-solving, planning, organization, and multi-tasking in persons with TBI. The therapy dosage varied with each study, but all resulted in positive outcomes within a week post-treatment in various cognitive tasks with a trend of positive maintenance and generalization outcomes. The authors concluded that utilizing MSI for persons with TBI increased overall problem-solving skills.

Recent literature has emerged regarding the use of metacognitive therapy with people with aphasia (18, 25, 36–38). When provided during rehabilitation of cognitive-linguistic skills, metacognitive therapies are intended to enhance self-awareness and promote greater cognitive understanding and control during IADLs. Targeted skill sets include: the ability to set goals, evaluate performance throughout a task in relation to goals, decide how to change behavior in order to meet goals, and how to apply behaviors to new strategies in order to reach the desired outcome (39).

Aims

The purpose of this systematic review is to identify the therapeutic effect of using metacognitive intervention for individuals with ABI, including persons with aphasia. There are four research objectives: (1) Describe and appraise the studies and the methodological quality of the studies reviewed (2) investigate whether metacognitive interventions result in positive outcomes (cognitive, language, social) for persons with ABI (3) determine whether there is a specific type of metacognitive intervention that is more widely utilized for individuals with ABI within the research literature (4) explore whether metacognitive intervention is or has the potential to be effective for persons with aphasia given extent and quality of the current literature.

Methods

Selection of Articles

Articles were selected from six electronic databases, including PubMed, Scopus, Linguistics and Language Behavior Abstracts (LLBA), American Speech Language and Hearing Association Journals (ASHA Journals), PsychInfo and ProQuest. An initial search of the databases was completed June 2018, with an updated search completed October 2019. Reference lists of identified studies were reviewed to identify studies that did not show up in the database search. Preferred Reporting Items for Systematic Reviews and Meta Analysis Guidelines [PRISMA, (40)] was employed. Keywords were: “metacognitive”; “online awareness” AND “treatment”; “intervention”; “rehabilitation” AND “aphasia”; “acquired brain injury”; “stroke”. Deduplication and screening were performed manually.

Eligibility Criteria

In order to identify research articles appropriate for this systematic review, parameters for inclusion and exclusion criteria were set and included: full-text, peer-reviewed journal article in English, describing a completed metacognitive behavioral treatment published; original data from the study had to be reported; participants of interest were adults, over the age of 18, with a history of ABI, including penetrating head injury traumatic brain injury (PHI TBI), closed-head injury traumatic brain injury (CHI TBI), hypoxia, CVA, tumor, anoxia, arterial venous malformation, encephalitis or aneurysm.

We included only manuscripts that used treatments specifically designed to increase aspects of metacognition in participants, such as error detection, self-awareness, online awareness and the ability to identify and carry out appropriate compensatory strategies during a given task. Studies that sought to identify metacogntive deficits but did not explicity treat metacognition either directly or indirectly, were excluded. In this case, the researchers were looking for treatments explicitly targeting metacognition. For example, a study that treated attention as a primary outcome but that included metacognition as a secondary outcome would be included [i.e., (18, 37)] whereas one that only identified individuals with a metagcognitive deficit during the treatment process but did not track change, would have been excluded. Other articles excluded from the systematic review include: non-behavioral treatment studies such as those that use medication; studies including participants with a diagnosis of a degenerative disease (i.e., dementia). Gray literature and non-experimental publications (i.e., reviews) were excluded.

Three reviewers, the first, second, and fourth authors, completed initial parsing of the initial 266 journal articles based on the appraisal of the title and abstract of each paper included in the search results. If reviewers were not able to determine the eligibility of the paper based solely on the title and abstract, the full text was reviewed individually by two separate reviewers. Any disagreements between reviewers were brought to consensus through discussion.

Methodological Quality Review

In order to identify the methodological quality of the studies included in the systematic review, the Physiotherapy Evidence Database Rating Scale-Plus (PEDro+) and the Single Case Experimental Design Scale-Plus (SCED+) were utilized (41). The PEDro was chosen due to its reliability in evaluating the quality of randomized control trials, including evaluation of the study's internal validity and adequacy in communicating interpretable statistical results (42). The SCED was chosen as a reliable quality measure of single subject research designs (43). The PEDro+ and SCED+ designs, as amended by Cherney et al. (41), were utilized in order to account for each study's treatment fidelity and treatment replicability in addition to the original quality measures.

Four reviewers, the authors of this manuscript, extracted data and completed the PEDro+ and SCED+ quality ratings. For reliability, each study was independently assessed by two reviewers. Upon reviewer disagreement of the rating score, further review of the article followed by discussion resulted in rater consensus.

Data Extraction

The following data were systematically extracted from each article: year of publication, number of participants, age and gender of participants, type and severity of brain injury, time post onset of brain injury of each participant, concomitant diagnoses of the participants, whether the study utilized a control measure, design of the study, treatment type utilized in the study, duration of the intervention, and whether or not home practice was required as part of the study. The following outcome measures were obtained: cognitive, language, and rating scale outcome measures as well as any reports of the maintenance and/or generalization of the outcome skills. The clinical implications, study conclusions, study limitations and future research directions were also collected from each research article.

Results and Discussion

Aim 1: Describe and appraise the studies and the methodological quality of the studies reviewed

Literature Retrieved

The database search produced 257 articles following manual removal of duplicates. Through reference scanning and citation tracking, nine additional articles were determined to fit the study criteria, resulting in a total of 266 studies. Two-hundred and twenty-seven articles were removed following title and abstract screening. The remaining 39 articles underwent full article review, and ten additional articles were excluded due to inappropriate sample population [e.g., (44, 45)], treatment methods that did not involve metacognitive aspects [e.g., (46–49)], lack of peer review [e.g., (50–52)], or reporting of upcoming studies [e.g., (53)]. Of the 28 remaining articles, five were designed for people with aphasia. See Figure 1 for the PRISMA flow diagram, illustrating study selection.

FIGURE 1

Figure 1. Prisma flow diagram.

In all, 29 studies were found to fit the criteria for metacognitive intervention. Twenty-eight of the 29 studies reported positive outcome measures on at least one of the measures utilized.

Methodological Quality Rating

Six studies were reviewed using the PEDro+ rating scale and 24 studies were reviewed with the SCED+ rating. Levine et al. (54) included both a randomized control trial as well as a single subject design, so each was rated by the appropriate scale. Point by point interrater reliability was calculated for raters using the PEDro+ and the SCED+ scales, the interrater reliability scores were 93.65% and 95.24%, respectively. In order to be considered of adequate quality, a study must receive credit for at least half of the items on the checklist (41).

The results of the PEDro+ rating scale for each study are shown in Table 1. The methodological quality of the studies ranged from weak to high quality studies with scores ranging between six and eleven (out of a maximum of 13) (41). Five out of six RCTs received high ratings in this sample. On the PEDro+ rating scale, studies often lacked concealed allocation, blinding of therapists, blinding of assessors, and description of treatment fidelity.

TABLE 1

Table 1. Pedro+ rating scale.

The results of the SCED+ rating scale for each study are shown in Table 2. Scores on the SCED+ ranged from three to 12 (out of a maximum of 12). Sixteen out of 24 single case series designs achieved a high rating on the SCED+. In the SCED+, methodological quality ratings tended to be negatively impacted due to lack of interrater reliability, independence of assessors, and description of treatment fidelity.

TABLE 2

Table 2. SCED-plus rating scale.

Aim 2: investigate whether metacognitive interventions result in positive outcomes (cognitive, language, social) for persons with ABI

Study Characteristics

Population

Table 3 describes the characteristics of participants included in each study. Three-hundred and seventy individuals with ABI took part in the 29 aforementioned studies. ABI severity ranged from mild to severe; ages ranged from 18 to 83; and all had at least an 8th grade education. Etiology included: TBI (CHI and PHI), hypoxia, CVA, tumor, anoxia, arterial venous malformation, encephalitis, and aneurysm. Time post onset (TPO) of the injury ranged from 1 week to 34 years. Concomitant impairments included dysarthria, right hemianopsia, anxiety, depression, hearing loss, amnesia, hemiparesis or paralysis, and apraxia of speech.

TABLE 3

Table 3. Population characteristics.

Though reports of patient characteristics were generally considered detailed, defining four or more characteristics regarding the participant, there were trends of missing characteristics important to patient history noted. The missing components tended to include site of lesion data, handedness, prior treatment history and vision and hearing status.

Research Design

A wide range of study designs are included in this systematic review. The most frequently used research designs among studies were repeated measures (12, 61, 64) and single subject study designs [see Table 4; (2, 3, 15, 18, 25, 37, 38, 54, 62)].

TABLE 4

Table 4. Research & treatment designs.

Treatment Paradigm

Table 4 identifies treatment dosages, treatment types and treatment designs utilized across studies. Treatment dosages between and within treatment types were variable. Intensive treatment programs ranged from 4 to 8 h a day, spanned treatment durations of 4 days up to 12 weeks. The dosage of intensive treatment protocols ranged greatly from as few as 16 h over 4 days/2 weeks (56) up to 369 h over 5 days/12 weeks (36) of active treatment.

A wide range of treatment protocols with a focus on increasing and utilizing metacognitive skills were also used across studies. In each, the metacognitive treatment was either implemented as a standalone treatment or as a concurrent treatment with cognitive or language-based treatments such as: Attention Processing Training-III [APT-III; Lee and Sohlberg (37)], cognitive rehabilitation therapy [CRT; (12, 14, 32, 62)] or pragmatic language functioning (38). The treatments also varied between individual treatment, group treatment or a hybrid of both. In each treatment included in the systematic review, the goal was to increase an individual's self-awareness while simultaneously training the individual in the use of compensatory strategies to increase independence. A total of 21 different treatment paradigms were utilized across studies (see Table 4), though there were many patterns seen across treatment paradigms. The most commonly utilized metacognitive treatment paradigms, included Metacognitive Strategy Instruction (MSI), Goal Management Training (GMT), Cognitive Orientation to Occupational Performance Intervention (CO-OP), and Verbal and Video Feedback. See Supplementary Material A for more details on metacognitive treatment type.

Outcomes

Outcome scores were recorded in three areas: rating scale outcomes (Table 5), cognitive assessment outcome measures (Table 6) and language outcome measures (Table 7). Positive changes in treatment were defined by authors of each paper and reported accordingly in this review.

TABLE 5

Table 5. Rating scale outcomes.

TABLE 6

Table 6. Cognitive measure outcomes.

TABLE 7

Table 7. Language outcome measures.

Rating Scale Outcomes

In order to capture metacognitive changes in participants, self-report questionnaires were utilized; questionnaires were completed by either the participant themselves or a caregiver. Table 5 identifies rating scales used in each study as well as the results observed in each treatment study. Results varied within and between rating scales across studies, revealing either no significant change in metacognitive skills pre to post treatment to significant positive changes in metacognitive functioning. See Supplementary Material B for further details.

Cognitive Testing Outcomes

To identify cognitive outcomes in participants, an array of assessments were utilized and included tests of attention, executive function, problem solving, visual and verbal memory, task completion, error awareness, and error frequency as well as visual scanning. A complete list of cognitive tests utilized in the studies can be found in Table 6. The most widely used tests were the Trails B, Connor's Continuous Performance Test-II (CPT-II), and error frequency.

Table 6 identifies immediate positive outcomes (within a week of cessation of treatment) of each of the studies. Of the 29 studies included in this review, 12 assessed maintainence of skills at least 1 month post treatment. Of the 12 studies, 11 studies reported maintainence of skills among participants (2, 3, 14, 54, 56, 58, 59, 62–65) and one study did not report maintance of skills among participants (16). See Supplementary Material C for further detail.

Language Testing Outcomes

Language outcome measures varied throughout studies (See Table 7 for list of measures utilized). The only measure that was used in more than one study was the AIM's maze reading measure utilized in a series of studies conducted by Lee and Sohlberg (18) and Lee et al. (37); the rest of the outcome measures were unique to each study. Measures also varied between standardized and non-standardized outcomes. The assessments ranged in what they were testing including expressive language, discourse measurements and reading comprehension. Metacognitive rehabilitation was shown to be effective for language in some studies (25, 39), but demonstrated mixed effects in other studies (18, 25, 36, 65) and were not effective in two studies (38, 61).

Do Metacognitive Interventions Demonstrate Positive Outcomes for Persons With ABI?

The studies included in the final analysis utilized various research designs, therapeutic strategies, and assessment measures, making comparison difficult, but collectively they have provided insight regarding the use of metacognitive intervention with the ABI population. All 29 metacognitive interventions resulted in positive or mixed outcomes on rating scale, cognitive or language assessments (21 mixed outcomes and eight positive outcomes across scales). Of the studies measuring rating scale outcomes, eight of 19 studies had positive results, nine of 19 had mixed results between assessments, and two of 19 resulted in no change. Seventeen out of 19 studies utilizing rating scales observed at least one positive trend at completion. Nineteen out of 20 studies utilizing cognitive outcome measures reported at least one positive trend in cognitive measures following treatment that included metacognition. Fourteen of the 21 studies reported positive outcomes on all measures used, five out of the 21 studies had mixed outcomes between assessments [i.e, clinically significant change on the TEA Map Search, but not on CPT-II; (37)], and two out of the 21 studies reported no change on any outcome measure (34, 55). Similarly, seven out of nine metacognitive interventions resulted in at least one positive language outcome amongst measures.

Positive characteristics of metacognitive treatment include the feasibility and functionality of the treatment program, where participants can apply treatment to their everyday life (25, 58, 59). The programs are also flexible, allowing each treatment to be tailored to an individual's needs (25, 33, 38).

Authors also reported caveats of treatment protocols, including that the treatment can be taxing on cognitive skills that may be interrupted due to brain injury. The reliance on cognitive skills that may have been impacted due to injury may inhibit treatment outcomes due to the inability of individuals to apply learned material, thus making metacognitive treatment difficult for some. Inadequate length of treatment was also deemed to have had a negative effect on treatment outcomes, with authors suggesting that more time in treatment may have a more positive impact on effectiveness and generalizability of the treatment (55, 60). Considering these factors, metacognitive treatment for persons with ABI does appear to be effective, where positive changes across participants were observed on at least one measure in 22 out of 29 of the studies.

Aim 3: determine whether there is a specific type of metacognitive intervention that is more widely utilized for individuals with ABI within the research literature

MSI, GMT, CO-OP and APT-II are the only metacognitive interventions that were utilized in more than one study, however, no one of these appears to have been used more widely than others. On the other hand, there are some techniques that are more commonly used within each treatment paradigm. Goal setting and providing feedback on errors emerged as important components within treatment paradigms. MSI was utilized across two studies for individuals with mild to severe ABI (39, 59). Each study included eight participants where Finch et al. (59) utilized metacognitive intervention for social communication and Copley et al. (39) utilized MSI to address receptive language skills. Finch et al. (59) obtained mixed rating scale outcome measures (positive PPIC and GAS, no change seen onLCQ) and Copley et al. (39) identified positive language outcome measures.

GMT was utilized in three studies for individuals with ABI (severity unreported), with a total of 54 participants. Positive outcome measures were observed on both rating scale measures and cognitive measures (16, 54, 56).

CO-OP was utilized in two studies included in this systematic review; three participants were diagnosed with mild to severe TBI (58) and one participant was diagnosed with a moderate to severe right hemisphere stroke (15). Both research teams identified positive rating scale outcome measures upon completion of treatment.

APT-III was used to treat reading comprehension in ten individuals with mild to moderate aphasia in two studies and in both explicit feedback on performance was provided following each treatment session (18, 37). Findings on cognitive and language outcome measures were mixed between participants within these studies on both the TEA and Maze reading tasks. Two studies utilized feedback groups on persons with mild to severe TBI through completion of iADLs.

Although there is an inherent difference between self-awareness retraining (MSI, GMT, CO-OP) and error awareness training (Verbal and Video Feedback), there are many similarities between the two. In MSI, CO-OP, and GMT, breaking down the steps to completion and review of a task are still the main tenants; each of these metacognitive treatment paradigms include a review of success of task completion and errors. The self-awareness treatments also encourage participants to self-generate strategies to ensure successful task completion in the future, which implies recognizing errors. In Verbal and Video Feedback groups, participants were asked to rate their performance prior to initiating a task as well as following task completion. This strategy promotes self-awareness of abilities in addition to highlighting error awareness throughout the task. There is a cross-over between these two approaches, though each focuses different skills.

Several different interventions were utilized among the studies examined in this systematic review. GMT was the most commonly utilized and had consistently positive outcome scores compared to the other interventions. It was utilized in only three studies but included a relatively large number of participants (n = 54). Three studies are not sufficient to deem it the most effective type of treatment, but there is strong evidence supporting its use.

Aim 4: Explore whether metacognitive intervention is or has the potential to be effective for persons with aphasia given extent and quality of the current literature.

Though ABI includes individuals with aphasia following a stroke, those with different etiologies may respond differently to treatment. Only five of the 29 studies, with a total of 15 participants, focused on metacognitive rehabilitation for people with aphasia (18, 25, 36–38). Fourteen of the individuals with aphasia presented with mild to moderate aphasia, and one presented with severe aphasia. Three of the five studies focused on a combination of cognitive and language outcome measures (18, 36, 36) and two of the studies focused on language outcomes only (25, 66). Of the 15 participants, nine participants demonstrated positive results on language measures in response to metacognitive treatment, incuding increased attention, expressive language, and oral reading skills. With over half of the participants demonstrating a positive response to metacognitive treatment, one could make a case for implementing these aspects into language treatments. More research needs to be done to determine the optimal candidate with aphasia for this type of treatment.

Table 8 provides characteristics and outcomes from the studies focused on aphasia.

TABLE 8

Table 8. Aphasia outcomes.

Is Metacognitive Intervention Effective for Persons With Aphasia?

Treatments of persons with aphasia focused on those diagnosed with nonfluent aphasia with a given severity ranging from mild to severe. The individual with severe aphasia did not show improvement on language measures, though more research should be completed to identify the optimal population for metacogntive treatment. No singular treatment protocol was used throughout studies; APT-III was the only treatment protocol used more than once in the studies included for analysis (18, 37).

Effectiveness of treatments varied across measures and participants within each study. Though some positive effects were observed by researchers, most treatments did not identify positive outcomes for every participant and/or on every measure used (including cognitive, language and rating scale outcome measures). Of the studies utilizing rating scale measures, three out of three participants demonstrated positive changes on quality of life and goal attainment measures (36). Of the studies utilizing cognitive measures in addition to language measures, six out of 13 participants demonstrated positive increases on the RBANS and subtests of the TEA (18, 36, 37). Nine out of the 15 participants demonstrated positive increases on the language scores as well (18, 25, 36–38).

In summary, the evidence shows that though metacognitive paradigms can be applied to language therapy for people with aphasia, the results are equivocal with only a little more than half of all participants benefitting overall. The varied outcomes can be explained by the heterogeneous samples of participants utilized between studies with vastly different designs, and only small sample sizes available for evaluation. Though the results demonstrate the viability of utilizing metacognitive treatment for people with aphasia, there is not yet enough evidence to conclude the benefits of such treatment in this population. See Supplementary Material D for further details on treatments utilized.

How Outcomes in the TBI Literature Influence Potential Treatment for Persons With Aphasia?

The cogntive deficits that commonly follow TBI make metacognitive training an obvious choice for this population. Researchers tend to think of those with aphasia as having language disorder distinct from cognitive deficits, making metacogitive training appear to have less relevance. In fact, the relatively stronger cognitive skills may make those with aphasia stronger candidates. If memory and attention are less impaired, it follows that there is a higher likelihood of effectively using those skills to self-monitor language production.

The most commonly utilitzed outcome measures for testing cognitive skills as related to metacognition include attention and cognitive flexibility. Attention and cognitive flexibility, facets of executive function, which are closely related to metacognition, should be tested pre and post treatment in addition to target language outcomes. The treatment itself should utilize the breakdown of goals as seen in GMT, CO-OP and MSI as goal breakdown was observed in seven of the studies reporting positive results. Further, in order to foster self-awareness, verbal and video feedback should be utilized in addition to self-rating scales. In each of the studies that this was done, the effect was positive (14, 32). Using what has worked for individuals with TBI provides aphasiologists with the foundation needed to identify whether there is a therapeutic effect of metacognitive treatment for people with aphasia.

Conclusion

Metacogitive treatment has proven to be efficacious for many individuals with brain injury and provides a potential new avenue of exploration for those recovering from aphasia. The treatment itself is meant to foster self-awareness and error awareness in individuals, thus increasing each individual's independence in their use of treatment techniques. Teaching the participant to break goals into manageable steps and recognize when errors occur will hopefully spill into their everyday life and lead to generalization of skills. The recognition of strengths and weakness, as well as breaking down of goals may be more efficacious for those with nonfluent aphasia, where recognition of errors may be more helpful to those with fluent aphasia. With that, the use of both strategies within therapy for every type of aphasia may lead to development of optimal metacognitive skills.

Researchers can deduce that the success of treatment itself highly relies on the individual's constitution and motivation toward achieving goals, not unlike other treatment methods. In the realm of research studies, most individuals are motivated as volunteers, although drop-outs and missing data occurs (12). Several questions remain including the appropriate dosage of treatment and the appropriate population (severity) of the individual being treated. For people with aphasia and some with TBI, there is also a question of how receptive language skills may interfere with learning and internalizing the breakdown of steps to reach set goals. Though these questions remain, further research on the feasibility and utility of metacognitive treatment in order to improve the functioning of individuals with brain injury, namely aphasia, should be completed.

Future Directions

In the future, replication of studies is needed to validate the functionality and efficacy of metacognitive treatment for persons with ABI. Extant research shows that metacognitive treatment is useful, but the most effective treatment for different severity and presentations of persons with ABI remains unknown. Though studies included some individuals above the age of 65, we must focus research on the treatment practicality for persons over the age of 65, the ages where brain injury, namely stroke, is a common occurrence. Cognitive performance in the older population tends to decrease over time (67) so response to treatment is unknown for those above 65 years old.

Efficacy of metacognitive treatment for people with aphasia is not yet substantiated due to lack of evidence. There is also a lack of homogeneity amongst research studies, where different populations and different treatment paradigms were utilized. Hybrid treatment studies—those that involve both metacognitive treatment and language treatment– with a substantial number of participants need to be executed in order to begin determining whether metacognitive training is, in fact, appropriate for people with aphasia. Each study should focus on testing the metacognitive treatment protocol on different populations, starting with individuals with mild to moderate nonfluent aphasia, as the minimal evidence in this review show that this population responded adequately to metacognitive treatment. Studies should make use of information learned from this review, namely that treatment outcomes are likely to be improved when explicit education is provided to the participant in carrying out the various steps to complete desired outcomes. Video and verbal feedback should be considered for incorporation into future studies, as should self-awareness checklists to help increase an individual's awareness and independence while completing a task. In accordance with information gathered in this review, outcome measures should focus not only on language but on attention and cognitive flexibility as well. Following these guidelines, we will be able to take steps to discover the practicality and effectiveness of metacognitive treatment for people with aphasia.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fresc.2022.813416/full#supplementary-material

References

1. Toglia J, Kirk U. Understanding awareness deficits following brain injury. Neurorehabilitation. (2000) 15:57–70. doi: 10.3233/NRE-2000-15104

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Ownsworth T, Fleming J, Desbois J, Strong J, Kuipers P. A metacognitive contextual intervention to enhance error awareness and functional outcome following traumatic brain injury: a single-case experimental design. J Int Neuropsychol Soc. (2006) 12:54–63. doi: 10.1017/S135561770606005X

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Ownsworth T, Quinn H, Fleming J, Kendall M, Shum D. Error self-regulation following traumatic brain injury: a single case study evaluation of metacognitive skills training and behavioural practice interventions. Neuropsychol Rehabil. (2010) 20:59–80. doi: 10.1080/09602010902949223

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Sunderaraman P, Cosentino S. Integrating the constructs of anosognosia and metacognition: a review of recent findings in dementia. Curr Neurol Nerosci Rep. (2017) 17:27. doi: 10.1007/s11910-017-0734-1

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Mazancieux A, Souchay C, Casez O, Moulin CJA. Metacognition and self-awareness in multiple sclerosis. Cortex. (2019) 3:238–55. doi: 10.1016/j.cortex.2018.11.012

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Dean MP, Della Sala S, Beschin N, Cocchini G. Anosognosia and self-correction of naming errors in aphasia. Aphasiology. (2017) 31:725–40. doi: 10.1080/02687038.2016.1239014

CrossRef Full Text | Google Scholar

7. Rizio AA, Diaz MT. Language, aging and cognition: frontal aslant tract and superior longitudinal fasciculus contribute toward working memory performance in older adults. Clin Neurosci. (2016) 27:689–93. doi: 10.1097/WNR.0000000000000597

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Baltes PB, Staudinger UM, Lindenberger U. Lifespan psychology: theory andapplication to developmental psychology. Ann Rev Psychol. (1999) 50:471–507. doi: 10.1146/annurev.psych.50.1.471

PubMed Abstract | CrossRef Full Text

9. Salthouse TA. Mental exercise and mental aging. Evaluating the validity of the “Use it or lose it” Hypothesis. Perspect Psychol Sci. (2006) 1:68–87. doi: 10.1111/j.1745-6916.2006.00005.x

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Zimprich D, Martic M, Kleigel M, Dellenbach M, Rast P, Zeintl M. Cognitive abilities in old age: results from the Zurich longitudinal study of cognitive aging. Swiss J Psychol. (2008) 67:177–95. doi: 10.1024/1421-0185.67.3.177

CrossRef Full Text | Google Scholar

11. Hertzog C, Dunlosky J. Metacognition in later adulthood: spared monitoring can benefit older adults' self regulation. Assoc Psychol Sci. (2011) 20:262–7. doi: 10.1177/0963721411409026

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Goodwin RA, Lincoln NB, Bateman A. Dysexecutive symptoms and carer strain following acquired brain injury: changes measured before and after holistic neuropsychological rehabilitation. NeuroRehabilitation. (2016) 39:53–64. doi: 10.3233/NRE-161338

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Novakovic-Agopian T, Chen AJ, Rome S, Abrams G, Castelli H, Rossi A, et al. Rehabilitation of executive functioning with training in attention regulation applied to individually defined goals: a pilot study bridging theory, assessment, and treatment. J Head Trauma Rehabil. (2011) 26:325–38. doi: 10.1097/HTR.0b013e3181f1ead2

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Schmidt J, Fleming J, Ownsworth T, Lannin NA. Maintenance of treatment effects of an occupation-based intervention with video feedback for adults with TBI. NeuroRehabilitation. (2015) 36:175–86. doi: 10.3233/NRE-151205

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Skidmore ER, Holm MB, Whyte EM, Dew MA, Dawson D, Becker JT. The feasibility of meta-cognitive strategy training in acute inpatient stroke rehabilitation: case report. Neuropsychol Rehabil. (2011) 21:208–23. doi: 10.1080/09602011.2011.552559

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Waid-Ebbs JK, BCBA-D, Daly J, Wu SS, Berg WK, Bauer RM, et al. Response to goal management training in veterans with blast-related mild traumatic brain injury. J Rehabil Res Dev. (2014) 51:1555–66. doi: 10.1682/JRRD.2013.12.0266

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Fridriksson J, Nettles C, Davis M, Morrow L, Montgomery A. Functional communication and executive function in aphasia. Clin Linguist Phonet. (2006) 20:401–10. doi: 10.1080/02699200500075781

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Lee JB, Sohlberg MM. Evaluation of attention training and metacognitive facilitation to improve reading comprehension in aphasia. Am J Speech Lang Pathol. (2013) 22:S318–33. doi: 10.1044/1058-0360(2013/12-0099)

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Ramsberger G. Achieving conversational success in aphasia by focusing on non-linguistic cognitive skills: a potentially promising new approach. Aphasiology. (2005) 19:1066–73. doi: 10.1080/02687030544000254

CrossRef Full Text | Google Scholar

20. Berthier ML. Poststroke aphasia: epidemiology, pathophysiology and treatment. Drugs Aging. (2005) 22:163–82. doi: 10.2165/00002512-200522020-00006

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Darley FL. Aphasia. Philadelphia, PA: W.B. Saunders.

Google Scholar

22. McNeil MR. Aphasia in the adults. In: Lass NJ, McReynolds LV, Northern JL, Yoder DE, editors. Handbook of Speech-Language Pathology and Audiology. Philadelphia, PA: W.B. Saunders Company (1988). p. 738–86.

23. Aranzeta M, Webster J, Laka I, Matinez-Zabaleta M, Howard D. What happens when they think they are right? Error awareness analysis of sentence comprehension deficits in aphasia. Aphasiology. (2018) 32:1418–44. doi: 10.1080/02687038.2017.1423270

CrossRef Full Text | Google Scholar

24. Cahana-Amity D, Albert ML. Redefining Recovery From Aphasia. New York, NY: Oxford University Press (2015).

Google Scholar

25. Mayer JF, Mitchinson SI, Mayer LL. Addressing concomitant executive dysfunction and aphasia: previous approaches and the new brain budget protocol. Aphasiology. (2017) 31:837–60. doi: 10.1080/02687038.2016.1249333

CrossRef Full Text | Google Scholar

26. O'Seaghdha PG, Frazer AK. The exception does not rule: attention constrains form preparation in word production. J Exp Psychol Learn Mem Cogn. (2014) 30:797–810. doi: 10.1037/a0035576

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Villard S, Kiran S. To what extent does attention underlie language in aphasia? Aphasiology. (2017) 31:1226–45. doi: 10.1080/02687038.2016.1242711

CrossRef Full Text | Google Scholar

28. Levelt WJM. Speaking: From Intention to Articulation. Cambridge, MA: MIT Press (1989).

Google Scholar

29. Goverover Y, Genova H, Griswold H, Chiaravalloti N, DeLuca J. Metacognitive knowledge and online awareness in persons with multiple sclerosis. Neurorehabilitation. (2014) 35:315–23. doi: 10.3233/NRE-141113

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Nozari N, Novick J. Monitoring and control in language production. Curr Dir Psychol Sci. (2017) 26:403–10. doi: 10.1177/0963721417702419

CrossRef Full Text | Google Scholar

31. Cicerone KD, Dahlberg C, Malec JF, Langenbahn DM, Felicetti T, Kneipp S, et al. Evidence-based cognitive rehabilitation: updated review of the literature from 1998 through 2002. Arch Phys Med Rehabil. (2005) 86:1681–92.

PubMed Abstract | Google Scholar

32. Schmidt J, Fleming J, Ownsworth T, Lannin NA. Video feedback on functional task performance improves self-awareness after traumatic brain injury: a randomized controlled trial. Neurorehabil Neural Repair. (2012) 27:316–24. doi: 10.1177/1545968312469838

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Fitzgerald MCC, O'Keefe F, Carton S, Coen RF, Kelly S, Dockree P. Rehabilitation of emergent awareness of errors post traumatic brain injury: a pilot intervention. Neuropsychol Rehabil. (2017) 29:821–43. doi: 10.1080/09602011.2017.1336102

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Fong KNK, Howie DR. Effects of an explicit problem-solving skills training program using a metacomponential approach for outpatients with acquired brain injury. Am J Occup Therapy. (2009) 63:525–34. doi: 10.5014/ajot.63.5.525

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Kennedy MRT, Coelho C, Turkstra L, Ylvisaker M, Sohlberg MM, Yorkston K, et al. Intervention for executive functions after traumatic brain injury: a systematic review, meta-analysis and clinical recommendations. Neuropsychol Rehabil. (2008) 18:257–99. doi: 10.1080/09602010701748644

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Gilmore N, Ross K, Kiran S. The intensive cognitive-communication rehabilitation program for young adults with acquired brain injury. Am J Speech Lang Pathol. (2019) 28:341–58. doi: 10.1044/2018_AJSLP-17-0153

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Lee JB, Sohlberg MM, Harn B, Horner R, Cherney LR. Attention process training-3 to improve reading comprehension in mild aphasia: a single-case experimental design study. Neuropsychol Rehabil. (2018) 30:430–61. doi: 10.1080/09602011.2018.1477683

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Rosell-Clari V, Hernandez-Sacristan C. Anomia rehabilitation viewed from a pragmatic-functional paradigm: a case study. Rev Investig Logopedia. (2017) 7:47–70. Available online at: https://www.redalyc.org/articulo.oa?id=350851047003

Google Scholar

39. Copley A, Smith K, Savill K, Finch E. Does metacognitive strategy instruction improve impaired receptive cognitive-communication skills following acquired brain injury? Brain Injury. (2015) 29:1309–16. doi: 10.3109/02699052.2015.1043343

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. (2021) 372:n160. doi: 10.1136/bmj.n160

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Cherney LR, Simmons-Mackie N, Raymer A, Armstrong E, Holland A. Systematic review of communication partner training in aphasia: methodological quality. Int J Speech Lang Pathol. (2013) 15:535–45. doi: 10.3109/17549507.2013.763289

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Herbert R, Moseley A, Sherrington C. PEDro: a database of RCTs in physiotherapy. Health Inform Manag. (1998–1999) 28:186–8. doi: 10.1177/183335839902800410

PubMed Abstract | CrossRef Full Text

43. Tate R, McDonald S, Perdices M, Togher L, Schultz R, Savage S. Rating the methodological quality of singlesubject designs and n-of-1 trials: Introducing the Single-Case Experimental Design (SCED) Scale. Neuropsychol Rehabil. (2008) 18:385–401. doi: 10.1080/09602010802009201

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Strazzer S. Attention remediation following traumatic brain injury in childhood and adolescence. Neuropsychology. (2009) 1:40–9. doi: 10.1037/a0013409

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Van der Merwe A. Self-correction in apraxia of speech: the effect of treatment. Aphasiology. (2007) 21:658–69. doi: 10.1080/02687030701192174

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Griffiths GG, Sohlberg MM, Kirk C, Fickas S, Biancarosa G. Evaluation of use of reading comprehension strategies to improve reading comprehension of adult college students with acquired brain injury. Neuropsychol Rehabil. (2016) 26:161–90. doi: 10.1080/09602011.2015.1007878

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Mauszycki SC, Bailey DJ, Wambaugh JL. Acquired apraxia of speech: the relationship between awareness of errors in word productions and treatment outcomes. Am J Speech Lang Pathol. (2017) 26:664–73. doi: 10.1044/2017_AJSLP-16-0111

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Ownsworth T, Little T, Turner B, Hawkes A, Shum D. Assessing emotional status following acquired brain injury: the clinical potential of depression, anxiety and stress scales. Brain Injury. (2008) 22:858–69. doi: 10.1080/02699050802446697

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Sohlberg MM, Griffiths GG, Fickas S. An evaluation of reading comprehension of expository text in adults with traumatic brain injury. Am J Speech Lang Pathol. (2014) 23:160–75. doi: 10.1044/2013_AJSLP-12-0005

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Kelkar AS. Treating Problem Solving Deficits in Traumatic Brain Injury (2014). Available online at: https://thescholarship.ecu.edu/bitstream/handle/10342/4691/Kelkar_ecu_0600E_11288.pdf (accessed February 27, 2019).

Google Scholar

51. Lui KPY. Mental Imagery Relearning for Patients Suffered From Stroke. The Hong Kong Polytechnic University (2002). Available online at: http://www.lib.polyu.edu.hk (accessed February 28, 2019).

PubMed Abstract | Google Scholar

52. Seeberger SK. Reading Attitudes and Proficiency of Individuals With Aphasia: Accessing Knowledge and Application of Psycholinguistic Reading Strategies Through Retrospective Miscue Analysis. Ann Arbor, MI: ProQuest LLC (2012).

53. Fleming J, Ownsworth T, Doig E, Hutton L, Griffin J, Kendall M, et al. The efficacy of prospective memory rehabilitation plus metacognitive skills training for adults with traumatic brain injury: study protocol for a randomized controlled trial. Trials. (2017) 18:3. doi: 10.1186/s13063-016-1758-6

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Levine B, Robertson IH, Clare L, Carter G, Hong J, Wilson BA, et al. Rehabilitation of executive functioning: an experimental–clinical validation of goal management training. J Int Neuropsychol Soc. (2000) 6:299–312. doi: 10.1017/S1355617700633052

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Goverover Y, Johnston MV, Toglia J, DeLuca J. Treatment to improve self-awareness in persons with acquired brain injury. Brain Injury. (2007) 21:913–23. doi: 10.1080/02699050701553205

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Tornås S, Løvstad M, Solbakk A.-K., Evans J, Endestad T, Kristian Hol PK, et al. Rehabilitation of executive functions in patients with chronic acquired brain injury with goal management training, external cuing, and emotional regulation: a randomized controlled trial. J Int Neuropsychol Soc. (2016) 22:436–52. doi: 10.1017/S1355617715001344

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Villalobos D, Bilbao Á, Espejo A, García-Pacios J. Efficacy of an intervention programme for rehabilitation of awareness of deficit after acquired brain injury: a pilot study. Brain Injury. (2018) 32:158–66. doi: 10.1080/02699052.2017.1387931

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Dawson DR, Gaya A, Hunt A, Levine B, Lemsky C, Polatajko HJ. Using the cognitive orientation to occupational performance (CO-OP) with adults with executive dysfunction following traumatic brain injury. Can J Occup Therapy. (2009) 76:115–27. doi: 10.1177/000841740907600209

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Finch E, Cornwell P, Copley A, Doig E, Fleming J. Remediation of social communication impairments following traumatic brain injury using metacognitive strategy intervention: a pilot study. Brain Injury. (2017) 31:1830–9. doi: 10.1080/02699052.2017.1346284

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Laatsch L, Stress M. Neuropsychological change following individualized cognitive rehabilitation therapy. NeuroRehabilitation. (2000) 15:189–97. doi: 10.3233/NRE-2000-15305

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Laatsch L, Little D, Thulborn K. Changes in fMRI following cognitive rehabilitation in severe traumatic brain injury: a case study. Rehabil Psychol. (2004) 49:262–7. doi: 10.1037/0090-5550.49.3.262

CrossRef Full Text | Google Scholar

62. Ramanathan P, Turner HA, Stevens MC. Intensive cognitive rehabilitation therapy for chronic traumatic brain injury: a case study of neural correlates of functional improvement. Aphasiology. (2019) 33:289–319. doi: 10.1080/02687038.2018.1461801

CrossRef Full Text | Google Scholar

63. Raskin SA, Smith MP, Mills G, Pedro C, Zamroziewicz M. Prospective memory intervention using visual imagery in individuals with brain injury. Neuropsychol Rehabil. (2019) 29:289–304. doi: 10.1080/09602011.2017.1294082

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Toglia J, Johnston MV, Goverover Y, Dain B. A multicontext approach to promoting transfer of strategy use and self regulation after brain injury: an exploratory study. Brain Injury. (2010) 24:664–77. doi: 10.3109/02699051003610474

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Kintz S, Hibbs V, Henderson A, Andrews M, Wright HH. Discourse-based treatment in mild traumatic brain injury. J Commun Disord. (2018) 76:47–59. doi: 10.1016/j.jcomdis.2018.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Rosell-Clari V, Hernandez-Sacristan C. MetAphAs. Protocolo de Exploracion de Habilidades Metalinguisticas en la Afasia. Valencia: Nua Libres (2014).

Google Scholar

67. Murman D. The impact of age on cognition. Semin Hear. (2015) 36:111–21. doi: 10.1055/s-0035-1555115

PubMed Abstract | CrossRef Full Text | Google Scholar