Metacognitive Therapy for Individuals at High Risk of Developing Psychosis: A Pilot Study

Developing effective interventions for preventing first episode psychosis have been an important research focus in the last decade. Cognitive behavioral therapy is a currently indicated treatment for people at ultra-high risk of psychosis, however, access and resource issues limit its delivery within the NHS. Treatments which partial out potential active ingredients and are aimed at a range of psychological difficulties seen within this population have the potential to be more efficacious and efficient. We conducted a single-arm exploratory pilot trial, designed to investigate the feasibility and acceptability of Metacognitive therapy for individuals at ultra-high risk (UHR) of developing psychosis. Trial uptake was good, with 11 out of 12 referred individuals meeting for an eligibility assessment (one individual was excluded prior to the assessment). Of these, 10 individuals were eligible and included in the trial. Retention to treatment was high with 80% treatment adherence gained and an overall average of 8 sessions completed. All participants were offered follow-up assessments immediately post-treatment and at 6 months, which comprised measures of psychotic like experiences, anxiety and depression, and metacognitive processes implicated in the model. Retention to the post-treatment (12-week) follow-up was good, with 80% completion; however retention to the 6-month follow-up was lower at 60%. Clinically significant results were observed in psychotic like experiences, anxiety, depression and functioning with medium to large effect sizes. Measures related to beliefs and processes targeted within MCT showed clinically significant change with medium to large effect sizes. Our results suggest that MCT based upon a specific metacognitive model for individuals meeting ARMS criteria may be an important treatment target and warrants further attention. Limitations and possible focuses for future research are discussed.

Registration: ISRCTN53190465 http://www.isrctn.com/ISRCTN53190465.

Introduction

Given the cost to individuals, families and services of psychosis, it is unsurprising that there has been great emphasis in research on the prevention of the development of a first episode. There are reliable and valid criteria available to identify help-seeking individuals in diverse settings who are at high risk of developing psychosis. Yung et al. (1998) developed operational criteria to identify three subgroups possessing an “at risk mental state” (ARMS) for psychosis. Two subgroups specify state risk factors, defined by the presence of either transient psychotic symptoms, called Brief Limited Intermittent Psychotic Symptoms (BLIPS) or attenuated (subclinical) psychotic symptoms (AS). The other subgroup comprises trait-plus-state risk factors, operationally defined by the presence of diminished functioning plus a first-degree relative with a history of psychosis. All subgroups are within a specified age range known to be at greatest risk for the onset of psychosis, and all participants in studies of ARMS to date have been help-seeking.

In addition to identification, developing effective interventions to prevent or delay transition to psychosis have been an important research focus, given the potential benefits for symptoms, recovery and other outcomes. To date, there have been eight randomized controlled trials, each using similar operational definitions of ARMS, that have investigated antipsychotic medication, omega-3 polyunsaturated fatty acids and/or psychological interventions. The studies were conducted in Australia (McGorry et al., 2002; Yung et al., 2011), North America (McGlashan et al., 2006; Addington et al., 2011), the United Kingdom (Morrison et al., 2004a, 2006, 2012), the Netherlands (van der Gaag et al., 2012; Ising et al., 2016) and Austria (Amminger et al., 2010). Significant benefits at 12 months post-intervention were found for both cognitive behavioral therapy (Morrison et al., 2004a, 2006; van der Gaag et al., 2012) and omega-3 polyunsaturated fatty acids (Amminger et al., 2010). Therefore, at present, the recommended psychological treatment for young people at high risk of developing psychosis is cognitive behavioral therapy. The treatment duration indicated by the current evidence base (see Stafford et al., 2013) may be up to 26 sessions if following an appropriate manual for cognitive behavioral therapy for young people at risk of psychosis (e.g., French and Morrison, 2004). However, only a small number of young people meeting the at-risk of psychosis criteria are offered such indicated interventions in the NHS. In 2016 an audit found that only 41% of clients under EIS nationwide are offered CBTp within 6 months of acceptance into EIT (Health Quality Improvement Partnership and the Royal College of Psychiatrists, 2016). The audit operationalized an offer of CBTp as an offer of 16 sessions, delivered by appropriately trained and supervised therapists.

The United Kingdom access and waiting time standards (Nhs England, the National Collaborating Centre for Mental Health, and the National Institute for Health, and Care Excellence, 2016), which came into force from 1st April 2016, ensures that Early Interventions Services offer assessments for ARMS. However, the ability for NHS services to offer indicated interventions in line with the research protocols for people meeting ARMS criteria is limited given their stretched resources. If we can find out more about the active ingredients in psychological interventions, we may be able to make our interventions more efficacious and efficient. There has also been a call for more intervention trials aimed at the range of psychopathology observed in those at high risk of developing psychosis, to inform best care practices (Carpenter, 2018).

The cognitive model developed by Morrison (2001) implicates both cognitive and metacognitive processes in the development of psychosis. It is possible that using a specifically metacognitive approach could be a more efficient, quicker treatment compared with a mixed model. Studies have demonstrated a role of metacognitive beliefs and processes in the development and maintenance of psychosis (Smári et al., 1994; Morrison et al., 2002, 2004b, 2005, 2007; Morrison and Wells, 2003, 2007; Sellers et al., 2018). Metacognitive processes are also prevalent and important in those meeting criteria for ARMS (e.g., Morrison et al., 2007; Brett et al., 2009; Debbané et al., 2009; Barkus et al., 2010; Debbané et al., 2012; Palmier-Claus et al., 2013; Cotter et al., 2017).

Different metacognitive models and the approaches they derive for the treatment of psychosis have been summarized by Lysaker et al. (2018). These models are distinct and underpinned by different theoretical perspectives. The approach described here is underpinned by the S-REF (self-regulatory executive function) model proposed by Wells and Matthews (1994, 1996) and is not to be confused with metacognition as defined within other models being used within the area of psychosis (e.g., Lysaker et al., 2005). Wells and Matthews propose that it is not the occurrence of mental events (i.e., negative thoughts and emotion) that give rise to prolonged distress, but the resulting perseverative thinking style called the cognitive attentional syndrome (CAS). The CAS is comprised of strategies aimed at managing distressing thoughts and emotions which include worry, rumination, threat monitoring thought control strategies and maladaptive coping behaviors such as avoidance and reassurance seeking (Wells, 2008). The model implicates a central role of the CAS which becomes employed in response to negative thoughts and feelings causing an extension to psychological distress and worsening and extending negative affect. The S-REF model hypothesizes that CAS activity is promoted by underlying metacognitive beliefs both positive and negative in orientation. For example, people hold positive beliefs such as “worrying will help me to be prepared” and on the other hand negative beliefs about the uncontrollability and danger of thoughts and feelings such as “I cannot control my worrying once it begins.”

A large body of evidence implicates a central role of metacognition in numerous mental health problems including generalized anxiety disorder, social anxiety, depression, PTSD and psychosis (Wells, 1995; Clark and Wells, 1995; Morrison, 2001; Papageorgiou and Wells, 2001), and metacognitive therapies for such problems are being applied successfully (Wells, 2000, 2008; Normann and Morina, 2018). A metacognitive model of the positive symptoms of psychosis has been developed (Morrison, 2001). This evidence-based model predicts that metacognitive therapy may help to reduce psychotic like symptoms and target symptoms of co-morbid emotional disorders.

Where metacognitive processes are implicated, it is likely that specific metacognitive therapy for people at high risk of developing psychosis will be effective. Recent evidence has suggested that metacognitive therapy (MCT; Wells and Matthews, 1994, 1996) is a useful alternative to CBT for understanding and treating disorders such as generalized anxiety disorder, post-traumatic stress disorder, obsessive compulsive disorder and depression (Wells and King, 2006; Wells, 2008; Normann and Morina, 2018). MCT is a shorter treatment than traditional CBT, requiring around 6–8 sessions for symptom improvement (Wells, 2008). It has low drop-out rates and appears to be well-tolerated in emotional disorders. MCT could be particularly useful as an alternative treatment for young people at high risk of developing psychosis as it does not directly challenge the patients’ belief systems, but rather focuses on the process, of thinking. It is also potentially generalizable to the other axis I disorders which have high co-morbidity within young at-risk individuals (e.g., Leicester et al., 2002).

A single arm feasibility study of 12 sessions of MCT for individuals with psychosis has been conducted (Morrison et al., 2014). This study successfully recruited 10 participants and adherence to MCT was shown to be acceptable; all participants received at least one session and 9/10 received 6 sessions or more (a mean of 10.6). The treatment demonstrated encouraging within-subjects effect sizes on positive symptoms (Cohen’s d = 1.27) and delusional beliefs (Cohen’s d = 0.71), and on negative symptoms (Cohen’s d = 0.62), for which the evidence base in support of CBT is sparse. These positive results for individuals who by definition have a more ‘serious’ symptom profile suggests that MCT may also be useful for people at high risk of developing psychosis.

This pilot trial provides a preliminary investigation into the acceptability and feasibility of MCT for people meeting ARMS criteria who were experiencing distressing symptoms. It also provides an initial investigation into the efficacy of MCT in producing relief from psychotic like symptoms. In line with standard feasibility aims, the objectives of this study were to assess recruitment rate and to examine the appropriateness, feasibility and acceptability of the intervention and measures. It was hypothesized that MCT would produce symptom relief from unusual or overvalued beliefs (e.g., paranoia) and perceptual experiences (e.g., hallucinations), defined by significantly reduced CAARMS scores at both end of treatment and follow-up.

Materials and Methods

Design

This study was a single-arm exploratory trial, designed to investigate the feasibility and acceptability of MCT for individuals at ultra-high risk (UHR) of developing psychosis. A National Research Ethics Committee approved the study prior to commencing data collection (13/NW/0238).

Participants

All participants were being seen by NHS services specifically developed to work with people at high risk of developing psychosis [e.g., an Early Detection and Intervention Team (EDIT) or Early Intervention Service (EIS)] in the community. All participants met criteria for being at UHR of developing psychosis as operationally defined by the Comprehensive Assessment of At-Risk Mental States (CAARMS: Yung et al., 2005). NHS patients are typically referred to such services by primary care clinicians e.g., General Practitioners (GPs) or primary care psychology services, and should be offered assessment, ongoing monitoring of their mental health and Cognitive Behavioral Therapy (CBT).

Our exclusion criteria were: (i) a moderate to severe learning disability; (ii) a neurological impairment of organic origin (e.g., head injury or dementia); (iii) limited command of the English language, sufficient to impede the use of standardized assessments or accessibility of therapy; (iv) currently receiving inpatient care; (v) judged by their case manager to be clinically unstable over the 4 weeks prior to participation; (vi) taking prescribed antipsychotic medication; or (vii) a primary diagnosis of substance dependency.

Measurements

Primary Outcomes

The primary outcome measure was to assess feasibility and acceptability outcomes including levels of recruitment into the trial, retention of participants across baseline assessment, intervention and follow-up periods, number of “drop outs,” defined as an individual who attended three or less therapy sessions, and adherence to the therapy protocol.

Secondary Outcomes

A number of secondary outcomes were included. The CAARMS (Yung et al., 2005); a semi-structured interview credited as the gold-standard for assessing “at risk” symptoms. The CAARMS interview comprises six subscales assessing unusual thought content, non-bizarre ideas, perceptual abnormalities, disorganized speech, aggressive behavior, and suicidality over the previous month. Each subscale is rated on two seven-point scales according to the severity and frequency of any endorsed symptom. Scores for each subscale are derived from the product of the severity (0–6) and frequency scores (0–6). A number of studies have shown that the CAARMS has excellent inter-rater reliability, in addition to concurrent, discriminant and predictive validity (Yung et al., 2005).

The Global Assessment of Functioning (GAF; Hall, 1995) was used to measure personal, social and psychological functioning. The GAF is a semi-structured interview measure used in conjunction with the CAARMS and scores range from 0 to 100, with higher scores indicating greater global functioning. The GAF has been widely validated (Jones et al., 1995) and has been used extensively in studies examining UHR samples (Hartmann et al., 2016).

Anxiety and depression symptoms were measured using the Hospital Anxiety and Depression Scale (HADS; Zigmond and Snaith, 1983), a 14 item self-report questionnaire providing separate total scores for anxiety and depression severity. All items are rated on four-point scales in reference to the previous week and higher scores indicate greater symptom severity. The HADS is considered to possess adequate to good properties of sensitivity, case-finding, concurrent validity and internal consistency (Bjelland et al., 2002). It is a brief and straightforward self-report measure which assesses both depression and anxiety (Hansson et al., 2009) thus reducing the burden of completion for participants. It has been used in similar populations to ours, such as adolescents, young people and adults with psychosis (Bernard et al., 2006; White et al., 2011; Pyle et al., 2019), therefore allowing comparability of our results with similar studies.

Metacognitive beliefs were assessed using the metacognitive questionnaire (MCQ-30; Wells and Cartwright-Hatton, 2004). This is a 30-item measure comprising five subscales: positive beliefs about worry; negative beliefs about uncontrollability and danger of extended processing; cognitive confidence; cognitive self-consciousness; and need to control thoughts. All items are rated on four-point Likert scales ranging from 1 (“do not agree”) to 4 (“agree very much”). The MCQ has strong psychometric properties and has been used in studies examining UHR populations (Morrison et al., 2007; Bright et al., 2018).

Activation of the CAS was measured using the CAS scale (CAS-1; Wells, 2008); a 16-item questionnaire assessing worry, threat monitoring, strategies in response to negative thoughts or feelings, and metacognitive beliefs about extended processing and thought control strategies. For this study, only items assessing degrees of worry and threat monitoring were included. Both items are rated on eight-point scales ranging from 0 (“none of the time”) to 8 (“all of the time”), in reference to experiences during the previous week. The CAS-1 has strong psychometric properties, including good internal consistency, concurrent and predictive validity (Sellers et al., 2018).

Appraisals of voice hearing were measured using the Interpretations of Voices Inventory (IVI; Morrison et al., 2002), a 26-item self-report questionnaire assessing positive and negative hypothetical interpretations of voices. All items are rated on four-point Likert scales ranging from 1 (“not at all”) to 4 (“very much”). The IVI comprises three subscales: metaphysical beliefs, positive beliefs and beliefs about loss of control. The IVI is reliable and valid for use with people defined as high in psychosis-proneness (Morrison et al., 2004b).

Metacognitive beliefs about paranoia were assessed using the Beliefs about Paranoia Scale – Short form (BAPS; Gumley et al., 2011), an 18-item self-report questionnaire measuring conviction in positive and negative interpretations. Each item is measured on a four-point Likert scale ranging from 1 (“not at all”) to 4 (“very much”). The BAPS can be subdivided into three scales: negative beliefs about paranoia, positive beliefs about paranoia as a survival strategy and normalizing beliefs. The BAPS has strong psychometric properties (Morrison et al., 2011) and has been used with UHR samples (Morrison et al., 2015).

Procedure

All participants were recruited from EDIT and EIS teams. Individuals were identified and approached by members of their clinical teams to participate in this study. Participants completed a battery of assessments (CAARMS, HADS, MCQ-30, CAS-1, IVI, BAPS) at baseline (pre-therapy), end of therapy (3 months) and 6 months post-therapy. Two clinical measures (HADS, CAS-1) were administered prior to each therapy session. All assessments were conducted by a trainee clinical psychologist or qualified clinical psychologist with extensive training and experience of administering the CAARMS. Throughout the study, all CAARMS scores were reviewed in supervision and ratings were finalized through group discussion. To reduce bias, all follow-up assessments (end of therapy and 6-months post therapy) were conducted by a therapist who was not involved in the delivery of therapy, for each respective participant.

Intervention

The MCT intervention consisted of 12 sessions over a period of 12 weeks following baseline assessment, and followed the treatment manual developed by Wells (2009). We adapted the metacognitive model of generalized anxiety disorder (Wells, 1995) for use with UHR individuals, in a similar way to the therapy previously described for people with a diagnosis of schizophrenia (Hutton et al., 2014; Morrison et al., 2014). The metacognitive model asserts that psychological distress results from extended processing in response to negative cognitions (comprising thoughts, images, voices etc.). Examples of extended processing include worry, rumination and unhelpful thought control strategies, collectively termed as the CAS (Wells and Matthews, 1996). MCT aims to reduce the CAS by exploring new ways of responding to worrying thoughts and modifying metacognitive beliefs, which contribute to worry, rumination and distress. The MCT intervention consisted of:

1. Case conceptualization via assessment of a recent episode of worry in which the person became distressed by the worry. This allows for the generation of an idiosyncratic version of the metacognitive model depicting positive and negative metacognitive beliefs as well as coping behaviors and thought control strategies.

2. Socialization to the metacognitive model through sharing the conceptualization and exploring the role of beliefs about worry via verbal and behavioral experiment strategies as well as the effects of behaviors.

3. Questioning and challenging beliefs about uncontrollability via verbal techniques and loss of control experiments including the introduction of detached mindfulness and worry postponement.

4. Challenging beliefs about the danger of worry via verbal reattribution and consolidating learning via behavioral experiments.

5. Challenging positive metacognitive beliefs about worry and generating alternative ways of responding to internal events.

6. Developing and reinforcing new plans for processing worry where old plans and new plans are described side by side.

7. Relapse prevention.

Therapists

Four therapists delivered the intervention. All therapists received training in the MCT manual and received weekly supervision to ensure adherence to the model. With participant consent, supervisors reviewed audio recordings of therapy to maximize fidelity.

Statistical Analysis

Data were analyzed using Stata 14.0 (Stata Corporation, 2011). Emphasis was placed on descriptive and summary statistics, and flow across the different stages of the trial. In the absence of normally distributed data, Wilcoxon signed-rank tests assessed differences between assessment scores at baseline and post-therapy, and at baseline and 6-month follow-up. Summary effect sizes were also calculated (Cohen’s D) using SD pre as the pre-test value provides an arguably better estimate of the true population value for within-subjects designs. This value is also thought to provide a better comparison to the d statistic in paired-design experiments thereby making it useful in meta-analysis (Cummings, 2012). Regression, with clustering at the participant level (Rogers, 1993), was used to explore the relationship between meta-cognitive beliefs and worry (CAS-1) across the sessional measures in a ‘long-form’ of the data.

Results

The trial ended in December 2015 with a final sample size of 10. Demographic information for the sample is presented in Table 1. The consort diagram (Figure 1) shows the size of the sample across the different stages of the study. Uptake of the trial was good. Out of 12 referred individuals, 11 met for baseline eligibility assessments. One individual was excluded prior to assessment, and one was found to be ineligible post-baseline assessment due to not meeting CAARMS threshold criteria. Adherence to MCT was adequate with participants completing an average of 8.0 sessions (SD, 4.4; range, 1–12.). Two participants only completed one session due to changes in employment and unstable life circumstances, respectively. Eighty percent of participants completed the post-treatment (12-week) assessment, (one of these participants only completed the CAARMS assessment and not the questionnaire measures) whereas 60% repeated these assessments at the 6-month follow-up. This was due to clients moving out of area (n = 1), declining assessment (n = 2), and physical health complications (n = 1).

TABLE 1

Table 1. Demographic information at baseline.

FIGURE 1

Figure 1. CONSORT diagram for flow at the different stages of the trial.

Participants’ scores for psychotic like experiences and functioning suggested that they had higher levels of symptoms and were functioning poorer when compared with larger samples of young people meeting UHR criteria (e.g., Morrison et al., 2011). In line with this, only one participant was not in the NEET category whilst 9/10 participants were not in education employment or training. The sample also demonstrated above cut off levels for anxiety (within the severe range) and depression (within the moderate range). Scores on the MCQ were considerably higher than equivalent populations (e.g., Bright et al., 2018), indicative of a sample who had higher levels of psychopathology than seen in previous trials of other interventions e.g., CBT.

Of the eight participants who completed the 12-week assessments, three were still at risk of psychosis, four no longer met ARMS criteria, and one had transitioned to a first psychotic episode, although declined to be referred for further treatment. At the 6 months follow-up (n = 6), two clients were at-risk of developing psychosis and four no longer met ARMS criteria. Summary statistics for primary and secondary outcomes (mean, SD) and effect size analyses (Cohen’s d) are presented in Table 2. In summary, at 12 weeks participants had significantly lower scores on four out of six CAARMS subscales: non-bizarre ideas (p = 0.018), perceptual abnormalities (p = 0.026), disorganized speech (p = 0.043), suicidal behavior (p = 0.042). HADS scores were significantly lower (anxiety: p = 0.017, depression p = 0.046), as were CAS-1 scores (worry: p = 0.018, threat monitoring: p = 0.028). IVI scores were significantly lower (p = 0.027) and GAF scores significantly higher (p = 0.035).

TABLE 2

Table 2. Summary statistics and outcome data for key variables.

At 6 months, IVI scores remained significantly lower (p = 0.027) whilst only two subscales described above remained significantly lower: CAARMS non-bizarre ideas (p = 0.027) and HADS anxiety (p = 0.043). There were no significant differences at 12 weeks or 6 months on CAARMS unusual thought content (p = 0.345 and p = 0.068, respectively) or aggressive behavior subscales (p = 0.068 and p = 0.916, respectively). There were also no significant differences in BAPS score at 12 weeks (p = 0.173) or 6 months (p = 0.114).

In line with the mechanism of change, MCQ-30 total scores were significantly reduced at 12 weeks (p = 0.018) and 6 months (p = 0.028), as were three out of five subscales: negative beliefs about uncontrollability and danger (12 weeks: p = 0.018, 6 months: p = 0.026), cognitive confidence (12 weeks: p = 0.018, 6 months: p = 0.026), and negative beliefs about need to control thoughts (12 weeks: p = 0.018, 6 months: p = 0.027). There were no differences at either time-point on positive beliefs about worry (12 weeks: p = 0.172, 6 months: p = 0.068) or cognitive self-consciousness (12 weeks: p = 0.173, 6 months: p = 0.114).

Successful completion of sessional measures was high (98.75%). As can be seen in Figure 2, CAS-I worry scores generally declined over the course of therapy, which coincided with reductions in CAS-I meta-cognitive belief scores. Regression, with clustering at the participant level, suggested that the strength of metacognitive beliefs significantly predicted levels of worry across the sessional measures (β = 0.73, SE: 0.07, p < 0.001, CI: 0.58–0.89).

FIGURE 2

Figure 2. Line graph of worry and metacognitive belief scores across sessions (graphs by participant).

Discussion

Our results suggest that MCT is an acceptable treatment for young people with an At Risk Mental State, evidenced by high rates of trial uptake and therapy adherence (80% treatment adherence and an overall average of eight sessions). We observed clinically significant reductions in psychotic like experiences at the post-treatment assessment (CAARMS subscales: Non-Bizarre ideas, Perceptual abnormalities, disorganized speech), retained on one subscale (Non-Bizarre ideas) at the 6-month follow-up point. We also found an important reduction in participants meeting ARMS criteria at the post-treatment assessment (four out of eight) which was retained at the 6-month follow-up. Only one participant made transition to first-episode psychosis across the follow-up period. Trial results also showed statistically significant improvements in anxiety and depression (Hospital Anxiety and Depression Scale) and functioning (Global Assessment of Functioning) at the post-treatment assessment. The significant change in anxiety, but no other secondary outcomes, was retained at the 6-month follow-up assessment. Our effect sizes suggest that the magnitude of the differences found are considered medium to large, although this must be interpreted with caution as given the trial design and small sample the effect sizes are likely to be inflated.

Measures related to beliefs and processes targeted within MCT were assessed over time and during treatment. Results demonstrated clinically significant change on metacognitive beliefs (MCQ-30 total score, beliefs about uncontrollability, need for control and danger and cognitive confidence) at both follow-up assessment points. This was also the case for people’s self-rated beliefs about voices, on a measure (IVI) informed by a metacognitive model. We also observed statistically significant change on processes described within the CAS (worry and threat monitoring) at the post-treatment assessment, although this was not retained at 6 months. As with the previous outcomes, effect sizes on these measures demonstrated medium to large effects across assessments, although the effect sizes need to be interpreted with caution as previously described. Weekly measurement of worry and threat monitoring (recorded via CAS-1 subscales) showed that both reduced, at similar rates, throughout the course of treatment for those who were retained. These results suggest that MCT based upon a specific metacognitive model is capable of changing metacognitive beliefs and processes; which are hypothesized mechanisms within the model and therefore important treatment targets.

These positive results suggest that MCT appears to show potential in reducing psychotic like experiences, anxiety and depression, and increasing functioning for young people at UHR of psychosis. MCT is a relatively brief treatment compared with CBT, and therefore could be a useful treatment within of the contexts of resource restrictions and the importance of timely interventions for young people. It is not possible to conclude from such a small sample about who may benefit the most from CBT versus MCT, or indeed who could be recommended for either treatment. However, if a definitive trial were to replicate our findings this could provide rationale for service users to be offered choice from a range of evidence-based treatments.

We did, however, observe no statistically significant differences in subscales related to psychotic like experiences (CAARMS unusual thought content), positive beliefs about worry, cognitive self-consciousness and beliefs about paranoia. The associated effect sizes for cognitive self-consciousness and beliefs about paranoia were medium to large, small to large on the CAARMS unusual thought content subscale, so it is possible that these findings are related to a lack of statistical power. There appeared to be little effect on positive metacognitive beliefs at both assessment points. This may reflect the relative shortness of attended therapy sessions (average of 8 sessions), and given that modification of positive metacognitive beliefs takes place in the later phase of therapy, it may not have been addressed adequately within the treatment window. It may be that, for this population, modifications should be explored in the length of treatment window offered in order to address this given the documented importance of assertive outreach principles in this area (Morrison, 2017). It is also possible that the relative inexperience of the therapists meant that the efficiency normally derived from MCT was not achieved in this study. As adherence and competency measures were not taken during the trial it’s not possible to know if this is the case.

Acceptability of the MCT was high, with eight of ten participants adhering to treatment (operationalized as attendance to least four sessions). The remaining two participants withdrew early (after a single session) due to changes in life circumstances making attendance at therapy sessions difficult (physical health complications and moving out of area). We observed acceptable retention at the post-treatment assessment (80% completion) but higher rates of attrition at the 6-month follow-up (40%). The reasons for two participants not attending the follow-up assessment (described above) were unrelated to the trial procedures. However, two additional participants declined to take part in the 6-month follow-up assessment and it was not possible to explore their reasons for declining. Therefore, we cannot comment on any possible acceptability issues related to trial procedures for either of these participants. It will be important for future trials to explore the acceptability of trial procedures via qualitative interviews or feedback processes.

As would be expected in an exploratory trial of this kind there are a number of important methodological limitations that require consideration. The small sample size both reduces the statistical power and the generalizability of the findings, in part because our sample size did not meet statistical requirements (e.g., normality) required for hypothesis testing (e.g., Shader, 2015). Small sample sizes also have a higher risk of providing imprecise estimates and therefore we must be highly cautious when interpreting the meaning of these results to the wider population. However, MCT has been found to produce large effects in other groups (e.g., GAD treatment), and therefore is consistent with previous findings. Given the limitations of our study, we are limited in being able to compare our findings to those of larger studies of psychological interventions for ARMS populations, and more data on MCT for ARMS populations is required for verification. However, in line with the aims of pilot studies, we were able to examine the feasibility of processes and procedures (i.e., recruitment, retention, implementation of MCT) in preparation for a larger RCT (Leon et al., 2011).

The trial design did not allow for comparison with a control group, and the assessors were not blind to the presence of the treatment; therefore rater bias and possible non-specific treatment effects have not been guarded against Additionally, we did not complete formal ratings of treatment adherence or therapist competency. This limits any analysis of fidelity to the treatment protocol, although therapists did audio-tape sessions (where consent allowed) and receive supervision by the first author following the treatment manual previously described. A further limitation is that we did not obtain any qualitative feedback from participants on their views of the treatment, or any satisfaction scores. Nonetheless, we found significant effects on a number of outcome measures and potential mechanisms implicated within the MCT model, some of which were derived from self-rated questionnaires which showed statistically significant change with associated medium to large effects.

MCT for individuals meeting ARMS criteria warrants further attention. In the future, it will be important to conduct another pilot trial to further test of the acceptability and feasibility of offering MCT compared with treatment-as-usual under randomized conditions and with a longer follow-up period. The CAARMS assessment seems to be an appropriate outcome measure and would allow for comparison with other trials of ARMS interventions. Further qualitative work is required to explore participants’ unique experiences of the trial, and obtain their views on the appropriateness of the CAARMS as a primary outcome measure in a future definitive trial.

Ethics Statement

This study was carried out in accordance with the recommendations of A National Research Ethics Committee, North West Greater Manchester West Ethics Committee (Ref. 13/NW/0238) with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the North West Greater Manchester West Ethics Committee.

Author Contributions

SP conceived and designed the study. SP, PM, LM, and JP-C provided the therapy. SP and SB supervised the study. SP, PM, LM, and JP-C performed the data collection. JP-C analyzed the data. SP, JP-C, and LM interpreted the data. SP, LM, JP-C, and SB drafted the manuscript. All authors revised and finalized the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank the participants who contributed and the services who were referred to in the study.

References

Addington, J., Epstein, I., Liu, L., French, P., Boydell, K. M., and Zipursky, R. B. (2011). A randomized controlled trial of cognitive behavioural therapy for individuals at clinical high risk of psychosis. Schizophr. Res. 125, 54–61. doi: 10.1016/j.schres.2010.10.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Amminger, G. P., Schäfer, M. R., Papageorgiou, K., Klier, C. M., Cotton, S. M., Harrigan, S. M., et al. (2010). Long-chain ω-3 fatty acids for indicated prevention of psychotic disorders: a randomized, placebo-controlled trial. Arch. Gen. Psychiatry 67, 146–154.

Google Scholar

Barkus, E., Stirling, J., French, P., Morrison, A., Bentall, R., and Lewis, S. (2010). Distress and metacognition in psychosis prone individuals: comparing high schizoptypy to the at-risk mental state. J. Nerv. Ment. Dis. 198, 99–104. doi: 10.1097/NMD.0b013e3181cc418a

PubMed Abstract | CrossRef Full Text | Google Scholar

Bernard, M., Jackson, C., and Jones, C. (2006). Written emotional disclosure following first-episode psychosis: effects on symptoms of post-traumatic stress disorder. Br. J. Clin. Psychol. 45, 403–415. doi: 10.1348/014466505x68933

PubMed Abstract | CrossRef Full Text | Google Scholar

Bjelland, I., Dahl, A. A., Haug, T. T., and Neckelmann, D. (2002). The validity of the hospital anxiety and depression scale: an updated literature review. J. Psychosom. Res. 52, 69–77.

Google Scholar

Brett, C., Johns, L., Peters, E., and McGuire, P. (2009). The role of metacognitive beliefs in determining the impact of anomalous experiences: a comparison of help-seeking and non-help-seeking groups of people experiencing psychotic-like anomalies. Psychol. Med. 39, 939–950. doi: 10.1017/S0033291708004650

PubMed Abstract | CrossRef Full Text | Google Scholar

Bright, M., Parker, S., French, P., Morrison, A. P., Tully, S., Stewart, S., et al. (2018). Assessment of metacognitive beliefs in an at risk mental state for psychosis: a validation study of the metacognitions Questionnaire–30. Clin. Psychol. Psychother. 25, 710–720. doi: 10.1002/cpp.2301

PubMed Abstract | CrossRef Full Text | Google Scholar

Carpenter, (2018). Clinical high risk controversies and challenge for the experts, William T CarpenterSearch for other works by this author on: Oxford Academic PubMed Google Scholar. Schizophr. Bull. 44, 223–225. doi: 10.1093/schbul/sbx182

PubMed Abstract | CrossRef Full Text | Google Scholar

Clark, D. M., and Wells, A. (1995). “A cognitive model of social phobia,” in Social Phobia: Diagnosis, Assessment, and Treatment, eds R. G. Heimberg, M. R. Liebowitz, D. A. Hope, and F. R. Schneier (New York, NY: The Guilford Press), 69–93.

Google Scholar

Cotter, J., Yung, A. R., Carney, R., and Drake, R. J. (2017). Metacognitive beliefs in the at-risk mental state: a systematic review and meta-analysis. Behav. Res. Ther. 90, 25–31. doi: 10.1016/j.brat.2016.12.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Cummings, (2012). Understanding the New Statistics. New York, NY: Routledge.

Google Scholar

Debbané, M., Van der Linden, M., Balanzin, D., Billieux, J., and Eliez Stephan, M. D. (2012). Associations among metacognitive beliefs, anxiety and positive schizotypy during adolescence. J. Nerv. Ment. Dis. 200, 620–626. doi: 10.1097/NMD.0b013e31825bfc1a

PubMed Abstract | CrossRef Full Text | Google Scholar

Debbané, M., Van der Linden, M., Gex-Fabry, M., and Eliez, S. (2009). Cognitive and emotional associations to positive schizotypy during adolescence. J. Child Psychol. Psychiatry 50, 326–334. doi: 10.1111/j.1469-7610.2008.01961.x

PubMed Abstract | CrossRef Full Text | Google Scholar

French, P., and Morrison, A. P. (2004). Early Detection and Cognitive Therapy for People at High Risk of Developing Psychosis. Hoboken, NJ: Wiley.

Google Scholar

Gumley, A. I., Gillan, K., Morrison, A. P., and Schwannauer, M. (2011). The development and validation of the Beliefs about Paranoia Scale (Short Form). Behav. Cogn. Psychother. 39, 35–53. doi: 10.1017/S135246581000055X

PubMed Abstract | CrossRef Full Text | Google Scholar

Hall, R. C. (1995). Global assessment of functioning: a modified scale. Psychosomatics 36, 267–275.

PubMed Abstract | Google Scholar

Hansson, M., Chotai, J., Nordstöm, A., and Bodlund, O. (2009). Comparison of two self-rating scales to detect depression: HADS and PHQ-9. Br. J. Gen. Pract. 59, e283–e288. doi: 10.3399/bjgp09X454070

PubMed Abstract | CrossRef Full Text | Google Scholar

Hartmann, J. A., Yuen, H. P., McGorry, P. D., Yung, A. R., Lin, A., Wood, S. J., et al. (2016). Declining transition rates to psychotic disorder in “ultra-high risk” clients: investigation of a dilution effect. Schizophr. Res. 170, 130–136. doi: 10.1016/j.schres.2015.11.026

PubMed Abstract | CrossRef Full Text | Google Scholar

Health Quality Improvement Partnership and the Royal College of Psychiatrists, (2016). Report of the Early Intervention in Psychosis Audit. London: Royal College of Psychiatrists.

Google Scholar

Hutton, P., Morrison, A. P., Wardle, M., and Wells, A. (2014). Metacognitive therapy in treatment-resistant psychosis: a multiple-baseline study. Behav. Cogn. Psychother. 42, 166–185. doi: 10.1017/S1352465812001026

PubMed Abstract | CrossRef Full Text | Google Scholar

Ising, H. K., Kraan, T. C., Rietdijk, J., Dragt, S., Klaassen, R. M., Boonstra, N., et al. (2016). Four-year follow-up of cognitive behavioral therapy in persons at ultra-high risk for developing psychosis: the Dutch early detection intervention evaluation (EDIE-NL) trial. Schizophr. Bull. 42, 1243–1252. doi: 10.1093/schbul/sbw018

PubMed Abstract | CrossRef Full Text | Google Scholar

Jones, S. H., Thornicroft, G., Coffey, M., and Dunn, G. (1995). A brief mental health outcome scale: reliability and validity of the Global Assessment of Functioning (GAF). Br. J. Psychiatry 166, 654–659. doi: 10.1192/bjp.166.5.654

PubMed Abstract | CrossRef Full Text | Google Scholar

Leicester, S., Amminger, L., Phillips, L., Yuen, P., and McGorry, P. (2002). DSM-IV Axis I disorders in individuals at ultra-highrisk for psychosis. Acta Psychiatr. Scand. 106:27.

Google Scholar

Leon, A. C., Davis, L. L., and Kraemer, H. C. (2011). The role of interpretation of pilot studies in clinical research. J. Psychiatr. Res. 45, 626–629. doi: 10.1016/j.jpsychires.2010.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Lysaker, P. H., Carcione, A., Dimaggio, G., Johannesen, J. K., Nicolo, G., Procacci, M., et al. (2005). Metacognition amidst narratives of self and illness in schizophrenia: associations with insight, neurocognition, symptom and function. Acta Psychiatr. Scand. 112, 64–71. doi: 10.1111/j.1600-0447.2005.00514.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Lysaker, P. H., Gagen, E., Moritz, S., and Schweitzer, R. D. (2018). Metacognitive approaches to the treatment of psychosis: a comparison of four approaches. Psychol. Res. Behav. Manag. 11, 341–351. doi: 10.2147/PRBM.S146446

PubMed Abstract | CrossRef Full Text | Google Scholar

McGlashan, T. H., Zipursky, R. B., Perkins, D., Addington, J., Miller, T., Woods, S. W., et al. (2006). Randomized, double-blind trial of olanzapine versus placebo in patients prodromally symptomatic for psychosis. Am. J. Psychiatr. 163, 790–799. doi: 10.1176/ajp.2006.163.5.790

PubMed Abstract | CrossRef Full Text | Google Scholar

McGorry, P. D., Yung, A. R., Phillips, L. J., Yuen, H. P., Francey, S., Cosgrave, E. M., et al. (2002). Randomized controlled trial of interventions designed to reduce the risk of Progression to first-episode psychosis in a clinical sample with subthreshold symptoms. Arch. Gen. Psychiatry 59, 921–928.

PubMed Abstract | Google Scholar

Morrison, A., Gumley, A., Schwannauer, M., Campbell, M., Gleeson, A., Griffin, E., et al. (2005). The beliefs about paranoia scale: preliminary validation of a metacognitive approach to conceptualizing paranoia. Behav. Cogn. Psychother. 33, 153–164. doi: 10.1017/S1352465804001900

CrossRef Full Text | Google Scholar

Morrison, A. P. (2001). The interpretation of intrusions in psychosis: an integrative cognitive approach to hallucinations and delusions. Behav. Cogn. Psychother. 29, 257–276. doi: 10.1017/s1352465801003010

CrossRef Full Text | Google Scholar

Morrison, A. P. (2017). A manualised treatment protocol to guide delivery of evidence-based cognitive therapy for people with distressing psychosis: learning from clinical trials. Psychosis 9, 271–281. doi: 10.1080/17522439.2017.1295098

CrossRef Full Text | Google Scholar

Morrison, A. P., French, P., Parker, S., Roberts, M., Stevens, H., Bentall, R. P., et al. (2006). Three-year follow-up of a randomized controlled trial of cognitive therapy for the prevention of psychosis in people at ultrahigh risk. Schizophr. Bull. 33, 682–687. doi: 10.1093/schbul/sbl042

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., French, P., Stewart, S. L. K., Birchwood, M., Fowler, D., Gumley, A. I., et al. (2012). Early detection and intervention evaluation for people at risk of psychosis: multisite randomised controlled trial. BMJ 344:e2233. doi: 10.1136/bmj.e2233

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., French, P., Walford, L., Lewis, S. W., Kilcommons, A., Green, J., et al. (2004a). Cognitive therapy for the prevention of psychosis in people at ultra-high risk: randomised controlled trial. Br. J. Psychiatry 185, 291–297. doi: 10.1192/bjp.185.4.291

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., French, P., and Wells, A. (2007). Metacognitive beliefs across the continuum of psychosis: comparisons between patients with psychotic disorders, patients at ultra-high risk and non-patients. Behav. Res. Ther. 45, 2241–2246. doi: 10.1016/j.brat.2007.01.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., Gumley, A. I., Ashcroft, K., Manousos, I. R., White, R., Gillan, K., et al. (2011). Metacognition and persecutory delusions: tests of a metacognitive model in a clinical population and comparisons with non-patients. Br. J. Clin. Psychol. 50, 223–233. doi: 10.1348/014466510X511141

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., Nothard, S., Bowe, S. E., and Wells, A. (2004b). Interpretations of voices in patients with hallucinations and non-patient controls: a comparison and predictors of distress in patients. Behav. Res. Ther. 42, 1315–1323. doi: 10.1016/j.brat.2003.08.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., Pyle, M., Chapman, N., French, P., Parker, S. K., and Wells, A. (2014). Metacognitive therapy in people with a schizophrenia spectrum diagnosis and medication resistant symptoms: a feasibility study. J. Behav. Ther. Exp. Psychiatry 45, 280–284. doi: 10.1016/j.jbtep.2013.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., Shryane, N., Fowler, D., Birchwood, M., Gumley, A. I., Taylor, H. E., et al. (2015). Negative cognition, affect, metacognition and dimensions of paranoia in people at ultra-high risk of psychosis: a multi-level modelling analysis. Psychol. Med. 45, 2675–2684. doi: 10.1017/s0033291715000689

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., and Wells, A. (2003). A comparison of metacognitions in patients with hallucinations, delusions, panic disorder, and non-patient controls. Behav. Res. Ther. 41, 251–256. doi: 10.1016/s0005-7967(02)00095-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., and Wells, A. (2007). Relationships between worry, psychotic experiences and emotional distress in patients with schizophrenia spectrum diagnoses and comparisons with anxious and non-patient groups. Behav. Res. Ther. 45, 1593–1600. doi: 10.1016/j.brat.2006.11.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A. P., Wells, A., and Nothard, S. (2002). Cognitive and emotional predictors of predisposition to hallucinations in non-patients. Br. J. Clin. Psychol. 41, 259–270. doi: 10.1348/014466502760379127

PubMed Abstract | CrossRef Full Text | Google Scholar

Nhs England, the National Collaborating Centre for Mental Health, and the National Institute for Health, and Care Excellence, (2016). Implementing the Early Intevrention in Psychosis Access and Waiting Time Standard: Guidance. Ra’anana: NICE.

Google Scholar

Normann, N., and Morina, N. (2018). The efficacy of metacognitive therapy: a systematic review and meta-analysis. Front. Psychol. 9:2211. doi: 10.3389/fpsyg.2018.02211

PubMed Abstract | CrossRef Full Text | Google Scholar

Palmier-Claus, J. E., Dunn, G., Taylor, H., Morrison, A. P., and Lewis, S. W. (2013). Cognitive-self consciousness and metacognitive beliefs: stress sensitization in individuals at ultra-high risk of developing psychosis. Br. J. Clin. Psychol. 52, 26–41. doi: 10.1111/j.2044-8260.2012.02043.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Papageorgiou, C., and Wells, A. (2001). Metacognitive beliefs about rumination in recurrent major depression. Cogn. Behav. Pract. 8, 160–164. doi: 10.1111/sjop.12089

PubMed Abstract | CrossRef Full Text | Google Scholar

Pyle, M., Broome, M. R., Joyce, E., MacLennan, G., Norrie, J., Freeman, D., et al. (2019). Study protocol for a randomised controlled trial of CBT vs antipsychotics vs both in 14-18 year olds: managing adolescent first episode psychosis a feasibility study (MAPS). Trials 20:395. doi: 10.1186/s13063-019-3506-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Rogers, W. H. (1993). Regression standard errors in clustered samples. Stata Tech. Bull. 13, 19–23.

Google Scholar

Sellers, R., Wells, A., Parker, S., and Morrison, A. P. (2018). Do people with psychosis engage in unhelpful metacognitive coping strategies? a test of the validity of the Cognitive Attentional Syndrome (CAS) in a clinical sample. Psychiatr. Res. 259, 243–250. doi: 10.1016/j.psychres.2017.10.032

PubMed Abstract | CrossRef Full Text | Google Scholar

Shader, R. I. (2015). Proof of feasibility: what a pilot study Is and Is Not. Clin. Ther. 37, 1379–1380. doi: 10.1016/j.clinthera.2015.06.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Smári, J., Stefánsson, S., and Thorgilsson, H. (1994). Paranoia, self-consciousness, and social cognition in schizophrenics. Cogn. Ther. Res. 18:387. doi: 10.1007/bf02357512

CrossRef Full Text | Google Scholar

Stafford, M. R., Jackson, H., Mayo-Wilson, E., Morrison, A. P., and Kendall, T. (2013). Early interventions to prevent psychosis: systematic review and meta-analysis. BMJ 346:f185. doi: 10.1136/bmj.f185

PubMed Abstract | CrossRef Full Text | Google Scholar

Stata Corporation, (2011). Stata Statistical Software: Release 12. Texas: College Station.

Google Scholar

van der Gaag, M., Nieman, D. H., Rietdijk, J., Dragt, S., Ising, H. K., Klaassen, R. M., et al. (2012). Cognitive behavioral therapy for subjects at ultrahigh risk for developing psychosis: a randomized controlled clinical trial. Schizophr. Bull. 38, 1180–1188. doi: 10.1093/schbul/sbs105

PubMed Abstract | CrossRef Full Text | Google Scholar

Wells, A. (1995). Meta-cognition and worry: a cognitive model of generalized anxiety disorder. Behav. Cogn. Psychother. 23, 301–320. doi: 10.1017/s1352465800015897

CrossRef Full Text | Google Scholar

Wells, A. (2000). Emotional Disorders and Metacognition: Innovative Cognitive Therapy. Chichester: Wiley.

Google Scholar

Wells, A. (2008). Metacognitive therapy: cognition applied to regulating cognition. Behav. Cogn. Psychother. 36, 651–658. doi: 10.1016/j.psychres.2016.07.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Wells, A. (2009). Metacognitive Therapy for Anxiety and Depression. New York, NY: Guilford press.

Google Scholar

Wells, A., and Cartwright-Hatton, S. (2004). A short form of the metacognitions questionnaire: properties of the MCQ-30. Behav. Res. Ther. 42, 385–396. doi: 10.1016/s0005-7967(03)00147-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Wells, A., and King, P. (2006). Metacognitive therapy for generalized anxiety disorder: an open trial. J. Behav. Ther. Exp. Psychiatry 37, 206–212. doi: 10.1016/j.jbtep.2005.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Wells, A., and Matthews, G. (1994). Attention and Emotion. A Clinical Perspective. Hove: Erlbaum.

Google Scholar

Wells, A., and Matthews, G. (1996). Modelling cognition in emotional disorder: the S-REF model. Behav. Res. Ther. 34, 881–888. doi: 10.1016/s0005-7967(96)00050-2

PubMed Abstract | CrossRef Full Text | Google Scholar

White, R., Gumley, A., McTaggart, J., Rattrie, L., McConville, D., Cleare, S., et al. (2011). A feasibility study of acceptance and commitment therapy for emotional dysfunction following psychosis. Behav. Res. Ther. 49, 901–907. doi: 10.1016/j.brat.2011.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Yung, A. R., Pan Yuen, H., McGorry, P. D., Phillips, L. J., Kelly, D., and Stanford, C. (2005). Mapping the onset of psychosis: the comprehensive assessment of at-risk mental states. Aust. N. Z. J. Psychiatry 39, 964–971. doi: 10.1080/j.1440-1614.2005.01714.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Yung, A. R., Phillips, L. J., McGorry, P. D., McFarlane, C. A., Francey, S., and Jackson, H. J. (1998). Prediction of psychosis. a step towards indicated prevention of schizophrenia. Br. J. Psychiatry 172, 14–20. doi: 10.1192/s0007125000297602

CrossRef Full Text | Google Scholar

Yung, A. R., Phillips, L. J., Nelson, B., Francey, S. M., Yuen, H. P., Simmons, M. B., et al. (2011). Randomized controlled trial of interventions for young people at ultra high risk for psychosis: 6-month analysis. J. Clin. Psychiatry 72;430. doi: 10.4088/jcp.08m04979ora

PubMed Abstract | CrossRef Full Text | Google Scholar

Zigmond, A. S., and Snaith, R. P. (1983). The hospital anxiety and depression scale. Acta Psychiatr. Scand. 67, 361–370.

PubMed Abstract | Google Scholar