What Comes First Metacognition or Negative Emotion? A Test of Temporal Precedence

The Self-Regulatory Executive Function model predicts that emotional symptoms and metacognition can causally affect each other. Crucially, for the model metacognition must cause emotion disorder symptoms. Therefore, in time-series data involving repeated measurements, metacognitions should predict subsequent changes in emotion. 265 participants completed a questionnaire battery three times over a 2 month period. Structural equation modeling (SEM) using cross-lagged panel analysis tested the inter-relationships between metacognitive beliefs, anxiety and depression symptoms over time. The cross-lagged structural model was a significantly better fit than the autoregressive model. Metacognitive beliefs were found to predict subsequent symptoms of anxiety while symptoms of anxiety predicted later metacognition over different time courses. The metacognition factor representing uncontrollability and danger of thoughts appeared to be prominent in the effects observed. Metacognitions and depression were also positively related over time to a lesser degree, but in the cross-lagged model these temporal relationships were non-significant. This is likely due to low levels of depression within the sample and low variability over time. The findings for anxiety are consistent with the S-REF model and with experimental and prospective studies supporting metacognitive beliefs as a causal mechanism in psychological distress symptoms.

Introduction

A crucial question in formulating the role of metacognitive factors in emotional symptoms concerns whether or not these factors have a causal or contributory role or merely represent an effect of such dysfunction. The Self-regulatory executive function model (S-REF; Wells and Matthews, 1994, 1996) proposes that specific metacognitions increase emotional dysfunction by, for example, interacting with environmental factors and giving rise to a pattern of extended negative thinking in response to stress. Thus, metacognition should precede symptoms in causal time-series data. Never the less, the model also allows for reciprocal causation, in which emotion can also impact on metacognition. For example, some anxiety or mood symptoms may impair cognitive capacity or be interpreted as a sign of loss of mental functioning thereby strengthening metacognitions of lack of control. A pattern of temporal relationships not consistent with the model would occur if negative emotional symptoms only gave rise to later dysfunctional metacognitions, a result that would diminish the causal status of metacognition and present a challenge to the model.

The S-REF model proposes that psychological distress (e.g., anxiety or depression symptoms) is associated with the activation of a style of thinking called the cognitive attentional syndrome (CAS). The CAS is a state of perseverative negative thinking comprised of worry, rumination, focusing on threat, and other maladaptive coping strategies that inadvertently intensify and prolong emotion responses. The CAS is hypothesized to result from metacognitions which exist in the form of knowledge, experiences, and strategies. Such components direct attention, determine thinking style and coping in response to stress cognitions and challenges (Wells, 2009). Metacognitive knowledge is relatively stable and refers to the beliefs that individuals hold about their thinking and can be categorized into positive and negative content. Positive metacognitive beliefs concern the usefulness of cognitive activities that constitute the CAS, e.g., “If I worry, I will be prepared,” while negative metacognitive beliefs concern the uncontrollability, dangerousness and importance of thoughts, e.g., “I cannot control my thinking.” Such metacognitions, especially negative beliefs are thought to impact on emotion regulation by biasing control efforts leading to perseveration of negative thinking with the effect of increasing or extending negative emotions.

A large number of studies have now demonstrated that the metacognitions predicted by the model are associated with stress symptoms, anxiety or depression (e.g., Wells and Papageorgiou, 1995; Roussis and Wells, 2008; Bennett and Wells, 2010; Yılmaz et al., 2011; Hjemdal et al., 2013; O’Carroll and Fisher, 2013; Halvorsen et al., 2015; Bailey and Wells, 2016; Fergus and Bardeen, 2016; Capobianco et al., 2018a, b). For example, Takarangi et al. (2017) conducted a longitudinal study evaluating whether metacognitive beliefs and metamemory beliefs were associated with the development and maintenance of post-traumatic stress disorder. They found that metacognitive beliefs predicted severity of PTSD symptoms after exposure to a trauma, and the maintenance of PTSD symptoms over time (time 1 to time 2).

Results from experimental manipulations of metacognitive beliefs support a causal role in negative emotion symptoms. Myers and Wells (2013) experimentally manipulated thought-event fusion beliefs (a specific type of metacognitive belief) using a fake-EEG paradigm in individuals with high and low obsessions. They found that inducing such beliefs led to OCD-like symptomology, with this effect being strongest in those with pre-existing high levels of obsessions. Capobianco et al. (2018b) also conducted an experimental manipulation of metacognitive beliefs using a similar fake-EEG paradigm. They evaluated if manipulating the belief of thought importance impacted on physiological and subjective responses to induced stress. Individuals in the experimental condition showed higher levels of negative affect and lower levels of positive affect in response to stress and maintained low positive affect at recovery. In addition to metacognitive beliefs, metacognitive strategies have also been shown to prospectively predict traumatic stress symptoms (Holeva et al., 2001; Roussis and Wells, 2008), anxiety or depression (Yılmaz et al., 2011).

Following from the S-REF model and the results demonstrating an effect of metacognitions we tested the hypothesis that metacognitive beliefs would positively predict later psychological distress measured as anxiety or depression symptoms. We did so using Structural equation modeling (SEM) as this framework allows for the use of the time ordered nature of panel data to address questions of causal orderings (Berrington et al., 2006).

Materials and Methods

Participants

For purposes of this study data from two samples were combined, in order to provide a sample size sufficient for SEM. Sample size and power calculations for SEM can be challenging (Wolf et al., 2013). Guidelines for SEM sample size varies; it has been suggested that a minimum sample size of 100–200 participants is required (Boomsma, 1982, 1985), but other suggestions include 5 or 10 participants per estimated parameter (Bentler and Chou, 1987), or 10 participants per variable (Nunnally, 1967). Therefore, based on the above recommendations as well as a recent evaluations using Monte Carlo simulations of sample size estimates based on model fit, suggesting a sample size of 250–300 participants (Westland, 2010; Wolf et al., 2013), we opted to combine samples from two sources to provide a sufficient sample size to conduct SEM. Two-hundred and sixty-six participants completed a questionnaire battery. In sample 1, participants (n = 150) were recruited from the University of Manchester. In sample 2, participants (n = 115) were recruited from both the University of Manchester and an online crowdsourcing website. Both samples used the same inclusion criteria; participants had to be at least 18 years of age and proficient in English. Participants ages ranged from 18 to 74 (M = 25.99, SD = 10.64). The sample was primarily female (213 women, 52 men). All participants from both studies completed the study using an online questionnaire software (SelectSurvey.Net). Both studies that provided data were approved by the University of Manchester Research Ethics Committee, reference 15286 (study 1) and reference 2017-2286-3683 (study 2).

Measures

Hospital Anxiety and Depression Scale (HADS; Zigmond and Snaith, 1983). The HADS is a 14-item measure with two subscales; anxiety and depression. A total score can also be calculated by summating all items. Items are rated using a 4 point likert scale, where higher scores indicate greater anxiety and depression. Subscales demonstrate good internal consistency, with alpha reliabilities of 0.80 for anxiety and 0.81 for depression (Bjelland et al., 2002) and 0.86 for the total scale (Crawford et al., 2001). The scale demonstrates good reliability and validity (Herrmann, 1997; Bjelland et al., 2002).

Meta-cognitions Questionnaire 30 (MCQ-30; Wells and Cartwright-Hatton, 2004). The MCQ-30 assesses metacognitive beliefs implicated by the S-REF model as linked to psychological vulnerability. The scale has five subscales: positive metacognitive beliefs about worry (e.g., “Worrying helps me to solve problems”), negative metacognitive beliefs about uncontrollability and danger (e.g., “When I start worrying, I cannot stop”), cognitive confidence (e.g., “I have a poor memory”), cognitive self-consciousness (e.g., “I pay close attention to the way my mind works”), and need for control (e.g., “It is bad to think certain thoughts”). Responses are scored on a scale ranging from 1 (do not agree) to 4 (agree very much). The scale demonstrates good convergent validity, internal consistency, and acceptable test–retest reliability (Wells and Cartwright-Hatton, 2004; Spada et al., 2008; Yılmaz et al., 2008).

Procedure

After expressing an interest in the study participants received a link to a web site (SelectSurvey.Net) containing the participant information sheet and consent form. Following consent they were able to access the questionnaires. The questionnaire battery was distributed three times within a 2 month period. A 2 month interval was selected as this has clinical relevance; within 1 month stress symptoms normally begin to decrease, however, if they persist longer it could be indicative of a chronic or delayed stress response (deRoon-Cassini et al., 2010), therefore this interval allowed us to investigate the short and long term effects of stress within a meaningful clinical time-frame. Questionnaires were administered at day 0, day 30, and day 60.

Statistical Analysis Plan

Statistical analyses were conducted in two steps: (1) first we examined invariance of factors over time to ensure we could include the MCQ factors in cross-lagged panel analysis, (2) we then estimated a cross-lagged panel model.

Structural equation modeling was conducted using AMOS for SPSS v 0.23 (Arbuckle, 2014) which uses the maximum likelihood (ML) method to evaluate model fit to the corresponding observed variance-covariance matrices. Model fit was evaluated using a range of fit indices including: the comparative fit index (CFI), Root Mean Square Error of Approximation (RMSEA), Standardized Root Mean Square Residual (SRMR), and Tucker Lewis Index (TLI). The following thresholds were used to assess a good model fit: CFI ≥ 0.90, RMSEA ≤ 0.06, and TLI ≥ 0.95 (Hu and Bentler, 1995), and SRMR ≤ 0.10 (Kline, 2005).

Latent Variable Identification

In SEM, the relationships between latent variables and between latent and observed measures can be evaluated (Bollen and Noble, 2011). As latent variables cannot be directly observed, they are modeled by specifying the observed, directly measurable variables that express the underlying construct. Latent and observed variables were specified a priori. Metacognitive beliefs were constructed as a latent variable to allow us to evaluate the contribution of individual subscales over time while anxiety and depression were modeled as observed variables in a single model that offered the potential of controlling overlaps between anxiety and depression symptoms at each time point and any temporal relationships between these symptoms.

Hospital Anxiety and Depression

The HADS was modeled using the corresponding HADS subscales (anxiety and depression), as suggested by the original psychometric analysis of the scale (Zigmond and Snaith, 1983). Each subscale was treated as an observed variable rather than a latent variable as we were interested in separate overall measures of anxiety and depression rather than the contribution of the individual items to a latent general factor. The Cronbach’s alpha of the anxiety subscale was 0.81 and for the depression subscale it was 0.75.

Metacognitive Beliefs

The metacognition latent variable was modeled using the items corresponding to the five subscales of the MCQ-30, which is consistent with Wells and Cartwright-Hatton (2004). The scales demonstrated good internal consistency. Cronbach’s alpha for the subscales for the current study were as follows: positive metacognitive beliefs = 0.87, negative metacognitive beliefs regarding uncontrollability and danger = 0.88, cognitive confidence = 0.89, need for control = 0.80, cognitive self-consciousness = 0.85.

Cross-Lagged Model Hypotheses

We ran 3-wave structural equation models using ML estimation within AMOS V23 to investigate the longitudinal relationships between metacognitive beliefs and emotional states i.e., anxiety and depression. We began with a basic auto-regressive model, in which the latent variables have a directional effect only on themselves (Figure 1). The autoregressive model is the simplest model and acts as a reference against which to compare more complex models. We used a standard formulation of this model, in which MCQ-30 (latent variable) and anxiety and depression (observed variables) are inter-correlated within time points. In addition, the errors on individual variables that model the latents (e.g., MCQ-30 subscales) are assumed to be correlated across time-points. Inclusion of correlated errors followed the suggestion by Fornell (1983), in that inclusion is theoretically driven. We then tested the robustness of this model to violations of the assumptions before proceeding further.

FIGURE 1

Figure 1. Autoregressive Model of Metacognitive Beliefs, Anxiety and Depression: standardized estimates. Solid line, significant path; dotted line, non-significant path; Pos, Positive Metacognitive Beliefs, Neg, Negative Metacognitive Beliefs Regarding Uncontrollability and Danger; CC, Cognitive Confidence; NC, Need for Control; CSC, Cognitive Self Consciousness.

Having established an appropriate autoregressive model, we next tested our hypotheses about the relationship of metacognitive beliefs to anxiety and depression through cross lagged panel analyses. Cross-lagged panel models control for contemporaneous and autocorrelations while identifying time-lagged reciprocal effects of constructs assessed repeatedly. We also accounted for the cross-lagged paths between anxiety and depression, which allowed us to evaluate and control any prospective relationships between anxiety and depression in testing if metacognition can prospectively predict anxiety or depression.

Results

Data Descriptives

Two-hundred and sixty-Five participants completed the study at all three time points. As less than 10% of the data was missing, mean values were imputed for missing data. Means and standard deviations of the questionnaires across time points are reported in Table 1.

TABLE 1

Table 1. Descriptive Statistics for each measure at each test interval.

Pearson’s correlations (Table 2) were computed to evaluate the pattern of relationships between measures. All metacognitive beliefs were moderately to strongly positively correlated with anxiety and depression symptoms over time.

TABLE 2

Table 2. Pearson Correlations Between Variables.

Measurement Invariance

Measurement invariance was evaluated using the four invariance steps (configural, metric, scalar, and residual) as described by Putnick and Bornstein (2016), which coincides with those previously outlined by Widaman and Reise (1997), and Vandenberg and Lance (2000). The structure of the metacognition variable was evaluated across three time points, where we compared (a) an unconstrained model where all factor loadings and intercepts were allowed to vary freely, (b) a metric invariance model, where factor loadings were constrained equal, (c) a structural invariance model where the factor variances and covariance’s were also constrained equal, and (d) a residual invariance model where the residuals of the observed variables were also constrained equal. Measurement invariance was met for the first three steps but not for the final residual invariance model. As the item residuals are not used in the interpretation of mean differences between the latent variables, this step was not strictly necessary to show measurement invariance in this case, but was included for completeness (Vandenberg and Lance, 2000). For this reason further investigations into which residual (s) differed between the two groups were not conducted. The results of the measurement invariance analysis support the interpretation of the subsequent cross-lagged analysis as probably not unduly influenced by instability in measurement models.

Model Testing

To evaluate if metacognitive beliefs might be a causal mechanism of anxiety and depression over time, cross-lagged panel models were used.

The initial autoregressive model (Figure 1; χ² = 397.09, df = 165, p < 0.001) demonstrated adequate fit to the data, as the CFI and SRMR values were within the cut-offs for good fit, however, the RMSEA and TLI values were slightly above the cut-offs for good fit; CFI of 0.95, RMSEA of 0.07, SRMR of 0.08, and TLI of 0.93.

We then evaluated the full cross-lagged model (Figure 2; χ² = 362.36 df = 153, p < 0.001), with cross-lagged paths from metacognition T1 to HADS anxiety T2 and HADS depression T2, from metacognition T2 to HADS anxiety T3 and HADS depression T3, from HADS anxiety T1 to metacognition T2, from HADS anxiety T2 to metacognition T3, from HADS depression T1 to metacognition T2, and from HADS depression T2 to metacognition T3. We also accounted for the causal associations between anxiety and depression across time, as such cross-lagged paths between HADS depression and anxiety across time-points were included. Correlations within time points between anxiety, depression, and metacognitive beliefs were also accounted for. These cross-sectional associations are not depicted in Figure 2 in order to increase legibility of the figure, however, they were included in the analysis. The cross-lagged paths significantly improved the goodness of fit (Δχ² = 34.73, df = 12, p < 0.001; RMSEA = 0.07; CFI = 0.95, SRMR = 0.06; TLI = 0.94).

FIGURE 2

Figure 2. Cross Lagged Panel Model of Metacognitive Beliefs, Anxiety and Depression: standardized estimates. Solid line, significant path; dotted line, non-significant path; Pos, Positive Metacognitive Beliefs, Neg, Negative Metacognitive Beliefs Regarding Uncontrollability and Danger; CC, Cognitive Confidence; NC, Need for Control; CSC, Cognitive Self Consciousness; intercorrelations between anxiety, depression, and metacognitive beliefs were not included in the figure to increase legibility of the figure, however, are included in the analysis.

Metacognition was a positive and significant predictor of subsequent metacognitive beliefs (T1-T2: β = 0.81, p < 0.001, T2-T3: β = 0.76, p < 0.001). Similarly, anxiety was a positive and significant predictor of subsequent anxiety (T1-T2:β = 0.74, p < 0.001, T2-T3:,β = 0.73, p < 0.001) and depression predicted later depression (T1-T2:β = 0.68, p < 0.001, T2-T3:β = 0.71, p < 0.001. These results highlight the stability in metacognitive beliefs, anxiety, and depression over the testing intervals.

To evaluate if metacognitive beliefs predicted subsequent anxiety and depression and if the converse relationships applied, we examined cross-lagged regression parameters as follows:

The path from Anxiety T1 to Metacognition T2 was not significant,β = 0.04, p = 0.53, however, the path from Anxiety T2 to Metacognition T3 was significant, with a small beta:β = 0.19, p = 0.006. The finding that anxiety at T2 is predictive of subsequent metacognition is not surprising and is consistent with the metacognitive model as anxious thoughts and emotion can theoretically give rise to metacognition as previously described. However, of greater importance for the theoretical model is the path from metacognition to anxiety, here we found that metacognition at time 1 was predictive of subsequent anxiety with a small beta:β = 0.19, p = 0.008. Although, the path from metacognition at T2 to anxiety at time 3 was not significant,β = 0.0.06, p = 0.34. The beta coefficients for the paths from metacognition at T1 to anxiety at T2, and from anxiety T2 to metacognition are similar in magnitude, which raises the possibility of reciprocal causation. The result is consistent with the hypothesis that metacognitions can precede and predict negative emotion expressed as anxiety symptoms.

Depression T1 did not predict metacognition at T2 (β = −0.01, p = 0.82), nor did depression T2 predict metacognition T3 (β = 0.07, p = 0.23). Similarly, metacognition did not predict subsequent depression, metacognition T1 to depression T2 (β = 0.04, p = 0.61), and metacognition T2 to depression T3 (β = −0.02, p = 0.71). This result is not consistent with the metacognitive model applied to depression symptoms in the current sample. But it is unsurprising given that the sample had low levels of depression symptoms over time with little variation.

We also evaluated the temporal relations between anxiety and depression given that anxiety and depression commonly co-occur and may cause each other. Depression T1 did not predict anxiety at T2 (β = 0.04, p = 0.43), nor did depression T2 predict anxiety at T3 (β = −0.08, p = 0.11). While anxiety T1 did not predict depression at T2 (β = 0.03, p = 0.61), anxiety T2 did predict depression at T3 (β = 0.13, p = 0.03).

Discussion

The current study evaluated if metacognitive beliefs prospectively predicted psychological distress symptoms measured as anxiety and/or depression. We found evidence of temporal precedence and reciprocal causation over different time lags in anxiety and the data suggested that metacognitions might be the more reliable predictor of anxiety symptoms than the converse. However, the observed effect of anxiety on metacognitions, suggests some reciprocity in these temporal relationships which is consistent with S-REF theory. However, these relationships are only indicative, we cannot rule out the possible influence of other variables that may be acting on both metacognition and symptoms. A more robust test of causal relations would require direct manipulations of metacognition and emotion in evaluating their respective causal effects.

The results for depression were different and did not appear to support any causal relationship between metacognition and mood symptoms. The results are inconsistent with other studies that show prospective relationships (Fergus and Bardeen, 2016; Ryum et al., 2017; Takarangi et al., 2017) and the prospective bivariate associations found in the current study. However, such studies and our bivariate analyses have not controlled for autoregressive and contemporaneous effects and the relationships observed may have been inflated by these factors. It is likely that the failure to find a cross-lagged relationship in the current study was impacted by the low-level of depression and lack of variability in these symptoms across time in the sample. Alternatively, it may be that the MCQ is less specific for assessing metacognitions associated with depression than with anxiety. None the less, previous studies have found that metacognitive beliefs are positively correlated with symptoms of depression and rumination both cross-sectionally (Papageorgiou and Wells, 2003; Halvorsen et al., 2015; Yılmaz et al., 2015; Huntley and Fisher, 2016) and longitudinally (Yılmaz et al., 2011).

The results add to a corpus of research supporting the idea that specific metacognitions may have a causal effect on emotion symptoms. For example, Capobianco et al. (2018b) demonstrated that induction of a metacognitive belief concerning the importance of thoughts impacted reactions to and recovery from stress exposure. For anxiety at least, the bivariate correlations in the present data set suggest that metacognitive beliefs concerning uncontrollability and danger have the strongest correlations with symptoms cross-sectionally and longitudinally, followed by metacognitions concerning “need for control” and cognitive self-consciousness. The relative strength of relationships is supported by the loadings of subscales on the latent metacognition factor where uncontrollability and need for control are the strongest contributors in the model. These findings are consistent with theory and meta-analyses, mainly of cross-sectional data, demonstrating a contribution from these metacognition domains in particular.

The limitations of the current study should, however, be considered when interpreting the findings. First, the study was primarily conducted in undergraduate students which limits the generalizability of findings, and the preponderance of women in the sample does not facilitate any examination or control of sex differences. The study did not evaluate the impact of environmental or additional factors that may influence the relationship between metacognition and symptoms and so it remains a preliminary and rudimentary test. The timescale of relationships between metacognition and emotion must also be considered. We included a time period that appeared to have clinical relevance based on the trajectory of stress responses linked more to anxiety (i.e., acute stress and PTSD) but the timescale may not be appropriate for other emotional responses to develop and remit (e.g., depression symptoms). The HADS scores were mainly below clinical cut-offs, limiting any testing of effects that might be more relevant to clinical populations. As such, further research is required to evaluate the replicability of models and pattern of results within clinical samples.

Research evaluating the temporal relationships of metacognitions and symptoms of distress are required in both clinical and non-clinical populations to determine the dynamic temporal relationships between these variables. It is recommended that such studies examine a range of time-frames with a greater number of measurement panels. They should also consider the possibility that some metacognitions may precede emotion effects whilst others may maintain symptoms leading to an increase in recovery time, especially following stress-exposure.

In conclusion, despite the limitations of the current study, the results are consistent with a pattern of temporal relationships between metacognition and anxiety that is consistent with the S-REF model. The results for depression symptoms are inconclusive and most probably affected by floor effects in the data, but an implication of the anxiety result is that anxiety might be prevented by interventions that modify specific dysfunctional metacognitive beliefs.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by University of Manchester Research Ethics Committee, reference 15286 (study 1) and reference 2017-2286-3683 (study 2). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

All authors have contributed to the manuscript revision, read and approved the submitted version. LC was responsible for manuscript writing, data collection, entry, and analysis. CH contributed to data analysis and manuscript writing. MB was responsible for data collection and data entry. AW was responsible for study oversight and manuscript writing. AW and LC contributed to the conception and design of the study.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer GC declared a past collaboration with one of the authors AW to the handling Editor.

References

Arbuckle, J. L. (2014). Amos (Version 23.0) [Computer Program]. Chicago: IBM SPSS.

Google Scholar

Bailey, R., and Wells, A. (2016). Is metacognition a causal moderator of the relationship between catastrophic misinterpretation and health anxiety? Prospect. Stud. Behav. Res. Ther. 78, 43–50. doi: 10.1016/j.brat.2016.01.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Bennett, H., and Wells, A. (2010). Metacognition, memory disorganization and rumination in posttraumatic stress symptoms. J Anxiety Disord. 24, 318–325. doi: 10.1016/j.janxdis.2010.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Bentler, P. M., and Chou, C. H. (1987). Practical issues in structural modeling. Sociol. Methods Res. 16, 78–117. doi: 10.1177/0049124187016001004

CrossRef Full Text | Google Scholar

Berrington, A., Smith, P. W. F., and Sturgis, P. (2006). An Overview of Methods for the Analysis of Panel Data. ESRC NCRM Methods Review Papers. NCRM/007. Southampton: University of Southampton.

Google Scholar

Bjelland, I., Dahl, A. A., Haug, T. T., and Neckelmann, D. (2002). The validity of the hospital anxiety and depression scale an updated literature review. J. Psychol. Res. 52, 69–77. doi: 10.1016/s0022-3999(01)00296-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Bollen, K. A., and Noble, M. D. (2011). Structural equation models and the quantification of behavior. Proc. Natl. Acad. Sci. U.S.A. 108, 15639–15646. doi: 10.1073/pnas.1010661108

PubMed Abstract | CrossRef Full Text | Google Scholar

Boomsma, A. (1982). “Robustness of LISREL against small sample sizes in factor analysis models,” in Systems Under Indirection Observation: Causality, Structure, Prediction (Part I), eds K. G. Joreskog, and H. Wold, (Amsterdam: North Holland), 149–173.

Google Scholar

Boomsma, A. (1985). Nonconvergence, improper solutions, and starting values in LISREL maximum likelihood estimation. Psychometrika 50, 229–242. doi: 10.1007/bf02294248

CrossRef Full Text | Google Scholar

Capobianco, L., Morris, J. A., and Wells, A. (2018a). Worry and rumination: do they prolong physiological and affective recovery from stress? Anxiety Stress Coping 31, 291–303. doi: 10.1080/10615806.2018.1438723

PubMed Abstract | CrossRef Full Text | Google Scholar

Capobianco, L., Morrison, A. P., and Wells, A. (2018b). The effect of thought importance on stress responses: a test of the metacognitive model. Stress 21, 128–135. doi: 10.1080/10253890.2017

PubMed Abstract | CrossRef Full Text | Google Scholar

Crawford, J. R., Henry, J. D., Crombie, C., and Taylor, E. P. (2001). Normative data for the HADS from a large non-clinical sample. Br. J. Clin. Psychol. 40, 429–434. doi: 10.1348/014466501163904

PubMed Abstract | CrossRef Full Text | Google Scholar

deRoon-Cassini, T. A., Mancini, A. D., Rusch, M. D., and Bonanno, G. A. (2010). Psychopathology and resilience following traumatic injury: a latent growth mixture model analysis. Rehabil. Psychol. 55, 1–11. doi: 10.1037/a0018601

PubMed Abstract | CrossRef Full Text | Google Scholar

Fergus, T. A., and Bardeen, J. R. (2016). Examining the incremental contribution of metacognitive beliefs beyond content-specific beliefs in relation to posttraumatic stress in a community sample. Psychol. Trauma. 9, 723–730. doi: 10.1037/tra0000247

PubMed Abstract | CrossRef Full Text | Google Scholar

Fornell, C. (1983). Issues in the application of covariance structure analysis: a comment. J. Consumer Res. 9, 443–447.

Google Scholar

Halvorsen, M., Hagen, R., Hjemdal, O., Eriksen, M. S., Sørli, ÅJ., Waterloo, K., et al. (2015). Metacognition and thought control strategies in unipolar major depression: a comparison of currently depressed, previously depressed, and never- depressed individuals. Cogn. Ther. Res. 39, 31–40. doi: 10.1007/s10608-014-9638-4

CrossRef Full Text | Google Scholar

Herrmann, C. (1997). International experiences with the hospital anxiety and depression scale — a review of validation data and clinical results. J. Psychos. Res. 42, 17–41. doi: 10.1016/s0022-3999(96)00216-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Hjemdal, O., Stiles, T., and Wells, A. (2013). Automatic thoughts and meta-cognition as predictors of depressive or anxious symptoms: a prospective study of two trajectories. Scand. J. Psychol. 54, 59–65. doi: 10.1111/sjop.12010

PubMed Abstract | CrossRef Full Text | Google Scholar

Holeva, V., Tarrier, N., and Wells, A. (2001). Prevalence and predictors of acute stress disorder and PTSD following road traffic accidents: thought control strategies and social support. Behav. Ther. 32, 65–83. doi: 10.1016/s0005-7894(01)80044-7

CrossRef Full Text | Google Scholar

Hu, L., and Bentler, P. M. (1995). “Evaluating model fit,” in Structural Equation Modeling: Issues, Concepts, And Applications, ed. R. H. Hoyle, (Thousand Oaks, CA: Sage), 76–99.

Google Scholar

Huntley, C. D., and Fisher, P. L. (2016). Examining the role of positive and negative metacognitive beliefs in depression. Scand. J. Psychol. 57, 446–452. doi: 10.1111/sjop.12306

PubMed Abstract | CrossRef Full Text | Google Scholar

Kline, R. B. (2005). Principles and Practice of Structural Equation Modeling. New York, NY: Guilford Press.

Google Scholar

Myers, S., and Wells, A. (2013). An experimental manipulation of metacognition: a test of the metacognitive model of obsessive-compulsive symptoms. Behav. Res. Ther. 51, 177–184. doi: 10.1016/j.brat.2013.01.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Nunnally, J. C. (1967). Psychometric Theory. New York, NY: McGraw-Hill.

Google Scholar

O’Carroll, P. J., and Fisher, P. (2013). Metacognitions, worry and attentional control in predicting OSCE performance test anxiety. Med. Educ. 47, 562–568. doi: 10.1111/medu.12125

PubMed Abstract | CrossRef Full Text | Google Scholar

Papageorgiou, C., and Wells, A. (2003). An empirical test of a clinical metacognitive model of rumination and depression. Cogn. Ther. Res. 27, 261–273. doi: 10.1023/A:1023962332399

CrossRef Full Text | Google Scholar

Putnick, D. L., and Bornstein, M. H. (2016). Measurement invariance conventions and reporting: the state of the art and future directions for psychological research. Dev. Rev. 41, 71–90. doi: 10.1016/j.dr.2016.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Roussis, P., and Wells, A. (2008). Psychological factors predicting stress symptoms: metacognition, thought control, and varieties of worry. Anxiety Stress Coping 21, 213–225. doi: 10.1080/10615800801889600

PubMed Abstract | CrossRef Full Text | Google Scholar

Ryum, T., Kennair, L. E. O., Hjemdal, O., Hagen, R., Halvorsen, J. O., and Solem, S. (2017). worry and metacognition as predictors of anxiety symptoms: a prospective study. Front. Psychol. 8:924. doi: 10.3389/fpsyg.2017.00924

PubMed Abstract | CrossRef Full Text | Google Scholar

Spada, M. M., Mohiyeddini, C., and Wells, A. (2008). Measuring metacognitions associated with emotional distress: factor structure and predictive validity of the metacognitions questionnaire 30. Personal. Individ. Differ. 45, 238–242. doi: 10.1016/j.paid.2008.04.005

CrossRef Full Text | Google Scholar

Takarangi, M. K. T., Smith, R. A., Strange, D., and Flowe, H. D. (2017). Metacognitive and metamemory beliefs in the development and maintenance of posttraumatic stress disorder. Clin. Psychol. Sci. 5, 131–140. doi: 10.1177/2167702616649348

CrossRef Full Text | Google Scholar

Vandenberg, R. J., and Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ. Res. Methods 2, 4–69. doi: 10.1177/109442810031002

CrossRef Full Text | Google Scholar

Wells, A. (2009). Metacognitive Therapy for Anxiety and Depression. New York, NY: Guilford Press.

Google Scholar

Wells, A., and Cartwright-Hatton, S. (2004). A short form of the metacognitions questionnaire: properties of the MCQ-30. Behav. Res. Ther. 43, 385–396. doi: 10.1016/s0005-7967(03)00147-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Wells, A., and Matthews, G. (1994). Attention and Emotion: A Clinical Perspective. Hove: Erlbaum.

Google Scholar

Wells, A., and Matthews, G. (1996). Modelling cognition in emotional disorder: the S-REF model. Behav. Res. Ther. 34, 881–888. doi: 10.1016/s0005-7967(96)00050-2

CrossRef Full Text | Google Scholar

Wells, A., and Papageorgiou, C. (1995). Worry and the incubation of intrusive images following stress. Behav. Res. Ther. 33, 579–583. doi: 10.1016/0005-7967(94)00087-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Westland, C. (2010). Lower bound on sample size in structural equation modelling. Electron. Comm. Res. Appl. 9, 476–487. doi: 10.1016/j.elerap.2010.003

CrossRef Full Text | Google Scholar

Widaman, K. F., and Reise, S. P. (1997). “Exploring the measurement invariance of psychological instruments: Applications in the substance use domain,” in The Science of Prevention: Methodological Advances from Alcohol and Substance Abuse Research, eds K. J. Bryant, M. E. Windle, and S. G. West, (Washington, DC: American Psychological Association), 281–324. doi: 10.1037/10222-009

CrossRef Full Text | Google Scholar

Wolf, E. J., Harrington, K. M., Clark, S. L., and Miller, M. W. (2013). Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educ. Psychol. Measure. 76, 913–934. doi: 10.1177/0013164413495237

PubMed Abstract | CrossRef Full Text | Google Scholar

Yılmaz, A. E., Gencöz, T., and Wells, A. (2008). Psychometric characteristics of the penn state worry questionnaire and metacognitions questionnaire-30 and metacognitive predictors of worry and obsessive-compulsive symptoms in a turkish sample. Clin. Psychol. Psycother. 15, 424–439. doi: 10.1002/cpp.589

PubMed Abstract | CrossRef Full Text | Google Scholar

Yılmaz, A. E., Gencöz, T., and Wells, A. (2011). The temporal precedence of metacognition in the development of anxiety and depression symptoms in the context of life-stress: a prospective study. J. Anxiety Disord. 25, 389–396. doi: 10.1016/j.janxdis.2010.11.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Yılmaz, A. E., Gencöz, T., and Wells, A. (2015). Unique contributions of metacognition and cognition to depressive symptoms. J. Gen. Psychol. 142, 23–33. doi: 10.1080/00221309.2014.964658

PubMed Abstract | CrossRef Full Text | Google Scholar

Zigmond, A. S., and Snaith, R. P. (1983). The hospital anxiety and depression scale. Acta Psychiatr. Scand. 67, 361–370.

PubMed Abstract | Google Scholar