Self-Management in Anxiety and Depression: A Psychometric Evaluation of a Questionnaire

Objective: To examine the underlying factor structure and psychometric properties of the Assessment of Self-management in Anxiety and Depression (ASAD) questionnaire, which was specifically designed for patients with (chronic) anxiety and depressive disorders. Moreover, this study assesses whether the number of items in the ASAD can be reduced without significantly reducing its precision.

Methods: The ASAD questionnaire was completed by 171 participants across two samples: one sample comprised patients with residual anxiety or depressive symptoms, while the other consisted of patients who have been formally diagnosed with a chronic anxiety or depressive disorder. All participants had previously undergone treatment. Both exploratory (EFA) and confirmatory factor analyses (CFA) were conducted. Internal consistency and test–retest reliability were also assessed.

Results: Both EFA and CFA indicated three solid factors: Seeking support, Daily life strategies and Taking ownership [Comparative Fit Index = 0.80, Tucker Lewis Index = 0.78, Root Mean Square Error of Approximation = 0.09 (CI 0.08–1.00), Standardized Root Mean Square Residual = 0.09 (χ² = 439.35, df = 168)]. The ASAD was thus reduced from 45 items to 21 items, which resulted in the ASAD-Short Form (SF). All sub-scales had a high level of internal consistency (> α = 0.75) and test–retest reliability (ICC > 0.75).

Discussion: The first statistical evaluation of the ASAD indicated a high level of internal consistency and test–retest reliability, and identified three distinctive factors. This could aid patients and professionals’ assessment of types of self-management used by the patient. Given that this study indicated that the 21-item ASAD-SF is appropriate, this version should be further explored and validated among a sample of patients with (chronic or partially remitted) anxiety and depressive disorders. Alongside this, to increase generalizability, more studies are required to examine the English version of the ASAD within other settings and countries.

Introduction

Anxiety and depressive disorders are among the most prevalent mental health disorders across the globe (World Health Organization, 2016). Many patients experience a chronic course of symptoms, with chronicity ranging from 24.5% for depressive disorders to 41.9% for anxiety disorders (Penninx et al., 2011). Anxiety and depressive disorders have been found to often coincide. Out of those patients that reach remission, approximately 57% experience a relapse within the first four years (Scholten et al., 2016). Both the chronicity of these disorders and the frequency of relapses increase the overall burden of disease (Keller, 2006; Penninx et al., 2011), and are associated with a decreased quality of life (Alonso et al., 2004). Alongside the significant burden that it places on patients and their relatives, anxiety and depression also cause a tremendous economic burden worldwide (Walker et al., 2015).

The chronic and recurrent nature of anxiety and depression requires the adoption of a chronic disease approach (Collins et al., 2011). The use of self-management skills are generally considered to be essential in managing a chronic disease (Grypdonck, 1999; Lorig and Holman, 2003), which is why it constitutes a key part of the chronic care model (Bodenheimer, 2002). Self-management has been defined in manifold ways. Barlow et al. (2002) define it as “the individual’s ability to manage the symptoms, treatment, physical and psychosocial consequences and life style changes inherent in living with a chronic condition.” According to Lorig and Holman (2003), the six core self-management skills, from a patient’s perspective, are: problem solving, decision-making, resource utilization, the formation of a patient-provider partnership, action planning and self-tailoring. With respect to chronic conditions, self-management pertains to practices that patients themselves employ in order to manage their symptoms, avoid relapse, optimize their well-being (Barlow et al., 2002; Lorig and Holman, 2003), and, ultimately, improve the quality of their lives. Notwithstanding the role of the patient, professionals also have an important role to play in self-management promotion (Lorig and Holman, 2003).

In recent years, there has been an emergent focus on recovery in mental health care (Happell, 2008; Casey and Webb, 2019). Recovery is not only focused on reducing symptoms, but also signals “a movement toward health and meaning rather than avoidance of symptoms” (Clarke et al., 2012). From this perspective, recovery also encompasses personal and functional recovery. Manifold studies have acknowledged that self-management can be critically important in recovering from mental health disorders, due to their often chronic nature (Young and Ensing, 1999; Deegan, 2005; Sterling et al., 2010). There is increasing support for the assumption that the promotion by professionals in (mental) health care of patients’ self-management skills deserves more attention through the systematic development of self-management interventions that are highly tailored to self-management abilities of individual patients. For this purpose, more clarity is required about the concept of self-management and its operationalization, this in general and with respect of specific patient groups (Barlow et al., 2002; Lorig and Holman, 2003).

Extant literature has highlighted that there is a complex interaction between anxiety and depressive disorders and the use of self-management skills. For example, prior research has shown that the ability of patients with anxiety and depressive symptoms to perform self-management skills is influenced by the level of their symptoms, and that more severe anxiety and depressive symptoms lead to a decrease in self-management activities (Orem, 2001). Research has also demonstrated that the effective use of self-management strategies by patients suffering from anxiety or depression can reduce the severity of the symptoms for these disorders (Levitt et al., 2009). Hence, although depression and anxiety likely reduce patients’ ability to utilize self-management strategies, if self-management strategies are utilized they are likely to decrease patients’ anxiety and depressive symptoms.

There are a number of studies that have investigated the range of self-management strategies utilized by patients with anxiety and depressive disorders, which are often categorized into clusters (van Grieken et al., 2014, 2015; Chambers et al., 2015; Villaggi et al., 2015). The clusters that are most often described are: Engaging in activities (Chambers et al., 2015; van Grieken et al., 2015; Villaggi et al., 2015), Creating a routine (Chambers et al., 2015; Villaggi et al., 2015), and Seeking professional treatment (van Grieken et al., 2014; Villaggi et al., 2015). Engaging in activities mainly focuses on staying active, while creating a routine pertains to having a sense of structure and routinely planning activities. To the best of our knowledge, there is only one study available that psychometrically evaluates a self-management questionnaire for patients with anxiety, depressive and bipolar disorders, namely the Canadian developed Mental Health Self-Management Questionnaire (MHSQ; Coulombe et al., 2015). Given the potential cross-cultural differences in self-management (Shimizu and Paterson, 2007), not to mention that the MHSQ can be considered incredibly extensive (64 items), there was a need to develop, and subsequently psychometrically assess, a Dutch questionnaire that measured the self-management strategies employed by patients with anxiety and depressive disorders.

Consequently, Zoun et al. (2016) constructed a valid questionnaire by using a sample of Dutch patients with chronic anxiety and depressive disorders, which is called the Assessment of Self-management in Anxiety and Depression (ASAD). This questionnaire contains 45 self-management strategies relating to coping with a chronic anxiety and/or a depressive disorder. However, this questionnaire has hitherto not been psychometrically examined. Psychometric examinations can be carried out by conducting exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). These statistical techniques reveal and confirm the underlying factor structure of a questionnaire, which results in either one or multiple factors explaining the common variance. Moreover, EFA and CFA are also statistical reduction techniques, which can help to determine if the items included in the questionnaire can be reduced, without significantly reducing the level of precision. Simply put, reducing the number of items greatly simplifies the survey administration, which, in turn, makes it less burdensome for patients.

The present study aims to gain insight into the underlying factor structure and psychometric properties of the ASAD, by using two samples of Dutch patients with either partially remitted or chronic anxiety and/or depressive disorders. Furthermore, this study examines whether the number of items in the ASAD can be reduced.

Materials and Methods

Study Design and Participants

This is a cross-sectional study that uses data from two separate studies: the GET READY study (Krijnen-De Bruin et al., 2019) and the SemCAD study (Zoun et al., 2016). These two studies were combined to increase the sample size and to assess the ASAD among patients who display the complete range of symptom severity, from mild to severe symptoms. As such, our sample is a good representation of patients who have been treated for anxiety or depressive disorders. An exploratory factor analysis (EFA), followed by a confirmatory factor analysis (CFA) were conducted to explore the underlying factor structure of the ASAD. Also, internal consistency, test–retest reliability and discriminant validity were tested.

Sample 1: GET READY Study

Data from the GET READY study was collected as part of an observational cohort study focused on relapse prevention. The participants in the GET READY study were adults who had undergone treatment for an anxiety and/or a depressive disorder within mental healthcare services in the Netherlands in the prior 2-year period, and who were subsequently in full or partial remission. Remission was defined by a score of ≤29 on the Beck Anxiety Inventory (BAI), and a score of ≤38 on the Inventory of Depressive Symptomatology-Self Report (IDS-SR). While measurements of the ASAD took place at baseline and after a 9-month period, only the baseline data was used for this study. A sub-sample completed the ASAD again 2 weeks after the baseline questionnaire, for test–retest measurements.

Fifty-four mental health professionals working in primary care practices throughout the Netherlands were recruited via telephone, letters, advertisements on websites, and through the researchers’ professional networks.

Patients were recruited by the mental health professionals at the end of their treatment, who provided brief information about the study. If patients were interested in participating, then the mental health professional asked consent from the patient to provide their contact details to the researchers. Next, consenting patients were contacted by the researchers and received additional information. This data was collected between April 2017 and November 2018. Further details on the GET READY study have been published previously (Krijnen-De Bruin et al., 2019).

Sample 2: SemCAD Study

The data from the SemCAD study was collected as part of a randomized controlled trial. The participants in the SemCAD study were adults who had been diagnosed with a chronic anxiety and/or depressive disorder, and had undergone treatment in outpatient specialist mental healthcare services in the Netherlands for at least a 2-year period, but who had yet to achieve recovery. The clinicians considered them to be treatment-resistant patients.

Participants were recruited from twelve Dutch specialized outpatient mental health care services for adults with anxiety and depressive disorders. Patients eligible for participating in the SemCAD study (Zoun et al., 2016) were asked to participate by their community psychiatric nurse. This nurse informed the patient about the study and provided an information letter to take home. After inclusion, patients received the baseline questionnaire. After administering this questionnaire, the patient was allocated to the experimental or control group of the trial.

The measurements took place at baseline (T0), 6 months (T1), 12 months (T2), and 18 months (T3). Given that the ASAD and Patient Health Questionnaire-9 (PHQ-9)/BAI were not collected at the same time, PHQ-9 and BAI data was used from T2, as this was the closest to the assessment of the ASAD. A detailed description of the SemCAD study has also previously been published (Zoun et al., 2016).

Written informed consent was obtained from all participants, and a Medical Ethics Committee approved both study protocols (Zoun et al., 2016; Krijnen-De Bruin et al., 2019). Given that the ASAD was developed for and with patients with chronic anxiety and/or depressive disorders, and we were seeking to psychometrically assess the ASAD among patients who matched these criteria, we included patients with at least mild symptoms in our analyses. Therefore, participants who scored less than 10 on the BAI, and less than 5 on the PHQ or less than 14 on the IDS-SR, were ultimately excluded from the EFA and CFA.

Materials

Demographic Variables

Demographic data was collected from all participants, and included age, gender, marital status, educational level and occupational status.

Clinical Variables

In order to measure the severity of anxiety symptoms, the BAI was used in both samples. The BAI is a questionnaire comprising 21 items. Each item is rated on a 4-point Likert scale from 0 (not at all) to 3 (severely). The BAI has good psychometric properties: Cronbach’s alpha = 0.94; test–retest reliability: r = 0.67; and good convergent and discriminant validity (Fydrich et al., 1992). In order to measure the severity of depressive symptoms, the IDS-SR was used in the GET READY sample. The IDS-SR is a self-report questionnaire, which consists of 30 items. Each item is rated on a 4-point Likert scale from 0 to 3. The IDS-SR has good psychometric properties: Cronbach’s alpha = 0.94; high item total correlations (≥0.50) in 70% of the items; convergent validity = 0.85 (Trivedi et al., 2004; Gili et al., 2010). In the SemCAD sample, the PHQ-9 was used to assess depression severity. The PHQ-9 is a questionnaire comprising nine items. Each item is rated on a 4-point Likert scale from 0 (not at all) to 3 (almost every day). The PHQ-9 is a reliable and valid questionnaire: Cronbach’s alpha = 0.86–0.89; excellent test–retest reliability; high divergent validity; sensitivity and specificity of both 88% for major depression (Kroenke et al., 2001).

Development of the Assessment of Self-Management in Anxiety and Depression Questionnaire

In a prior study (van Grieken et al., 2014), a self-management questionnaire was developed for patients with chronic depression. Using the same methods (concept mapping), the authors of the ASAD developed a questionnaire that focused on patients with chronic depression and chronic anxiety disorders.

In phase 1 of concept mapping, patients were asked about helpful self-management strategies they had employed. Two focus groups were formed, consisting of ten and eight patients with chronic anxiety disorders, respectively. In these focus groups, patients completed the statement “I can live with a non-recovered anxiety disorder if….” This technique allowed for self-management strategies to be explored and discussed together. The patients’ responses were then written on a whiteboard and adjusted until they were clear to all participants. This resulted in a total of 176 self-management strategies. Next, when analyzing the findings of these focus groups, any overlapping strategies were subsequently merged. Ultimately, 91 strategies remained. In phase 2 of concept mapping, these 91 strategies were then sent to the participants, who were requested to prioritize and arrange the strategies accordingly. These strategies were then clustered, based on common features, and prioritized by their relevance. In phase 3, the strategies were placed in a concept map and divided into different clusters through the use of the program “Ariadne” (Severens, 2015). In the final phase, the list with the 91 strategies was compared to the existing list of 50 self-management strategies utilized by patients with depression (van Grieken et al., 2014). Once again, any overlapping strategies were removed, and some were merged. This resulted in the Dutch ASAD, which consists of 45 self-management strategies for patients with an anxiety and/or a depressive disorder. Each item can be rated on a 5-point Likert scale, ranging from 1 (not at all) to 5 (very much).

The items in the questionnaire are described in Table 1, including the mean scores of the items, the standard deviations and skewness. The higher the mean score, the higher the application of the selfmanagement behavior. With respect to skewness: 37 of the answer distributions of the 45 items (82%) were symmetrical (scores between −0.5 and 0.5), so we may conclude that the majority of the distributions was not very skewed.

TABLE 1

Table 1. Items in the “Assessment of Self-management in Anxiety and Depression” questionnaire.

Data-Analysis

In order to assess the factor structure of the ASAD, both an EFA and CFA were conducted. With EFA, the number of factors can be determined: all measured variables–items–are related to every latent variable (factor). With CFA specification is possible of how each item loads with a unique pattern of zero and non-zero loadings onto hypothesized factors (Matsunaga, 2010). Before running the analysis, the number of factors as well as which measured variable was related to which latent variable was specified in a CFA model.

An EFA with principal axis factoring was conducted to examine the underlying factor structure of the ASAD. To test the internal consistency of the ASAD, Cronbach’s alpha was calculated for each factor, while the communalities (common variance) of the items, factor loadings, and overdetermination (factor to variable ratio) were also examined. In order to assess whether the number of items could be reduced, the correlation matrix of the EFA was inspected for extreme multicollinearity (correlations above 0.80) (Field, 2013). Items with a communality below 0.30, items that cross-loaded on multiple factors with less than 0.2 difference, and items that did not load on any factors were all removed, before the analysis was then subsequently re-run. Given that we anticipated that the factors might correlate, an oblique rotation was chosen (direct oblimin) and the pattern matrix was examined for the factor and item loadings. In order to determine the number of factors that needed to be retained, the scree plot and eigenvalues were inspected. Initially, items with eigenvalues above one were included. The factors deriving from the final factor solution were interpreted and named. The EFA was conducted via SPSS, version 26.

Upon completion of EFA, CFA was then conducted to both find evidence for the proposed factor structure in EFA, and to examine whether the factors found via EFA were clearly identifiable constructs as measured by the items they contain. To establish the latent structure of items and factors in this way, the CFA was applied with package LAVAAN in R. The input for the CFA were the factors with their items as identified with EFA. The following criteria were used for the goodness-of-fit of the final model, which was based on a reduced number of items: Comparative Fit Index (CFI) and Tucker-Lewis index (TLI) values >0.90, Root Mean Square Error of Approximation (RMSEA) values <0.06 and Standardized Root Mean Square Residual (SRMR) <0.08 (Hooper et al., 2008). Hooper et al. (2008) describe that these fit measures are suitable to determine how well a specified model fits the sample data. Missing data, which pertained to ten missing values distributed over six items, were imputed with case means.

Furthermore, psychometric properties were also examined, including internal consistency, test–retest reliability and discriminant validity. Internal consistency indicates the degree of interrelatedness among the items (Mokkink et al., 2010). This was measured using Cronbach’s alpha, and the sub-scales were assumed to be adequate when Cronbach’s alpha was >0.70 (Terwee et al., 2007). In addition, test–retest reliability was examined using intraclass correlation coefficients (ICC). ICC values were assumed to be excellent if ICC >0.75 (Fleiss, 1986). With respect to the test–retest reliability, there were two observations, as ICC values of at least 0.5 were considered to be acceptable, at least 22 participants were required to complete the retest (Bujang and Baharum, 2017). Discriminant validity concerns the extent to which the factors are diverse and uncorrelated. Therefore, correlations should not exceed 0.7.

Results

Participants

The 213 participants in this study comprised 113 participants from the GET READY study and 100 participants from the SemCAD study. In total, 42 participants were excluded from the study because their scores indicated that they had no current symptoms (BAI < 10 and PHQ < 5/IDS-SR < 14). This resulted in 171 participants: 88 from the GET READY study and 83 from the SemCAD study. Table 2 presents the demographic and clinical characteristics of these two samples. The total sample consisted of 103 females and 66 males (the gender of two participants was unknown). Out of the 51 participants from the GET READY sample who were invited to re-take the ASAD after a 2-week period, 43 (84.3%) responded positively, of which the scores of 12 patients indicated that they were in complete remission. Therefore, the test–retest reliability could be examined for 31 participants.

TABLE 2

Table 2. Demographic and clinical variables for each study.

As one can discern in Table 2, there were significant differences between the two samples concerning age, educational level, occupational status and the severity of anxiety symptoms. The SemCAD sample was significantly older, had, on average, a lower educational level, were significantly more likely to be unemployed, as well as experiencing more severe anxiety symptoms. The potential difference between the severity of depressive symptoms could not be statistically tested, as both samples utilized different questionnaires. However, the mean IDS-SR score indicates mild depressive symptoms, while the mean PHQ-9 score indicates moderate depressive symptoms.

Exploratory Factor Analysis

A principal axis factor analysis was conducted on the 45 items of the ASAD with oblique rotation (direct oblimin). The Kaiser-Meyer-Olkin (KMO) measure verified the sampling adequacy of the analysis, KMO = 0.85 [“meritorious” according to Hutcheson and Sofroniou (1999)] and all KMO values for individual items were greater than 0.54, which is above the acceptable limit of 0.5 (Field, 2013). Bartlett’s test of sphericity (Bartlett, 1951) was statistically significant (p < 0.001). Both tests supported the factorability of the correlation matrix. An initial analysis was run to obtain eigenvalues for each factor in the data. This resulted in 13 factors exceeding Kaiser’s criterion of eigenvalue one, the combination of which explained 67.87% of the variance. Of those 13 factors, three had an eigenvalue above two. The scree plot was ambiguous and showed inflections that would justify retaining either three or five factors. We opted to retain three factors because this provided the most interpretable solution. Parallel Analysis resulted in seven factors that were not interpretable in a clinically meaningful way; three of the factors in the parallel analysis corresponded to the factors had adjusted eigenvalues >1) and could be interpreted. So, also based in these findings it was decided to retain three factors.

Items with factor loadings larger than 0.3 were included in the factor. In total, 24 items were excluded from the analysis, due to the fact that they did not load on any factor (N = 7), showed cross-loadings between factors with less than 0.2 difference (N = 6), or had a communality below 0.3 (N = 11). Thus, the number of items was reduced from 45 to 21 after the analysis. The 21-item version of the ASAD, the ASAD-Short Form (SF), explained after rotation 51.27% of the variance, with factor one contributing 30.08%, factor two contributing 12.45%, and factor three contributing 8.74%.

Close inspection of the items that were grouped into the same factor (see Table 3) led to the factors being named as follows: factor 1 represents Seeking support; factor 2 represents Daily life strategies; and factor 3 represents Taking ownership. More information about the factors can be found in Table 3.

TABLE 3

Table 3. Explanation of the underlying factors.

Confirmatory Factor Analysis

After EFA, CFA was conducted to examine whether the factors found with EFA were clearly identifiable constructs as measured by the items they contain. This should be expressed in at least moderate goodness-of-fit measures for the estimated model with three (correlated) factors. With the 21 items corresponding to the three factors found via EFA as input, CFA resulted in moderate fit measures: χ² = 439.35 (df = 168), CFI = 0.80, TLI = 0.78, RMSEA = 0.09 (CI 0.08–1.00), SRMR = 0.09.

Correlation of the factors was permitted, the following values were found: R seeking support-daily life strategy = 0.49, R seeking support-taking ownership = 0.47, R daily life strategy -taking ownership = 0.39.

Internal Consistency

To assess the internal consistency of the factors, the communalities (common variance) of the items, overdetermination (factor to variable ratio), and factor loadings were examined. Starting with the communalities, the 21-item ASAD-SF had four items with communality above 0.50 and a mean communality of 0.44, which is relatively low. Overdetermination of factors was high, as high loadings were shown on at least three items for each factor (MacCallum et al., 1999). Therefore, despite low communalities, good recovery of factors was still achieved (MacCallum et al., 1999). Continuing with the factor loadings, factor 1 had five factor loadings above 0.60 and two factor loadings above 0.50, factor 2 had four factor loadings above 0.60 and two factor loadings above 0.50, while factor 3 had three factor loadings above 0.60. Therefore, factor 1 and 2 had five or more strongly loading items (0.50 or better), thus indicating solid factors (Costello and Osborne, 2005). Although factor 3 had only three items, these items could nevertheless still be interpreted in a meaningful way, and, indeed, three items can still indicate a solid factor (Costello and Osborne, 2005). Cronbach’s alpha was calculated for the 21-item ASAD-SF, resulting in high internal consistency, α = 0.88. All sub-scales also had high internal consistency (factor 1: α = 0.88, factor 2: α = 0.86, factor 3: α = 0.76). Factor loadings found with EFA and CFA respectively are presented in Table 4.

TABLE 4

Table 4. Factor loadings per item of the ASAD questionnaire.

Test–Retest Reliability

Test–retest reliability over a 2-week period was excellent (factor 1: ICC = 0.76, 95% CI 0.49–0.89; factor 2: ICC = 0.88, 95% CI 0.71–0.95; factor 3: ICC = 0.86, 95% CI 0.72–0.93).

Discriminant Validity

As one can see in Table 5, the Seeking support factor correlated with both the Daily life strategies (0.31) and the Taking ownership factor (0.31). Furthermore, the Daily life strategies factor also correlated with the Taking ownership factor (0.28). The fact that these correlations are well below 0.7 indicates that the variables in the factors were more strongly related to their own factors than other factors.

TABLE 5

Table 5. Factor correlation matrix.

Discussion

Main Findings

This study represents the first attempt to evaluate the ASAD questionnaire by performing EFA and CFA, in order to establish the underlying factor structure, psychometric properties, and the possibility of reducing items without sacrificing on precision. Three consistent factors were identified: Seeking support, Daily life strategies, and Taking ownership. Seeking support contained ten items that pertained to seeking and maintaining support from both professionals and significant others. Daily life strategies contained eight items related to maintaining an active lifestyle, healthy habits and engaging in structured activities. Taking ownership contained three items, which pertained to taking control over one’s life and focusing one’s attention on recovery. The number of items in the questionnaire was subsequently reduced from 45 to 21. The results thus support the psychometric properties of a short form (SF) of the ASAD. The internal consistency of the 21-item ASAD-SF as well as their sub-scales was high, test–retest reliability indicated excellent reliability on all factors, and discriminant validity was also shown.

Comparisons to Previous Literature

Of those studies that have psychometrically evaluated similar self-management questionnaires, Coulombe et al. (2015) and Smith et al. (2017) research are particularly worthy of comparison to this study.

The factors identified in this study are similar to the subscales found in Coulombe et al. (2015) study. They examined the Mental Health Self-Management Questionnaire (MHSQ), which is a 64-item questionnaire developed for patients who are recovering from depression, anxiety or bipolar disorders. The subscale Clinical appears to be similar to the factor Seeking support. However, the clinical subscale does not include support from significant others, whereas the Seeking support factor includes both professional support and support from significant others. The subscale Vitality resembles the factor Daily life strategies, insofar as both focus on engaging in healthy activities. Moreover, the subscale Empowerment bears some similarities with the factor Taking ownership. However, the Empowerment subscale is more extensive, and contains items like “loving myself as I am” and “congratulating myself for my successes.” However, the MHSQ contains almost three times as many items as the 21-item ASAD-SF, and, hence, is far more burdensome for patients to complete.

There was also some overlap between our study and the Partners in Health (PIH) scale. The PIH scale is an Australian questionnaire, which focuses on the self-management strategies employed by the chronically ill. Smith et al. (2017) conducted a CFA to establish the factors of the PIH. When comparing the ASAD-SF to the PIH scale, the same similarities and differences appear as with the MHSQ: the factor Patient-health professional partnership is similar to the factor Seeking support, albeit it only focuses on professional support, as opposed to support from significant others too. The factor Coping with a chronic illness is similar to the factor Daily life strategies, insofar as both focus on staying active. The similarities between, on the one hand, the identified factors of the ASAD-SF, and the PIH-factors Knowledge of illness and treatment and Recognition and management of symptoms, on the other, were less obvious, potentially because the PIH scale focuses primarily on self-management of physical illness rather than mental illness. Although, of course, there are many similarities between physical and mental illness, not to mention that the underlying constructs of handling a chronic disease might be comparable, one would expect to see differences between the PIH and the ASAD-SF.

van Grieken et al. (2014, 2015), Villaggi et al. (2015), and Schreurs et al. (1988) all conducted studies that qualitatively established clusters, dimensions and factors related to self-management.

Given that the ASAD was developed as an expansion of the self-management strategies identified by van Grieken et al. (2014), it is altogether understandable that the factors found in our factor analyses match the sub clusters found by these authors. The sub clusters identified by van Grieken et al. (2014, 2015) were clustered using concept mapping. While the authors psychometrically evaluated the 50 self-management strategies that were identified by their participants, no satisfactory EFA could be performed (van Grieken et al., 2018). Our study did partially confirm the sub clusters identified in their studies. More specifically, the factor Seeking support is in line with the sub clusters Active coping with professional treatment and Social engagement (van Grieken et al., 2014). Furthermore, the factor Daily life strategies appears to be in accordance with the sub clusters Active self-care, structure and planning, Daily life strategies and rules, Engaging in activities, and Structured attention to oneself (van Grieken et al., 2014, 2015). In contrast, no sub clusters were described that matched the factor Taking ownership.

There are also similarities between the present study and the factors identified in Villaggi et al. (2015) study. In their study, self-management strategies were explored among a sample of patients with anxiety disorders, depressive disorders and bipolar disorders. The factor Seeking support resembles the dimension Surrounding myself with people who make me feel better, although this dimension is only focused on informal support, rather than also on professional support. The factor Daily life strategies is in accordance with both the dimension Functional (e.g., Creating a routine and taking action) and the dimension Physical (e.g., Maintaining a healthy lifestyle and Managing one’s energy levels). The factor Taking ownership is in accordance with the sub dimension Empowering oneself. This study from Villaggi et al. further revealed that patients with anxiety, depressive and bipolar disorders utilized similar self-management strategies. This indicates that an approach that is focused on affective disorders might be appropriate for self-management (Coulombe et al., 2016).

Finally, the factors identified in this factor analysis can be compared with the Utrecht Coping List (Schreurs et al., 1988). The Utrecht Coping List is a 47-item questionnaire that encapsulates seven domains of coping. The domain Seeking social support appears to be very similar to our factor Seeking support. Moreover, the domain Active problem solving is in line with our factor Taking ownership. The other domains were not concordant with the factors found in this factor analysis, most likely because coping is not the same as self-management; while self-management focuses on how one handles a specific disease, coping is more focused on coping with problems in general.

When comparing our findings to the six core self-management skills defined by Lorig and Holman (2003), it appears that the skill Formation of a patient-provider partnership overlaps with our factor Seeking support. In addition, the skill Action planning bears some resemblance to our factor Daily life strategies. The other four self-management skills (Problem solving, Decision making, Resource utilization and Self-tailoring) appear to be intertwined in our three factors.

To conclude, most of the overlap between our findings and extant literature pertained to the factors Seeking support and Daily life strategies. Within other self-management questionnaires, the factor Seeking support is primarily focused on seeking support from a professional (Coulombe et al., 2015; Smith et al., 2017), and, indeed, no factors that combined both professional and informal support were found in extant literature. Factors similar to Daily life strategies were often identified (van Grieken et al., 2014, 2015; Coulombe et al., 2015; Villaggi et al., 2015; Smith et al., 2017). Although less pronounced, we also found a degree of overlap between our third factor Taking ownership, albeit these similar factors focused on empowerment (Coulombe et al., 2015; Villaggi et al., 2015). We did not identify any other important factors within extant literature that were not found in our study.

Strengths and Limitations

There are several strengths to the present study. First, this study is the first to statistically examine the ASAD, and thus provide insight into the underlying factor structure, internal consistency, test–retest reliability and discriminant validity of this questionnaire. Moreover, this study sheds light on how those items that are grouped together are subsequently translated into self-management styles. The 21-item ASAD-SF has proven to be an internally consistent and reliable measure for assessing self-management strategies among a sample of Dutch patients with (chronic and partially remitted) anxiety and depressive disorders.

The second strength is that the questionnaire focuses on patients suffering from both anxiety and depressive disorders. This is important, because most studies focus solely on either patients with anxiety disorders or patients with depressive disorders, despite the fact that previous research has shown that anxiety and depressive disorders are highly comorbid (Lamers et al., 2011). Comorbid anxiety and depressive disorders increase chronicity (Penninx et al., 2011), which is one of the reasons for encouraging modes of self-management.

The findings of this study should be interpreted with several limitations in mind. First, the sample of 171 patients is a relatively small sample for factor analysis (Tabachnick and Fidell, 2012). However, sample size is also dependent on factor loadings and communalities (Field, 2013). Matsunaga (2010) argues that when items are closely related to a factor, a sample size between 100 and 200 is sufficient. Our sample of 171 cases should suffice, especially since this study concerns a first exploration of factors. As the results demonstrated, the internal consistency of the ASAD-SF and its three factors is high. Although the communalities show less evidence of internal consistency, the factor loadings and overdetermination indicate that the factors are internally consistent. Furthermore, the preliminary analysis did indicate that the sample was appropriate for factor analysis. The advantage of a larger sample size is that it would have allowed us to split the sample, and conduct EFA on one sample, while conducting CFA on the other sample, which, in turn, would result in a more stringent assessment of the factor structure (Wang et al., 2013). However, this study did not aim to fit the data to suit the EFA and CFA, but rather aimed to provide insight into the covariances between factors, and, in turn, into the factors of the newly developed ASAD.

The second limitation of this study pertains to the use of two different datasets. These datasets were merged to increase the sample size, in order to be able to conduct the EFA and CFA. The participants in this study were screened for depression via two different questionnaires, the PHQ-9 and the IDS-SR, which might be less reliable than using a single screening questionnaire. However, both the PHQ-9 and the IDS-SR have been found to be reliable and valid questionnaires for assessing depression. Moreover, these questionnaires were only used to assess the severity of symptoms, in order to exclude those who had no symptoms. Nevertheless, using two datasets could well result in a greater level of heterogeneity. In the SemCAD sample, patients experienced, on average, moderate anxiety and depressive symptoms, while in the GET READY sample the anxiety and depressive symptoms were mild. One explanation for this is that the SemCAD sample comprised patients with chronic anxiety and/or depressive disorders, while the GET READY sample consisted of patients who were in partial remission from anxiety and/or depressive disorders.

Implications for Practice and Research

Several implications for practice can be suggested. First, this study provides a feasible self-management questionnaire that can be offered to patients with (chronic or partially remitted) anxiety and depressive disorders. The use of self-management strategies by patients is likely to be cost-effective for both the prevention and treatment of anxiety and depressive disorders, and could reduce the burden of disease, as well as the economic burden of mental health disorders. The results of this study could also provide guidance for patients regarding which specific self-management strategies to focus on. For example, they could self-manage their disorder by seeking support from professionals and important others (Seeking support), by maintaining a healthy lifestyle and engaging in activities (Daily life strategies), or by taking control and staying focused on their recovery (Taking ownership). Second, the three factors identified in this study could also be considered in the development of relapse prevention programs. With respect to E-health modules, attention should be paid to the themes Seeking support, Daily life strategies and Taking ownership. For example, information on how to engage with important others and MHPs could help patients to seek out support. Third, professionals could advise patients in how best to execute these self-management themes and strategies. One possible approach might be for patients to complete the ASAD-SF during their actual treatment. If they scored low on one or more factors, then this could help guide professionals in advising which self-management strategies to utilize. For example, if patients scored low on the factor Seeking support, the specific attention could be paid to how patients can better involve professionals and important others in their recovery. Fourth, depending on the purpose of completing the ASAD, patients and professionals could choose which version to use. If a detailed description of self-management strategies is required, then patients could complete the 45-item ASAD. Alternatively, if a shorter version is preferred that provides insight into important clusters of self-management, then patients could complete the 21-item ASAD-SF.

Given that this study indicated that the 21-item ASAD-SF is appropriate, this version should be further explored and validated among a sample of patients with (chronic or partially remitted) anxiety and depressive disorders. Alongside this, to increase generalizability, more studies are required to examine the English version of the ASAD within other settings and countries.

Conclusion

This study presents an initial test of a questionnaire that assesses self-management strategies for patients with chronic or partially remitted anxiety and/or depressive disorders. In this respect, the ASAD-SF demonstrated good psychometric properties. The statistical evaluation conducted here constitutes the first examination of factors of self-management strategies among this sample, and resulted in a stable 3-factor structure. The 21-item ASAD-SF could be offered to patients with (chronic or partially remitted) anxiety and depressive disorders, with the identified factors providing guidance to patients and professionals concerning which specific strategies to focus on. Ultimately, these factors could be incorporated into the development of new relapse prevention programs.

Data Availability Statement

The data analyzed in this study is subject to the following licenses/restrictions: Data from the SemCAD study cannot be made publicly available due to institutional restrictions. Data from the GET READY study can be made available upon request. Requests to access these datasets should be directed to EK-D, esther.krijnendebruin@inholland.nl.

Ethics Statement

The studies involving human participants were reviewed and approved by The Medical Ethics Committee of the Amsterdam UMC, location VU University Medical Center and the Institutional Review Board of the University Medical Center Utrecht. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

EK-D, AM, AE, AS, HS, JS, NB, and BM contributed to conception and design of the study. EK-D and SD performed the statistical analysis. EK-D wrote the first draft of the manuscript. AE and SD wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

Funding

Funding for this study was provided by the SIA-RAAK: The Taskforce for Applied Research, part of the Netherlands Organization for Scientific Research (NWO, grant number 2015-02-36P). SIA-RAAK had no role in the study design, collection, analysis or interpretation of the data, writing the manuscript, or the decision to submit the manuscript for publication.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors gratefully acknowledge the contribution of all patients of both the GET READY and the SemCAD study.

Abbreviations

ASAD, Assessment of Self-management in Anxiety and Depression; ASAD-SF, Assessment of Self-management in Anxiety and Depression-Short Form; EFA, exploratory factor analysis; CFA, confirmatory factor analysis; CFI, Comparative Fit Index; TLI, Tucker Lewis Index; RMSEA, Root Mean Square Error of Approximation; SRMR, Standardized Root Mean Square Residual; MHSQ, Mental Health Self-Management Questionnaire; PIH, Partners in Health; BAI, Beck Anxiety Inventory; IDS-SR, Inventory of Depressive Symptomatology–Self Report; PHQ-9, Patient Health Questionnaire-9; ICC, intraclass correlation coefficients; KMO, Kaiser-Meyer-Olkin.

References

Alonso, J., Angermeyer, M. C. C., Bernert, S., Bruffaerts, R., Brugha, T. S. S., Bryson, H., et al. (2004). Disability and quality of life impact of mental disorders in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatr. Scand. 109(Suppl. 420), 38–46. doi: 10.1111/j.1600-0047.2004.00329.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Barlow, J., Wright, C., Sheasby, J., Turner, A., and Hainsworth, J. (2002). Self-management approaches for people with chronic conditions: a review. Patient Educ. Couns. 48, 177–187. doi: 10.1016/S0738-3991(02)00032-0

CrossRef Full Text | Google Scholar

Bartlett, M. (1951). The effect of standardization on a Chi-square approximation in factor analysis. Biometrika 38, 337–344. doi: 10.1093/biomet/38.3-4.337

CrossRef Full Text | Google Scholar

Bodenheimer, T. (2002). Improving primary care for patients with chronic illness. JAMA 288, 1775–1779. doi: 10.1001/jama.288.14.1775

PubMed Abstract | CrossRef Full Text | Google Scholar

Bujang, M. A., and Baharum, N. (2017). A simplified guide to determination of sample size requirements for estimating the value of intraclass correlation coefficient: a review. Arch. Orofac. Sci. 12, 1–11.

Google Scholar

Casey, B., and Webb, M. (2019). Imaging journeys of recovery and learning: a participatory arts-based inquiry. Qual. Health Res. 29, 833–845. doi: 10.1177/1049732318804832

PubMed Abstract | CrossRef Full Text | Google Scholar

Chambers, E., Cook, S., Thake, A., Foster, A., Shaw, S., Hutten, R., et al. (2015). The self-management of longer-term depression: learning from the patient, a qualitative study. BMC Psychiatry 15:172. doi: 10.1186/s12888-015-0550-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Clarke, S., Oades, L. G., and Crowe, T. P. (2012). Recovery in mental health: a movement towards well-being and meaning in contrast to an avoidance of symptoms. Psychiatr. Rehabil. J. 35, 297–304. doi: 10.2975/35.4.2012.297.304

CrossRef Full Text | Google Scholar

Collins, P. Y., Patel, V., Joestl, S. S., March, D., Insel, T. R., Daar, A. S., et al. (2011). Grand challenges in global mental health. Nature 475, 27–30. doi: 10.1038/475027a

PubMed Abstract | CrossRef Full Text | Google Scholar

Costello, A. B., and Osborne, J. W. (2005). Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract. Assessment. Res. Eval. 10, 1–9.

Google Scholar

Coulombe, S., Radziszewski, S., Meunier, S., Provencher, H., Hudon, C., Roberge, P., et al. (2016). Profiles of recovery from mood and anxiety disorders: a person-centered exploration of People’s engagement in self-management. Front. Psychol. 7:584. doi: 10.3389/fpsyg.2016.00584

PubMed Abstract | CrossRef Full Text | Google Scholar

Coulombe, S., Radziszewski, S., Trépanier, S. G., Provencher, H., Roberge, P., Hudon, C., et al. (2015). Mental health self-management questionnaire: development and psychometric properties. J. Affect. Disord. 181, 41–49. doi: 10.1037/t47268-000

CrossRef Full Text | Google Scholar

Deegan, P. E. (2005). The importance of personal medicine: a qualitative study of resilience in people with psychiatric disabilities. Scand. J. Public Health 33, 29–35. doi: 10.1080/14034950510033345

PubMed Abstract | CrossRef Full Text | Google Scholar

Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics. Thousand Oaks, CA: Sage Publications, 1–916.

Google Scholar

Fleiss, J. (1986). The Design and Analysis of Clinical Experiments. New York, NY: John Wiley & Sons.

Google Scholar

Fydrich, T., Dowdall, D., and Chambless, D. L. (1992). Reliability and validity of the beck anxiety inventory. J. Anxiety Disord. 6, 55–61. doi: 10.1016/0887-6185(92)90026-4

CrossRef Full Text | Google Scholar

Gili, M., Luciano, J. V., Bauzá, N., Aguado, J., Serrano, M. J., Armengol, S., et al. (2010). Psychometric properties of the IDS-SR₃₀ for the assessment of depressive symptoms in spanish population. BMC Med. Res. Methodol. 11:131. doi: 10.1186/1471-2288-11-131

PubMed Abstract | CrossRef Full Text | Google Scholar

Grypdonck, M. (1999). Het Leven Boven de Ziekte Uittillen. Leiden: Spruyt Van Mantgem en de Does.

Google Scholar

Happell, B. (2008). Determining the effectiveness of mental health services from a consumer perspective: part 1: enhancing recovery. Int. J. Ment. Health Nurs. 17, 116–122. doi: 10.1111/j.1447-0349.2008.00519.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Hooper, D., Coughlan, J., and Mullen, M. R. (2008). Structural equation modelling: guidelines for determining model fit. Electron. J. Bus. Res. Methods 6, 53–60.

Google Scholar

Hutcheson, G., and Sofroniou, N. (1999). The Multivariate Social Scientist. London: Sage Publications, 1–276. doi: 10.4135/9780857028075

PubMed Abstract | CrossRef Full Text | Google Scholar

Keller, M. B. (2006). Social anxiety disorder clinical course and outcome: review of Harvard/Brown anxiety research project (HARP) findings. J. Clin. Psychiatry. 67(Suppl. 1), 14–19.

Google Scholar

Krijnen-De Bruin, E., Muntingh, A. D. T., Hoogendoorn, A. W., van Straten, A., Batelaan, N. M., Maarsingh, O. R., et al. (2019). The GET READY relapse prevention programme for anxiety and depression: a mixed-methods study protocol. BMC Psychiatry 19:64. doi: 10.1186/s12888-019-2034-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Kroenke, K., Spitzer, R. L., and Williams, J. B. W. (2001). The PHQ-9. J. Gen. Intern. Med. 16, 606–613. doi: 10.1046/j.1525-1497.2001.016009606.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Lamers, F., van Oppen, P., Comijs, H. C., Smit, J. H., Spinhoven, P., van Balkom, A. J. L. M., et al. (2011). Comorbidity patterns of anxiety and depressive disorders in a large cohort study: the Netherlands Study of Depression and Anxiety (NESDA). J. Clin. Psychiatry 72, 341–348. doi: 10.4088/JCP.10m06176blu

PubMed Abstract | CrossRef Full Text | Google Scholar

Levitt, A., Mueser, K., DeGenova, J., Lorenzo, J., Bradford-Watt, D., Barbosa, A., et al. (2009). Randomized controlled trial of illness management and recovery in multiple-unit supportive housing. Psychiatr Serv. 60, 1629–1636. doi: 10.1176/ps.2009.60.12.1629

PubMed Abstract | CrossRef Full Text | Google Scholar

Lorig, K. R., and Holman, H. R. (2003). Self-management education: history. Definition, outcomes, and mechanisms. Ann. Behav. Med. 26, 1–7. doi: 10.1207/S15324796ABM2601_01

CrossRef Full Text | Google Scholar

MacCallum, R. C., Widaman, K. F., Zhang, S., and Hong, S. (1999). Sample size in factor analysis. Psychol. Methods 4, 84–99. doi: 10.1037/1082-989X.4.1.84

CrossRef Full Text | Google Scholar

Matsunaga, M. (2010). How to factor-analyze your data right: do’s, Don’ts, and How-To’s. Int. J. Psychol. Res. 3, 98–111. doi: 10.21500/20112084.854

CrossRef Full Text | Google Scholar

Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., et al. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J. Clin. Epidemiol. 63, 737–745. doi: 10.1016/j.jclinepi.2010.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Orem, D. E. (2001). Nursing: Concepts of Practice, 5th Edn. St. Louis, MO: Mosby.

Google Scholar

Penninx, B. W. J. H., Nolen, W. A., Lamers, F., Zitman, F. G., Smit, J. H., Spinhoven, P., et al. (2011). Two-year course of depressive and anxiety disorders: results from the Netherlands Study of Depression and Anxiety (NESDA). J. Affect. Disord. 133, 76–85. doi: 10.1016/j.jad.2011.03.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Scholten, W. D., Batelaan, N. M., Penninx, B. W. J. H., van Balkom, A. J. L. M., Smit, J. H., Schoevers, R. A., et al. (2016). Diagnostic instability of recurrence and the impact on recurrence rates in depressive and anxiety disorders. J. Affect. Disord. 195, 185–190. doi: 10.1016/j.jad.2016.02.025

PubMed Abstract | CrossRef Full Text | Google Scholar

Schreurs, P., van de Willige, G., Tellegen, B., and Brosschot, J. (1988). De Utrechtse Coping Lijst: UCL-Handleiding [Utrecht Coping List: UCL manual]. Lisse: Swets & Zeitlinger.

Google Scholar

Severens, P. (2015). Manual Ariadne 3.0. Available online at: http://www.minds21.org/images_public/manual%20%20ARIADNE%203.0%20%20april%202015.pdf

Google Scholar

Shimizu, Y., and Paterson, B. L. (2007). A cross-cultural comparison of the developmental evolution of expertise in diabetes self-management. J. Clin. Nurs. 16, 350–357. doi: 10.1111/j.1365-2702.2007.02071.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Smith, D., Harvey, P., Lawn, S., Harris, M., and Battersby, M. (2017). Measuring chronic condition self-management in an Australian community: factor structure of the revised Partners in Health (PIH) scale. Qual. Life Res. 26, 149–159. doi: 10.1007/s11136-016-1368-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Sterling, E. W., von Esenwein, S. A., Tucker, S., Fricks, L., and Druss, B. G. (2010). Integrating wellness. Recovery, and self-management for mental health consumers. Commun. Ment. Health J. 46, 130–138. doi: 10.1007/s10597-009-9276-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Tabachnick, B., and Fidell, L. (2012). Using Multivariate Statistics, 6th Edn. Boston, MA: Allyn & Bacon.

Google Scholar

Terwee, C. B., Bot, S. D. M., de Boer, M. R., van der Windt, D. A. W. M., Knol, D. L., Dekker, J., et al. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. J. Clin. Epidemiol. 60, 34–42. doi: 10.1016/j.jclinepi.2006.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Trivedi, M. H., Rush, A. J., Ibrahim, H. M., Carmody, T. J., Biggs, M. M., Suppes, T., et al. (2004). The inventory of depressive symptomatology, clinician rating (IDS-C) and self-report (IDS-SR), and the quick inventory of depressive symptomatology, clinician rating (QIDS-C) and self-report (QIDS-SR) in public sector patients with mood disorders: a psych. Psychol. Med. 34, 73–82. doi: 10.1017/S0033291703001107

PubMed Abstract | CrossRef Full Text | Google Scholar

van Grieken, R. A., Kirkenier, A. C. E., Koeter, M. W. J., Nabitz, U. W., and Schene, A. H. (2015). Patients’ perspective on self-management in the recovery from depression. Heal Expect. 18, 1339–1348. doi: 10.1111/hex.12112

PubMed Abstract | CrossRef Full Text | Google Scholar

van Grieken, R. A., Kirkenier, A. C. E., Koeter, M. W. J., and Schene, A. H. (2014). Helpful self-management strategies to cope with enduring depression from the patients’ point of view: a concept map study. BMC Psychiatry 14:331. doi: 10.1186/s12888-014-0331-7

PubMed Abstract | CrossRef Full Text | Google Scholar

van Grieken, R. A., van Tricht, M. J., Koeter, M. W. J., van den Brink, W., and Schene, A. H. (2018). The use and helpfulness of self-management strategies for depression: the experiences of patients. PLoS One 13:e0209109. doi: 10.1371/journal.pone.0209109

PubMed Abstract | CrossRef Full Text | Google Scholar

Villaggi, B., Provencher, H., Coulombe, S., Meunier, S., Radziszewski, S., Hudon, C., et al. (2015). Self-management strategies in recovery from mood and anxiety disorders. Glob. Qual. Nurs. Res. 2:233339361560609. doi: 10.1177/2333393615606092

PubMed Abstract | CrossRef Full Text | Google Scholar

Walker, E. R., McGee, R. E., and Druss, B. G. (2015). Mortality in mental disorders and global disease burden implications. JAMA Psychiatry 72, 334–341. doi: 10.1001/jamapsychiatry.2014.2502

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, L., Watts, A., Anderson, R., and Little, T. (2013). “Common fallacies in quantitative research,” in Oxford Handbook of Qualitative Methods, ed. T. Little (New York, NY: Guilford), 718–758. doi: 10.1093/oxfordhb/9780199934898.013.0031

CrossRef Full Text | Google Scholar

World Health Organization (2016). Investing in Treatment for Depression and Anxiety Leads to Fourfold Return. Available online at: http://www.who.int/mediacentre/news/releases/2016/depression-anxiety-treatment/en/ (accessed June 27, 2017).

Google Scholar

Young, S. L., and Ensing, D. S. (1999). Exploring recovery from the perspective of people with psychiatric disabilities. Psychiatr. Rehabil. J. 22, 219–231. doi: 10.1037/h0095240

CrossRef Full Text | Google Scholar

Zoun, M. H. H., Koekkoek, B., Sinnema, H., Muntingh, A. D. T., van Balkom, A. J. L. M., Schene, A. H., et al. (2016). Effectiveness and cost-effectiveness of a self-management training for patients with chronic and treatment resistant anxiety or depressive disorders: design of a multicenter randomized controlled trial. BMC Psychiatry 16:216. doi: 10.1186/s12888-016-0927-1

PubMed Abstract | CrossRef Full Text | Google Scholar