Development and Preliminary Validation of a New Type 1 Diabetes Adjustment Scale (DAS-1)

Background: This study focuses on the development and validation of a new Type 1 Diabetes Adjustment Scale (DAS-1).

Method: A total of 204 participants aged 15–65 with type 1 diabetes completed the self-report measures of the DAS-1, which includes clinical and psychological variables.

Results: Robust confirmatory factor analysis detected a unidimensional structure of the item scores. The omega coefficient was 0.91 and test–retest reliability was 0.87. Classifying subjects as in a Positive or Negative mood state, ROC analysis yielded an optimal cut-off of 50 for the DAS-1 scores, with a clinical accuracy of AUC = 0.85. The DAS-1 demonstrated evidence of good reliability and acceptable construct validity.

Conclusion: The DAS-1 demonstrated good clinical utility, good sensitivity and adequate specificity. Clinical and theoretical implications of these results are discussed.

Introduction

Scientific interest in understanding how psychological factors may determine medical conditions is growing. Psychological aspects may seriously impact somatic symptoms and medical outcomes, especially in chronic diseases (Martino et al., 2019). After the initial impact of the diagnosis of a chronic disease such as type 1 diabetes (T1D), an adjustment period begins in which the individual must learn to live with the disease, which implies that these patients must take actions to become accustomed to their new circumstances (de Ridder et al., 2008). A diagnosis of T1D also leads to deterioration in the patient’s health. Some authors (Isla Pera et al., 2008) who have studied adaptation to T1D have considered the process to be equivalent to the mourning process described by Kübler-Ross (1993), which includes the following stages: denial, rebellion, negotiation, depression and acceptance. In the case of T1D this latter stage would be the adaptation to the disease involving a relative sense of well-being and balance. If the process concludes positively, patients with T1D are able to adapt to the new situation while maintaining quality of life. However, if the adaptation process does not occur or is inappropriate, these patients will find that their quality of life as well as their psychological and physical well-being will be affected (Malik and Koot, 2009). This perception of balance is unstable and can be disrupted by various situations such as personal or family issues or problems produced by T1D. Consequently, the person with T1D will have to achieve a new balance to adjust to the disease throughout his or her life. “Adjustment to diabetes” is therefore understood as the healthy rebalancing that the patient must undergo to the new circumstances of diabetes (de Ridder et al., 2008).

Factors including the demands of treatment, the impact of the diagnosis, uncertainty about possible complications and frustration with unexplained blood glucose levels or lifestyle restrictions (Anarte, 2004) can trigger the onset of negative emotions. These negative emotions can lead to elevated depressive symptoms (Robertson et al., 2012), making it difficult for the patient to adapt to diabetes. In addition, because T1D is a chronic disease, adjustment to it can change over a patient’s life course at different times such as when learning to self-manage the disease, during life transitions, impacting disease self-management, disease progression, and the onset of complications (Young-Hyman et al., 2016).

Other factors associated with diabetes treatment adherence may influence patient adjustment to T1D such as coping styles, depressive symptoms, perceived social support, and diabetes-related distress (Di Matteo, 2004; Ortiz, 2006; Sacco and Yanover, 2006; Fisher et al., 2014). These studies suggest it is difficult to achieve adequate glycemic control due to the influence of these variables.

Personal resources such as coping strategies produce improved adjustment to T1D. Greater psychological adjustment has been found in those patients who use strategies focused on the problem versus other strategies (Duangdao and Roesch, 2008), showing an increase in well-being, mood and self-esteem. Other authors agree that problem-focused coping is more beneficial to the patient’s health than emotion-focused coping (Frazier, 2000). The study of coping in T1D was of great interest after Peyrot et al. (1999) reported that emotion-centered coping and problem-centered coping may be associated with glycemic control. Similarly, Ortiz (2006) associates better metabolic adjustment with behavioral coping than with emotional coping, while Boland and Grey (1996) find that emotion-centered coping is associated with poor metabolic control, including nonadherence to the therapeutic regimen.

Psychosocial factors play an integral role in managing diabetes but differ according to the type. T1D and Type 2 diabetes (T2D) are distinct conditions that have different psychological effects. Delamater et al. (2001) have reported how Psychosocial functioning, Neurocognitive functioning, Psychosocial factors associated with regimen adherence, metabolic control, and quality of life are affected. Neurocognitive deficits have been observed in adults with T1D, particularly those with at least five episodes of severe hypoglycemia, and in patients with peripheral neuropathy. Among older adults with T2D, cognitive deficits have been reported in association with poor glycemic control. More recently, it has been found that adults with T1D report feeling judged negatively for not managing their diabetes “perfectly” (Browne et al., 2014) while adults with T2D report feeling blamed and shamed for “bringing the condition on themselves” (Browne et al., 2013).

It would appear that both types of diabetes need to be assessed with different instruments (Polonsky et al., 2005). Indeed, a number of specific instruments have been developed to assess quality of life (Bott et al., 1998), distress (Polonsky et al., 2005; Fisher et al., 2015), or confidence in diabetes self-care (Van Der Ven et al., 2003; Polonsky et al., 2009). Although several instruments have been developed to assess adjustment to diabetes, they have some limitations. Some evaluate acceptance either to disease in general (Linkowski, 1971) or to diabetes (Schmitt et al., 2018), and others assess adjustment in adolescents (Sullivan, 1979; Wysocki et al., 1992), emotional adjustment to diabetes (Deleon, 1995) or adjustment in patients with T2D (Ebrahimi et al., 2016). However, none of these instruments specifically evaluate adjustment to diabetes in adults with T1D. In addition, the concept of adjustment is different in each of these. For these reasons, the aim of this study was to develop a new instrument specifically designed to provide a reliable and valid measurement of adjustment to T1D in adults.

Materials and Methods

Participants

A total of 204 participants with T1D were included (48% men and 52% women), with a mean age of 33.82 years (SD = 11.09; Range: 15–65), from the Diabetes Unit (Department of Endocrinology and Nutrition, Regional University Hospital of Malaga, Spain) between 2009 and 2012. Women with gestational diabetes were excluded. The mean number of years with T1D was 15.16 (SD = 9.64; Range: 0.04–50), and mean glycosylated hemoglobin (HbA_1c) was 7.61% (SD = 1.41; Range: 5.10–14.00%). Additional sample characteristics are shown in Table 1.

TABLE 1

Table 1. Characteristics of the study sample.

Measures

Development of the Self-Rating Diabetes Adjustment Scale for Type 1 Diabetes (DAS-1)

The items were developed based on the factors that influence patient adjustment to diabetes according to the scientific literature. Patients encounter barriers in diabetes self-management or in their lifestyle (Interference: items 6,8,9,10,11,12,13). People with diabetes often feel frustrated, tired or overwhelmed by the demands of diabetes (Rubin, 2005), which affect its course. The demands of T1D treatment and the delicate balance that patients must maintain draw on all their resources, often triggering the onset of negative emotions (Negative emotions: items 1, 4, 5), such as distress or depression (Anarte, 2004; Peyrot et al., 2005; Lloyd et al., 2010). The impact of diabetes on patients and their relatives (family) can be important (Diabetes Impact: items 2,3,7,15). Indeed, this stage is related to nonclinical adjustment behaviors and symptoms in the subjects (Young-Hyman et al., 2016). The patient’s adjustment to the new situation depends considerably on the coping strategies used to manage the disease (Coping style: items 14,16,19,20). As indicated above, the type of coping used will determine better or worse psychosocial adjustment and glycemic control (Duangdao and Roesch, 2008; Jaser et al., 2012). Treatment adherence is a basic pillar for good control of T1D. Some patient behaviors, such as avoidance, may interfere with treatment adherence behaviors (items 17,18), which would imply worse glycemic control and poorer self-care (Jaser et al., 2012).

Based on theoretical background, a pool of 20 items (Table 2) was selected by consensus among four external experts (two psychologists, specialized in health psychology and diabetes, and two endocrinologists). Individuals with T1D (N = 25) completed the 20 items in the presence of a psychologist and were allowed to ask questions. It was determined whether the item was properly understood or was more understandable using different wording. Finally, the item pool was reviewed by all the authors to identify redundancy, language comprehension and other issues. The DAS-1 is a visual analog, numerical rating scale. Items are rated on a five point scale from 1 (not at all) to 5 (extremely). A higher score on the DAS-1 scale indicates poorer patient adjustment to the disease.

TABLE 2

Table 2. Type 1 diabetes adjustment scale (DAS-1): items scale.

Sociodemographic Variables

In order to study the sociodemographic variables of the patients, a structured interview was administered during the evaluation period. Age and number of years with diabetes were collected and used as continuous variables.

Biomedical Variables

For the collection of the biomedical data, medical staff completed a structured interview with each patient, recording the following variables:

- Dichotomous response variables (Yes/No): diabetes complications; other chronic diseases; and family, partner, work, and personal health support.

- Categorized response variables: body mass index (BMI), HbA_1c (%).

- Discrete response variables: number of hypoglycemic/hyperglycemic episodes per week, frequency of daily self-monitoring.

Body mass index was calculated using the formula: weight/height². Glycosylated hemoglobin (HbA_1c), measured by high pressure liquid chromatography with a Kyoto Daiichi Kagaku device, was used as an indicator of metabolic control.

Mood, Satisfaction and Concern About Diabetes

Variables were measured on a 5-point scale from 1 (not at all) to 5 (extremely) using the following questions: Mood State (MST): In general, what is your mood state? Satisfaction with time to control diabetes (TCD): Are you satisfied with the amount of time you take in controlling your diabetes? (1 Very much to 5 Very little). Satisfaction with the knowledge of diabetes (SKD): Are you satisfied with your knowledge about diabetes? (1 Very much to 5 Not at all). Worried about experiencing complications due to diabetes (SCD): Are you worried about experiencing complications because of diabetes in the future? (1 Very much to 5 Not at all).

Diabetes Quality of Life (DQOL)

The DQOL (The Diabetes Control and Complications Trial Research Group [DCCT], 1988) measures the quality of life of persons with diabetes. The Spanish version (Millán et al., 2002) consists of 43 items that form four dimensions: Satisfaction with treatment, Impact of treatment, Concern regarding social and vocational aspects and Concern regarding the future effects of diabetes. Items are rated on a 5-point Likert-type scale (1–5) with lower scores indicating higher quality of life. Cronbach’s α values for the four subscales were 0.87, 0.85, 0.80, and 0.75, respectively.

Beck Depression Inventory (BDI-II)

This instrument evaluates the intensity of the depressive symptoms that an individual presents. This self-administered questionnaire is composed of 21 multiple-response items (0, 1, 2, 3), according to the severity of the symptom (from 0, which indicates the absence of the symptom, to 3, which represents the maximum severity of the symptom). The Spanish adaptation of the BDI-II (Beck et al., 1996) by Sanz et al. (2005) was used. Cronbach’s α value was 0.94.

Self-Rating Depression Scale (SDS)

The SDS (Zung, 1965) is composed of 20 items of which two are affective, eight refer to somatic correlates and 10 to psychological correlates (10 positive and 10 negative). The items are assessed on a scale of 1–4 points concerning the frequency with which each behavior included in the scale occurs. The internal consistency in the Spanish adaptation (Conde, 1969) was 0.88.

The State-Trait Anxiety Inventory (STAI)

To evaluate anxiety, the Spanish adaptation of the STAI (Spielberger et al., 1982) by Seisdedos (1988) was used. It comprises two self-assessment scales with 20 items rated on a Likert-type scale from 0 to 3: the State Anxiety Inventory (STAI-S) and the Trait Anxiety Inventory (STAI-T). The STAI-S assesses the subject’s status when faced with threatening situations at a given time, while the STAI-T assumes a permanence of anxiety in the subject, evaluating the disposition to respond with high rates of anxiety to stressful situations. Cronbach’s α values were 0.92 and 0.95, respectively.

Procedure

This study was evaluated by the Research and Ethics Committee of the Regional University Hospital of Malaga and received a positive assessment. Consent was obtained from all participants after they were informed about the purpose of the study and the voluntary nature of their participation. The sociodemographic and clinical variables of the participants were collected through a structured interview together with the psychological variables by a clinical psychologist in an examination room of the Diabetes Unit during a visit with the endocrinologist. The participants completed the 20-item DAS-1, the questionnaires and the variables described above. To explore the test–retest reliability of the DAS-1, 36 randomly selected patients completed all the DAS-1 items twice, over a mean interval of 2 weeks. The participants did not receive economic compensation.

Data Analysis

Preliminary analyses with univariate descriptive statistics of the items (mean, standard deviation, skewness and kurtosis, outliers) were conducted. The definition used for an outlier was based on Moore et al. (2009). An observation was a suspected outlier when it fell more than 1.5 × the Interquartile Range (IQR) above the third quartile or below the first quartile. Kolmogorov–Smirnov tests for univariate normality and Mardia’s test for multivariate normality (Mardia, 1970) were also performed.

Parallel Analysis (Horn, 1965) and Very Simple Structure (VSS) (Revelle and Rocklin, 1979) were carried out to explore the number of underlying dimensions. As these indices suggested, a one-factor model was tested. Robust Confirmatory Factor Analysis (RCFA) and a Robust Weighted Least Squares estimation method for categorically ordered data were conducted using a polychoric correlation matrix. Goodness of fit was evaluated using the following indices: relative chi-square ratio (χ²/df), standardized root mean square residual (SRMR), root mean square error of approximation (RMSEA), the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the weighted root mean residual (WRMR). Model fit was assessed by the following criteria: For a good fit model, the ratio χ²/df should be as small as possible. A ratio 0≤χ²/df≤2 is indicative of a good fit and 2 < χ²/df≤3 for an acceptable data-model fit (Schermelleh-Engel et al., 2003). The 0≤RMSEA≤0.05 value is considered a good fit, 0.05 < RMSEA < 0.10 an acceptable fit, and RMSEA≥0.10 a poor fit (Browne and Cudeck, 1993). The 0≤SRMR≤0.05 is considered a good fit, whereas values 0.05 < SRMR≤0.10 may be interpreted as an acceptable fit (Hu and Bentler, 1995). The CFI and TLI values should be greater than or close to 0.90 for a good fit (Hopwood and Donnellan, 2010). Yu and Muthén (2002) recommend the WRMR over the SRMR for categorical indicators, with good fit at values close to and below 1.00.

Internal consistency was analyzed with omega (ω) (McDonald, 1999) and item analysis was calculated with ω deleting each item in turn. The 95% confidence intervals (CI) for ω and ω (-item) were also calculated. Test–retest reliability was obtained using the Interclass Correlation Coefficient (ICC). ICC values less than 0.50, between 0.50 and 0.75, between 0.75 and 0.90, and greater than 0.90 are indicative of poor, moderate, good, and excellent reliability, respectively (Koo and Li, 2016).

The criterion validity of the DAS-1 scores for the variables MST, TCD, SKD, and SCD was analyzed using Pearson’s correlation coefficients. According to Evers et al. (2013), a criterion validity value can be considered Inadequate (r < 0.20), Adequate (0.20 _≤ r < 0.35), Good (0.35 _≤ r < 0.50) or Excellent (r _≥ 0.50). Receiver operating characteristics (ROC) curve analysis was conducted to determine the optimal cut-off value for the DAS-1 scores (Zweig and Campbell, 1993; Bonillo et al., 2000) for differentiating between the established Positive (PMS) or Negative (NMS) mood state of the criterion groups, dichotomizing by the MST mean. Other criterion validity indices were evaluated using Pearson’s correlation between the DAS-1 and measures of quality of life, depression and anxiety.

Construct validity evidence was examined with Pearson’s correlation between the DAS-1 scale scores and the clinical variables (number of hypoglycemic/hyperglycemic episodes per week, frequency of daily self-monitoring), age, and number of years with diabetes. Interpretation of Pearson’s correlation was according to the criteria of Evers et al. (2013). Additional construct validity evidence was analyzed using Student-Welch’s t-test comparing DAS-1 means in groups defined by clinical variables including diabetes complications, other chronic diseases, and family, partner, work or personal health support. The effect size of the mean difference (d), power (1-β) and 95% CI were also calculated. DAS-1 median comparison was performed using the Kruskal–Wallis test in groups defined by HbA_1c and BMI. These analyses were performed using different R packages [psych version 1.7.3.21 (Revelle, 2015), paran version 1.5.1 (Dinno, 2009), MBESS version 4.2.0 (Kelley, 2007), lavaan 0.5–12 (Rosseel, 2012)] and IBM SPSS Statistics, version 14.0.

Results

Preliminary Analysis

Item descriptive statistics are shown in Table 3. All items ranged from 1 to 5. Item skewness values ranged from 0.20 to 2.36. There were six items with skew greater than one. Item kurtosis values were between −1.37 and 6.03, with item 18 showing a highly leptokurtic distribution. The number of suspected outliers of items based on the 1.5 x IQR rule are shown in Table 3. The significant Kolmogorov–Smirnov test (p < 0.001) indicated that the answers to each item did not satisfy normality.

TABLE 3

Table 3. Item descriptive statistics, ω and 95% CI ω indices.

There was no multivariate normality [Mardia’s skewness and kurtosis tests were significant, p < 0.001 with χ²(Skew) = 3008.23(88.48) and Z−value(Kurtosis) = 19.05(519.15)]. All correlations were below 0.78, indicating no multicollinearity in the data.

In the study of underlying dimensionality, Horn’s parallel analysis (mean = 3.29) and VSS Complexity 1 (maximum = 0.97) suggested a one-factor solution.

Robust Confirmatory Factor Analysis

A one-factor model was tested using RCFA. Figure 1 shows the corresponding path diagram with standardized loadings and estimated variances of the 20 DAS-1 items. All loadings exceeded the threshold 0.40 with the exception of item 19 (0.29) and all were significant (p < 0.001). The estimated item variances were >0.40 except for i2 (0.39), i5 (0.37), and i7 (0.34). Robust goodness of fit indices were: χ²/df = 2.69, RMSEA = 0.09 (90% CI: 0.08–0.10), SRMR = 0.08, WRMR = 1.13, CFI = 0.93, and TLI = 0.92. These results indicate an acceptable fit, and this dimension was named Adjustment to Diabetes.

FIGURE 1

Figure 1. Path diagram of the Robust Confirmatory Factor Analysis.

Reliability and Item Analysis

Internal Consistency of the DAS-1 was ω (0.91) and ω 95% CI [0.88, 0.93]. ω if each item is deleted ranged from 0.89 to 0.91. ω if each item is deleted was lower than 0.91, except for ω if item 19 is deleted being equal to 0.91, indicating that no item should be removed from the scale. These indices and their 95% CI are shown in Table 3. All 20 items are considered in the following analyses.

Test–Retest Reliability

The DAS-1 scores in Time-1 (M = 40.11, SD = 13.16, N = 36) and Time-2 (M = 39.33, SD = 14.75, N = 36) showed an ICC = 0.87.

The DAS-1 is a unidimensional scale with an acceptable fit by RCFA, has an excellent internal consistency (≥ 0.90) (Evers et al., 2013), and good test–retest reliability. The DAS-1 total score distribution showed a moderate skew (0.50 < skew < 1), to be slightly leptokurtic and with two outliers (Table 3).

Criterion-Related Validity

The variables MST, TCD, SKD, SCD have low or moderate skewness values (<1) and low kurtosis values with MST and SKD having fewer outliers, one and zero, respectively (Table 4). These variables did not indicate a strong deviation from the normal distribution in the sample, but the Kolmogorov–Smirnov normality test was significant (p < 0.001) for all the variables. Pearson’s correlations between DAS-1 scores and criterion variables were MST (r = 0.65), TCD (r = 0.52), SKD (r = 0.43), and SCD (r = 0.60). According to the criteria of Evers et al. (2013), these are all excellent values except for SKD, which has a good criterion validity value.

TABLE 4

Table 4. Distributional characteristics of the criterion variables.

The variable MST was dichotomized by the mean. The PMS (N = 143) and NMS (N = 61) categories were considered. ROC analysis was conducted taking into account the proportion (0.70) of patients in a PMS in this sample. The optimal cut-off score (50) for the DAS-1 was determined. Of our total sample, 150 subjects (73.5%) scored lower than 50 (better adjustment), and 54 subjects (26.5%) scored equal to or greater than 50 (poorer adjustment) according to the DAS-1. The AUC was 0.85. Efficacy indices for this cut-off were: sensitivity = 90.21% (95% CI: 85.28–93.92) specificity = 65.57% (95% CI: 89.35–99.95), overall efficiency = 82.84%. Correlation between categories (PMS and NMS) and classification on the DAS-1 (poorer/better adjustment; cut-off = 50) was φ = 0.58. Thus, 34% of the variability between PMS and NMS could be explained by the variability in the adjustment established by this cut-off.

Range, skewness, kurtosis, and number of outliers for the subscales (Dissatisfaction, Impact, Social concern, Concern for the future) and total DQOL are shown in Table 4. Among these, the following should be noted: the subscale Concern for the future has high skewness, high kurtosis and a high number of outliers. The Kolmogorov–Smirnov test was statistically significant for all the subscales, except for Dissatisfaction and DQOL. According to the criteria of Evers et al. (2013), the results showed excellent associations between the DAS-1 and the DQOL subscales Dissatisfaction (r = 0.77), Impact (r = 0.81), Social concern (r = 0.56), Concern for the future (r = 0.67), and between the DAS-1 and DQOL (r = 0.86). Better adjustment to T1D indicates better quality of life.

Properties of the score distributions for the instruments (BDI-II, SDS, STAI-T and STAI-S) are provided in Table 4. BDI-II and STAI-S distributions have skew >1, and they are more leptokurtic than the rest of the variables, highlighting the kurtosis (7.07) of STAI-S. The Kolmogorov–Smirnov test was statistically significant for all the scales. Excellent associations were found between the DAS-1 and BDI-II (r = 0.75), SDS (r = 0.70) STAI-T (r = 0.74), and STAI-S (r = 0.73). Better adjustment to T1D (lower DAS-1 scores) indicates a better emotional state (lower BDI, SDS, and STAI scores) and vice versa.

Additional Construct Validity Evidence

Inadequate associations were found between the DAS-1 and the following clinical variables: Number of episodes per week of hypoglycemia (r = −0.08, p = 0.25) or hyperglycemia (r = 0.15, p < 0.05), Self-monitoring frequency (r = −0.12, p = 0.10), Age (r = 0.08, p = 0.27), and Number of years with diabetes (r = −0.03, p = 0.71). A statistically significant relationship was found between the degree of adjustment to T1D measured with the DAS-1 and the number of hyperglycemic episodes per week given the poor metabolic control of a large number of participants (60.4%). The remaining clinical variables did not reach statistical significance.

Differences in the DAS-1 score means in the two groups (Yes/No) established by the variables Other chronic diseases, Family, Partner, Work and Personal health support, and Diabetes complications were analyzed to obtain further evidence of construct validity. Means and Standard deviations of the DAS-1 scores for these variables in each group are shown in Table 1. This Table also shows the number and percentage of sample subjects in each group (Yes/No). Student’s t-test results comparing the DAS-1 score means in both groups for each variable are provided in Table 5. Results showed that the DAS-1 score means differed significantly (p < 0.05) in both groups (Yes/No) established by the variables Other chronic diseases and Family, Partner and Work support. No statistically significant differences were found in Diabetes complications and Personal health support. Hypothesis testing showed high power in all variables except for Diabetes complications and Partner support (<0.80). The size of the mean differences (d) on the DAS-1 was large for Family (1.17) and Personal health support (1.00), medium for Other chronic diseases (0.50) and Work support (0.50) and small for Partner support (0.33) and Diabetes complications (0.49).

TABLE 5

Table 5. Results of the t-test on the DAS-1 scores in the criterion variables.

Preliminary analysis of HbA_1c% (N = 202) showed skew = 2.10, kurtosis = 6.46 and 9 outliers [Q3+1.5(IQR) = 10.45] in this sample. The HbA_1c% groups are shown in Table 1. The Kruskal–Wallis test (${\mathrm{\chi }}_2^2=9.33$, p < 0.001) displayed significant differences between groups in the DAS-1 medians (38, 38, 55 in the sample, respectively). Multiple comparisons testing showed significant differences (p < 0.05) between groups (HbA_1c% ≤ 7) and (7 < HbA_1c% < 10) regarding extremely poor metabolic control (HbA_1c% = 10). No significant differences (p = 0.97) were seen between groups (HbA_1c% ≤ 7) and (7 < HbA_1c% < 10). Subsequently, the DAS-1 detects differences in HbA_1c, differentiating between those who had good, poor or extremely poor metabolic control.

Preliminary analysis of BMI (N = 199) showed skew = 0.23, kurtosis = 4.46 and 5 outliers [Q3+1.5(IQR) = 34.16] in this sample. The BMI scores had outliers, showing a highly leptokurtic distribution. The minimum outlier value for BMI was 34.16. After eliminating the two outliers in the DAS-1 scores (N = 197), the BMI groups are presented in Table 1. Kruskal–Wallis testing (${\mathrm{\chi }}_2^2=0.09$, p = 0.95) showed no statistically significant differences between these groups in the DAS-1 medians (38.5, 39, 42.5 in the sample, respectively).

Discussion

The purpose of this study was to develop a scale to quantify adjustment to diabetes specifically in patients with T1D. This new instrument has shown adequate psychometric properties in this sample of T1D patients. The DAS-1 can be used to differentiate between good or poor adjustment to T1D using the optimal cut-off obtained (50), demonstrating its clinical utility. Using this new instrument, 26.5% of the subjects in this study were not well adjusted to diabetes. This figure is in line with a previous publication that situates the number of subjects in whom this phase is prolonged or who do not successfully adjust at around 30% (de Ridder et al., 2008).

The DAS-1 scores have also shown an adequate degree of validity. Factorial validity is supported by our findings affirming the acceptable fit of a one-factor structure adjustment to T1D. Scores also have excellent internal consistency and good test–retest reliability.

In addition to the results concerning the psychometric properties of the scale, the analysis of the relationship between the DAS-1 and other variables provides a series of data that should be mentioned. The association between the DAS-1 scale and the question “In general, what is your mood state?” (MST) is of note. One of the aspects that indicate the degree of the patient’s adjustment to the disease is mood (de Ridder et al., 2008). A negative perception indicates poor adjustment to diabetes. Accordingly, it is understood that the variability in the total DAS-1 scores explains much of the variability in the patient’s general mood perception (MST).

Criterion validity is also supported by an excellent/good association between the DAS-1 and the measure of negative emotions and quality of life. Thus, better adjustment to T1D (low DAS-1 scores), indicates a better emotional state (lower BDI, SDS and STAI scores), which is consistent with previous literature (Peyrot et al., 2005; Lloyd et al., 2010). High correlations with DQOL (total scores and Impact) were found. Higher scores on the DAS-1 (poorer adjustment to diabetes) are associated with higher scores on the DQOL (total scores and Impact), indicating poorer quality of life. This shows that high adjustment to T1D predicts high perceived quality of life (Malik and Koot, 2009).

The observed associations between the DAS-1 and several clinical variables display additional evidence of DAS-1 construct validity as a statistically significant relationship was found between DAS-1 scores and the number of hyperglycemic episodes per week. Poor metabolic control (60.4% of the participants in our study) indicates poor adjustment to T1D as has been noted in the scientific literature (Di Matteo, 2004; Ortiz, 2006; Sacco and Yanover, 2006; Fisher et al., 2014), and it has been suggested that it is difficult to achieve glycemic control without considering the factors that influence adjustment to diabetes.

Further evidence of the construct validity of the DAS-1 was analyzed by comparing the mean scores of the DAS-1 in groups defined by other clinical variables. It was found that people with family support, partner support or work support have better adjustment to T1D. This is also an important finding of this study, suggesting that these variables should be considered as possible protective factors that facilitate adjustment, contributing to reduce the impact of diabetes (Kovacs et al., 2013; Young-Hyman et al., 2016). Indeed, the DAS-1 total score showed a high association with the DQOL Impact subscale and total scores. Thus, as the literature demonstrates, the better the patient’s adjustment to the disease, the better the patient’s perceived well-being. Conversely, having additional chronic diseases increases DAS-1 scores, showing poorer adjustment (Malik and Koot, 2009).

Differences were found between the DAS-1 and HbA_1c levels. Higher DAS-1 scores predict higher HbA1c, differentiating between those with good, poor or extremely poor glycemic control, which represents further evidence of construct validity. This finding is consistent with previous literature (Trief et al., 1998), and provides new evidence differentiating not only among those who have good or poor glycemic control but also extremely poor glycemic control.

The results of this study support the conclusion that the DAS-1 is a good instrument to evaluate the adjustment to diabetes in persons with T1D. Although several instruments have been developed to assess this adjustment, no specific instruments are available to evaluate the adjustment to diabetes in the clinical population of adult T1D patients. The DAS-1 scale offers the following advantages: (a) it is a specific assessment instrument for adults with T1D; (b) it is a short instrument that is quick and easy to administer; and (c) it is based on the optimal cut-off.

The DAS-1 offers professionals who provide care to individuals with T1D a useful tool to quickly and reliably identify patients who adjust poorly to diabetes to prevent adverse consequences such as poor quality of life (Malik and Koot, 2009) or negative emotions that can lead to elevated depressive symptoms (Robertson et al., 2012). This is especially important as increased depressive symptoms are associated with an increase in the severity and number of diabetes complications (Egede, 2005). The relative risk of macrovascular complications among people with diabetes and depression is 2.5 times higher than those without this condition; the risk of microvascular complications is more than 11 times higher; almost 7 times higher for disability and almost 5 times higher for mortality (Von Korff et al., 2005). In addition, a high rate of depression is associated with low adherence to medical treatment of diabetes, resulting in a total health care cost that is 4.5 times higher than for those with diabetes without depression (Egede et al., 2002).

The need for psychosocial support in the care of patients with diabetes to optimize health outcomes and health-related quality of life is recognized by the American Diabetes Association [ADA] (2020) and recommended with the highest criteria (A). Use of the DAS-1 would allow patients to receive early intervention and suitable treatment. Specific intervention programs could be designed for these patients, and adequate psychosocial care could be provided by establishing protocols for adjustment to T1D to prevent the adverse consequences mentioned and to minimize the associated costs. These protocols should consider: the barriers encountered by patients in diabetes self-management or in their lifestyle (Rubin, 2005), negative emotions such as distress or depression (Anarte, 2004; Peyrot et al., 2005; Lloyd et al., 2010), the impact of diabetes on patients and their relatives (Young-Hyman et al., 2016), the use of appropriate diabetes coping strategies (Duangdao and Roesch, 2008; Jaser et al., 2012), and improving treatment adherence (Jaser et al., 2012).

Although the preliminary results obtained with the DAS-1 provide evidence of its reliability, validity and usefulness, further studies are needed and many issues remain to be addressed to improve our understanding, with particular regard to psychological factors as determinants in the adjustment to such diseases as diabetes (Martino et al., 2019). Use of the DAS-1 to assess psychological adjustment to diabetes may help predict risk in people with T1D by identifying individuals with low levels of adjustment to the disease.

Data Availability Statement

The datasets generated for this study are not made publicly available as the patients have provided their informed consent for the collection of data for the research but not for its publication. Requests should be directed to YW5hcnRlQHVtYS5lcw==.

Ethics Statement

The studies involving human participants were reviewed and approved by the University of Málaga. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

MA, MC, MD-L, and MR contributed to conception and design of the study. MC organized the database. TR performed the statistical analysis. MA and TR wrote the first draft of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

Funding

This research was supported by grants from the Spanish Ministry of Economy and Competitiveness (Project Ref. PSI2011-27820) and within the framework of CIBERDEM, and the Regional Ministry of Education, Science and Innovation of Andalusia (Research Group CTS-278).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Francisco Félix Caballero for his collaboration at the beginning of this study.

References

American Diabetes Association [ADA] (2020). Facilitating behavior change and well-being to improve health outcomes: standards of medical care in diabetes-2020. Diabetes Care 43, S48–S65. doi: 10.2337/dc20-S005

PubMed Abstract | CrossRef Full Text | Google Scholar

Anarte, M. T. (2004). Importancia del estrés en la diabetes. Célula Beta 18, 18–21.

Google Scholar

Beck, A. T., Steer, R. A., and Brown, G. K. (1996). Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation.

Google Scholar

Boland, E. A., and Grey, M. (1996). Coping strategies of school-age children with diabetes mellitus. Diabetes. Educ. 22, 592–597. doi: 10.1177/014572179602200607

PubMed Abstract | CrossRef Full Text | Google Scholar

Bonillo, A., Doménech, J. M., and Granero, R. (2000). Macros SPSS Para Análisis de Datos en Ciencias de la Salud. Barcelona: Signo.

Google Scholar

Bott, U., Muhlhauser, I., Overmann, H., and Berger, M. (1998). Validation of a diabetes-specific quality-of-life scale for patients with type 1 diabetes. Diabetes Care 21, 757–769. doi: 10.2337/diacare.21.5.757

PubMed Abstract | CrossRef Full Text | Google Scholar

Browne, J. L., Ventura, A., Mosely, K., and Speight, J. (2013). ‘I call it the blame and shame disease’: a qualitative study about perceptions of social stigma surrounding type 2 diabetes. BMJ Open 3:e003384. doi: 10.1136/bmjopen-2013-003384

PubMed Abstract | CrossRef Full Text | Google Scholar

Browne, J. L., Ventura, A., Mosely, K., and Speight, J. (2014). ‘I’m not a druggie, I’m just a diabetic’: a qualitative study of stigma from the perspective of adults with type 1 diabetes. BMJ Open 4:e005625. doi: 10.1136/bmjopen-2014-005625

PubMed Abstract | CrossRef Full Text | Google Scholar

Browne, M. W., and Cudeck, R. (1993). “Alternative ways of assessing model fit,” in Testing Structural Equation Models, eds K. A. Bollen and J. S. Long (Beverly Hills, CA: Sage), 136–162.

Google Scholar

Conde, V. (1969). La escala Hetero-Aplicada y Hetero-Valorada Para la Depresión de Zung-Conde. Salamanca: Cervantes.

Google Scholar

de Ridder, D., Geenen, R., Kuijer, R., and van Middendorp, H. (2008). Psychological adjustment to chronic disease. Lancet 372, 246–255. doi: 10.1016/S0140-6736(08)61078-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Delamater, A., Jacobson, A. M., Anderson, B., Cox, D., Fisher, L., Lustman, P., et al. (2001). Psychosocial therapies in diabetes. Report of the psychosocial therapies working group. Diabetes Care 24, 1286–1292. doi: 10.2337/diacare.24.7.1286

PubMed Abstract | CrossRef Full Text | Google Scholar

Deleon, O. A. (1995). Development, reliability and validation of the diabetes emotional adjustment scale in Spanish – preliminary findings. Int. J. Psychiatry Med. 25, 81–92. doi: 10.2190/95FU-2TVW-UA8C-LPN8

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Matteo, M. R. (2004). Social support and patient adherence to medical treatment: a meta-analysis. Health Psychol. 23, 207–218. doi: 10.1037/0278-6133.23.2.207

PubMed Abstract | CrossRef Full Text | Google Scholar

Dinno, A. (2009). Exploring the sensitivity of horn’s parallel analysis to the distributional form of simulated data. Multivariate Behav. Res. 44, 362–388. doi: 10.1080/00273170902938969

PubMed Abstract | CrossRef Full Text | Google Scholar

Duangdao, K. M., and Roesch, S. C. (2008). Coping with diabetes in adulthood: a meta-analysis. J. Behav. Med. 31, 291–300. doi: 10.1007/s10865-008-9155-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Ebrahimi, H., Karimi Moonaghi, H., Asghari Jafarabadi, M., Namdar Areshtanab, H., and Jouybari, L. (2016). Development and preliminary validation of diabetes adjustment assessment scale (DAAS): a new measure of adjustment with type 2 diabetes. J. Caring Sci. 5, 145–152. doi: 10.15171/jcs.2016.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Egede, L. E. (2005). Effect of comorbid chronic diseases on prevalence and odds of depression in adults with diabetes. Psychosom. Med. 67, 46–51. doi: 10.1097/01.psy.0000149260.82006.fb

PubMed Abstract | CrossRef Full Text | Google Scholar

Egede, L. E., Zheng, D., and Simpson, K. (2002). Comorbid depression is associated with increased health care use and expenditures in individuals with diabetes. Diabetes Care 25, 464–470. doi: 10.2337/diacare.25.3.464

PubMed Abstract | CrossRef Full Text | Google Scholar

Evers, A., Hagemeister, C., Høstmælingen, A., Lindley, P., Muñiz, J., and Sjöberg, A. (2013). EFPA Review Model for the Description and Evaluation of Psychological and Educational Tests, Version 4.2.6. Available online at: http://www.efpa.eu/download/650d0d4ecd407a51139ca44ee704fda4 (accessed November 16, 2019).

Google Scholar

Fisher, L., González, J. S., and Polonsky, W. H. (2014). The confusing tale of depression and distress in patients with diabetes: a call for greater clarity and precision. Diabetes Med. 26, 201–210. doi: 10.1111/dme.12428

PubMed Abstract | CrossRef Full Text | Google Scholar

Fisher, L., Polonsky, W. H., Hessler, D. M., Masharani, U., Blumer, I., Peters, A. L., et al. (2015). Understanding the sources of diabetes distress in adults with type 1 diabetes. J. Diabetes Complication 29, 572–577. doi: 10.1016/j.jdiacomp.2015.01.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Frazier, L. (2000). Coping with disease-related stressors in Parkinson’s disease. Gerontologist 40, 53–63.

Google Scholar

Hopwood, C. J., and Donnellan, M. B. (2010). How should the internal structure of personality inventories be evaluated? Pers. Soc. Psychol. Rev. 14, 332–346. doi: 10.1177/1088868310361240

PubMed Abstract | CrossRef Full Text | Google Scholar

Horn, J. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika 30, 179–185. doi: 10.1007/BF02289447

PubMed Abstract | CrossRef Full Text | Google Scholar

Hu, L., and Bentler, P. (1995). “Evaluating model fit,” in Structural Equation Modeling. Concepts, Issues, and Applications, ed. R. H. Hoyle (London: Sage), 76–99.

Google Scholar

Isla Pera, P., Moncho Vasallo, J., Guasch Andreu, O., and Torras Rabasa, A. (2008). Proceso de adaptación a la diabetes mellitus tipo 1 (DM1). Concordancia con las etapas del proceso de duelo descrito por Kübler-Ross. Endocrinol. Nutr. 55, 78–83. doi: 10.1016/S1575-0922(08)70640-5

CrossRef Full Text | Google Scholar

Jaser, S. S., Faulkner, M. S., Whittemore, R., Jeon, S., Murphy, K., Delamater, A., et al. (2012). Coping, self-management, and adaptation in adolescents with type 1 diabetes. Ann. Behav. Med. 43, 311–319. doi: 10.1007/s12160-012-9343-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Kelley, K. (2007). Methods for the behavioral, educational, and social sciences: an R package. Behav. Res. Methods 39, 979–984. doi: 10.3758/BF03192993

PubMed Abstract | CrossRef Full Text | Google Scholar

Koo, T. K., and Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J. Chiropr. Med. 15, 155–163. doi: 10.1016/j.jcm.2016.02.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Kovacs, K., Nicolucci, A., Holt, R. I. G., Willaing, I., Hermanns, N., et al. (2013). Diabetes attitudes, wishes and needs second study (DAWN2^TM): cross-national benchmarking indicators for family members living with people with diabetes. Diabet. Med. 30, 778–788. doi: 10.1111/dme.12239

PubMed Abstract | CrossRef Full Text | Google Scholar

Kübler-Ross, E. (1993). Sobre la Muerte y los Moribundos. Barcelona: Grijalbo.

Google Scholar

Linkowski, D. C. (1971). A scale to measure acceptance of disability. Rehabil. Couns. Bull. 14, 236–244.

Google Scholar

Lloyd, C. E., Pambianco, G., and Orchard, T. J. (2010). Does diabetes-related distress explain the presence of depressive symptoms and/or poor self-care in individuals with type 1 diabetes? Diabetes Med. 27, 234–237. doi: 10.1111/j.1464-5491.2009.02896.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Malik, J. A., and Koot, H. M. (2009). Explaining the adjustment of adolescents with type 1 diabetes. Diabetes Care 32, 774–779. doi: 10.2337/dc08-1306

PubMed Abstract | CrossRef Full Text | Google Scholar

Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika 57, 519–530. doi: 10.2307/2334770

CrossRef Full Text | Google Scholar

Martino, G., Langher, V., Cazzato, V., and Vicario, C. M. (2019). Psychological factors as determinants of medical conditions. Front. Psychol. 10:2502. doi: 10.3389/fpsyg.2019.02502

PubMed Abstract | CrossRef Full Text | Google Scholar

McDonald, R. P. (1999). Test Theory. A Unified Treatment. London: LEA.

Google Scholar

Millán, M. M., Reviego, J., and del Campo, J. (2002). Reevaluación de la versión española del cuestionario diabetes Quality of Life (EsDQOL). Endocrinol. Nutr. 49, 322–324. doi: 10.1016/S1575-0922(02)74482-3

CrossRef Full Text | Google Scholar

Moore, D. S., McCabe, G. P., and Craig, B. A. (2009). Introduction to the Practice of Statistics. New York: W. H. Freeman.

Google Scholar

Ortiz, M. (2006). Estrés, estilo de afrontamiento y adherencia al tratamiento de adolescentes diabéticos tipo 1. Ter. Psicol. 24, 139–147.

Google Scholar

Peyrot, M., McMurray, J., and Kruger, D. (1999). A biopsychosocial model of glycemic control in diabetes: stress, coping and regimen adherence. Health Soc. Behav. 40, 141–158. doi: 10.2307/2676370

PubMed Abstract | CrossRef Full Text | Google Scholar

Peyrot, M., Rubin, R. R., Lauritzen, T. F. J., Snoek, F. J., Matthews, D. R., and Skovlund, S. E. (2005). Psychosocial problems and barriers to improved diabetes management: results of the cross-national diabetes attitudes, wishes and needs (DAWN) study. Diabetes Med. 22, 1379–1385. doi: 10.1111/j.1464-5491.2005.01644.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Polonsky, W. H., Fisher, L., Earles, J., Dudl, R. J., Lees, J., Mullan, J., et al. (2005). Assessing psychosocial distress in diabetes: development of the diabetes distress scale. Diabetes Care 28, 626–631. doi: 10.2337/diacare.28.3.626

PubMed Abstract | CrossRef Full Text | Google Scholar

Polonsky, W. H., Fisher, L., Snoek, F. J., Weinger, K., Jelsovsky, Z., Parkin, C. G., et al. (2009). Evaluation of the confidence in diabetes scale (CIDS-2) for patients with poorly controlled T2DM. Diabetes 58, A479–A479.

Google Scholar

Revelle, W. (2015). PSYCH: Procedures for Personality and Psychological Research. Northwestern University, Evanston. Available online at: https://personality-project.org/r/psych (accessed July 27, 2016).

Google Scholar

Revelle, W., and Rocklin, T. (1979). Very simple structure – alternative procedure for estimating the optimal number of interpretable factors. Multivariate Behav. Res. 14, 403–414. doi: 10.1207/s15327906mbr1404_2

PubMed Abstract | CrossRef Full Text | Google Scholar

Robertson, S. M., Stanley, M. A., Cully, J. A., and Naik, A. D. (2012). Positive emotional health and diabetes care: concepts, measurement, and clinical implications. Psychosomatics 53, 1–12. doi: 10.1016/j.psym.2011.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosseel, Y. (2012). lavaan: an R package for structural equation modeling. J. Stat. Softw. 48, 1–36. doi: 10.3389/fpsyg.2014.01521

PubMed Abstract | CrossRef Full Text | Google Scholar

Rubin, R. R. (2005). “Counselling and psychotherapy in diabetes mellitus,” in Psychology in Diabetes Care, eds F. J. Snoek and T. C. Skinner (Chichester: John Wiley & Sons, Ltd.), 171–194.

Google Scholar

Sacco, W. P., and Yanover, T. (2006). Diabetes and depression: the role of social support and medical symptoms. J. Behav. Med. 29, 523–531. doi: 10.1007/s10865-006-9072-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Sanz, J., García-Vera, M. P., Espinosa, R., Fortún, M., and Vázquez, C. (2005). Adaptación española del inventario para la depresión de beck II (BDI-II): propiedades psicométricas en pacientes con trastornos psicológicos. Clín. Salud 16, 121–142.

Google Scholar

Schermelleh-Engel, K., Moosbrugger, H., and Müller, H. (2003). Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods Psychol. Res. 8, 23–74.

Google Scholar

Schmitt, A., Reimer, A., Kulzer, B., Icks, A., Paust, R., Roelver, K.-M., et al. (2018). Measurement of psychological adjustment to diabetes with the diabetes acceptance scale. J. Diabetes Complications 32, 384–392. doi: 10.1016/j.jdiacomp.2018.01.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Seisdedos, N. (1988). Cuestionario de Ansiedad Estado-Rasgo. Madrid: TEA.

Google Scholar

Spielberger, C. D., Gorsuch, R. L., and Lushene, R. E. (1982). Manual for the State-Trait Anxiety Inventory (Self Evaluation Questionnaire). Palo Alto, CA: Consulting Psychologists Press, Inc.

Google Scholar

Sullivan, B. J. (1979). Adjustment in diabetic adolescen girls: I. development of the diabetic adjustment scale. Psychosom. Med. 41, 119–126. doi: 10.1097/00006842-197903000-00005

PubMed Abstract | CrossRef Full Text | Google Scholar

The Diabetes Control and Complications Trial Research Group [DCCT] (1988). Reliability and validity of a diabetes quality-of-life measure for the diabetes control and complications trial (DCCT). Diabetes Care 11, 725–732. doi: 10.2337/diacare.11.9.725

PubMed Abstract | CrossRef Full Text | Google Scholar

Trief, P., Grant, W., Elbert, K., and Weinstock, R. (1998). Family environment, glycemic control, and the psychosocial adaptation of adults with diabetes. Diabetes Care 21, 241–245. doi: 10.2337/diacare.21.2.241

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Der Ven, N. C., Weinger, K., Yi, J., Pouwer, F., Adèr, H., Van Der Ploeg, H. M., et al. (2003). The confidence in diabetes self-care scale: psychometric properties of a new measure of diabetes-specific self-efficacy in Dutch and US patients with type 1 diabetes. Diabetes Care 26, 713–718. doi: 10.2337/diacare.26.3.713

PubMed Abstract | CrossRef Full Text | Google Scholar

Von Korff, M., Katon, W., Lin, E. H., Simon, G., Ciechanowski, P., Ludman, E., et al. (2005). Work disability among individuals with diabetes. Diabetes Care 28:1330. doi: 10.2337/diacare.28.6.1326

PubMed Abstract | CrossRef Full Text | Google Scholar

Wysocki, T., Hough, B. S., Ward, K. M., and Green, L. B. (1992). Diabetes mellitus in the transition to adulthood: adjustment, self-care and health status. J. Dev. Behav. Pediatr. 8, 194–201.

PubMed Abstract | Google Scholar

Young-Hyman, D., de Groot, M., Hill-Briggs, F., Gonzalez, J. S., Hood, K., and Peyrot, M. (2016). Psychosocial care for people with diabetes: a position statement of the American Diabetes Association. Diabetes Care 39, 2126–2140. doi: 10.2337/dc16-2053

PubMed Abstract | CrossRef Full Text | Google Scholar

Yu, C., and Muthén, B. (2002). “Evaluation of model fit indices for latent variable models with categorical and continuous outcomes,” in Paper Presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.

Google Scholar

Zung, W. W. (1965). A self-rating depression scale. Arch. Gen. Psychiatry 12, 63–70. doi: 10.1001/archpsyc.1965.01720310065008

PubMed Abstract | CrossRef Full Text | Google Scholar

Zweig, M. H., and Campbell, G. (1993). Receiver-operating characteristics (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin. Chem. 39, 561–577. doi: 10.1093/clinchem/39.4.561

CrossRef Full Text | Google Scholar