Adriana Mira1,2*
Carla Soler1
Marta Alda2,3,4
Rosa Baños5,6
Diana Castilla1,2,5Adoración Castro7,8Javier García-Campayo3,4,8
Azucena García-Palacios1,5
Margalida Gili7,8Mariena Hurtado9
Fermín Mayoral8,9
Jesús Montero-Marín2,3,8
Crisitina Botella1,5- 1Department of Basic and Clinical Psychology, and Psychobiology, Universitat Jaume I, Castellón, Spain
- 2Department of Psychology and Sociology, Universidad de Zaragoza, Zaragoza, Spain
- 3Aragon Institute for Health Research (IIS Aragon), Zaragoza, Spain
- 4University Hospital Miguel Servet, Zaragoza, Spain
- 5CIBER de Fisiopatología de la Obesidad y Nutrición (CIBEROBN), Madrid, Spain
- 6Department of Personality, Evaluation and Psychological Treatment, Universitat de València, Valencia, Spain
- 7Department of Psychology, Universitat de les Illes Balears, Palma de Mallorca, Spain
- 8Primary Care Prevention and Health Promotion Research Network, RedIAPP, ISCIII, Madrid, Spain
- 9Mental Health Clinical Management Unit, Institute of Biomedical Research of Malaga (IBIMA), Regional University Hospital Carlos Haya, University of Malaga, Málaga, Spain
Background: Depression is one of the most prevalent psychological disorders worldwide. Although psychotherapy for depression is effective, there are barriers to its implementation in primary care in Spain. The use of the Internet has been shown to be a feasible solution. However, the acceptability of Internet-based interventions has not been studied sufficiently.
Objective: To assess the acceptability of an Internet-based intervention (IBI) for depression in primary care, and explore the relationship between expectations and satisfaction and the improvement in the clinical variables in primary care patients receiving this intervention. Furthermore, it offers data about the effects of some sociodemographic characteristics on these acceptability variables and analyzes whether the expectations are related to finalizing the intervention.
Methods: Data were based on depressive patients who were participants in a randomized controlled trial. In the present study, we present the data from all the participants in the Internet intervention groups (N = 198). All the participants filled out the expectation and satisfaction scales (six-item scales regarding treatment logic, satisfaction, recommending, usefulness for other disorders, usefulness for the patient, and unpleasantness), the Beck Depression Inventory-II, and the secondary outcome measures: depression and anxiety impairment, and positive and negative affect.
Results: Results showed that participants’ expectations and satisfaction with the program were both high and differences in expectations and satisfaction depended on some sociodemographic variables (age: older people have higher expectations; sex: women have greater satisfaction). A positive relationship between these variables and intervention efficacy was found: expectations related to “usefulness for the patient” were a statistically related predictor to the results on the BDI-II (Beta = 0.364), and the perception of how logical the treatment is (Beta = 0.528) was associated with change in the clinical variable. Furthermore, the higher the expectations, the higher the improvements exhibited by the patients in all measures evaluated during the ten intervention modules. High expectations were also directly related to finalizing the intervention.
Conclusions: This is the first study in Spain to address this issue in the field of IBIs for depression in primary care. The IBI showed high acceptance related to the intervention’s efficacy and completion. Research on IBI acceptability could help to implement the treatment offered.
Clinical Trial Registration: www.ClinicalTrials.gov, identifier NCT01611818.
Background
Depression is one of the most disabling and prevalent psychological disorders worldwide (1, 2). It has high personal, social, and economic costs (3, 4), and it is among the most common reasons for consulting a general practitioner (GP) (5, 6). In Spain, depression is the most pervasive mental disorder, with a 12-month prevalence estimated at 3.9% and a lifetime prevalence estimated at 10.5% (2). Furthermore, according to WHO data, in Spain, there is a median of 1 clinical psychologist per 100,000 inhabitants, which is much lower than the European median of 3 psychologists (7). In primary care, it is common to prescribe medication for depressive patients (8), but many patients would also like to receive psychological treatment (9). In addition, psychotherapy for depression has been found to be effective in primary care contexts (5, 10, 11). However, there are barriers to the implementation of psychotherapy in primary care, such as the face-to-face time required, the cost, and the lack of trained professionals, which makes it difficult to reach everyone in need and provide the suitable treatment (12–15). Furthermore, some patients reject the use of traditional mental health services (15) because of stigmatization processes (16). According to previous studies (13, 14, 17), it is necessary to explore the usefulness of other alternatives for delivering psychotherapy.
The use of the Internet has been shown to be a feasible solution for the accessibility problem (18, 19). A growing body of research supports the efficacy of Internet-based interventions (IBIs) for the treatment of depression (17, 20–24). IBIs are self-applied, interactive, multimedia (images, video, etc.) interventions, and they are based on the most effective classic cognitive behavioral therapy (CBT) (25). It is true that in northern Europe (i.e., Holland, Sweden), research related to IBIs is more advanced than in the South, e.g., in Spain, where the use of information and communication technologies (ICTs) by psychologists is lower (22, 26, 27). A recent study in Spain shows that only 26% of the sample of psychologists analyzed used ICTs, and only 2.31% of the sample used web-based interventions (28). The main drawbacks emphasized in this study are potential difficulties related to the therapeutic alliance and the limitations of nonverbal communication, followed by confidentiality issues and technical problems in handling the data (28). To date, only one randomized controlled trial (RCT) in Spain has demonstrated the efficacy of an IBI for depression in primary care (29). This study opens the door for its implementation in natural contexts such as primary care, thus helping to reach anyone in need. As is pointed out above, in Spain, the median of clinical psychologists per inhabitant is much lower than the European median (7).
This sort of Internet-based program for depressed patients could be an appropriate solution in mental healthcare, specifically in primary care (30–33). In fact, this line of research is growing at an impressive rate (34), but little is known about the acceptability of the IBIs in this area (35, 36). A recent study in a primary care setting showed that patients’ acceptance of Internet-based treatments for depression in primary care was low, but it could be increased significantly using a brief intervention to facilitate acceptance using an informational video (35). However, it is important to remark that this study investigated behavioral intention to use an IBI, not acceptability in the sense of satisfaction (35).
Although clinical effectiveness is important, an additional criterion that is likely to affect implementation is the acceptability (37). Acceptability refers to the degree to which patients are satisfied with a service, think it is easy to use, and are willing to use it (38, 39). A treatment is acceptable when it is perceived as appropriate, fair, reasonable, and nonintrusive in addressing a problem (29). Taking an intervention’s acceptability into account, it is possible to improve the adherence and outcomes (40, 41). Some variables related to treatment acceptability are expectations and satisfaction (42–45). The literature shows that positive expectations have been associated with better outcomes (46, 47). Moreover, “satisfaction” provides information about the feasibility of the intervention, helping to optimize its effectiveness (45). There are two models that have been established about the predictions of eHealth acceptance: the technology acceptance model (TAM) model and theunified theory of acceptance and use of technology (UTAUT) model (48, 49). However, research points out the complex nature of acceptance and its determinants (50, 51).
In any case, the importance of treatment acceptability is known, and few studies have focused on its assessment in terms of IBIs and its relationship with effectiveness (29, 35, 42, 43, 45, 52). If the objective is to develop and deploy these kinds of programs in a specific context, we have to be sensitive to different participant attitudes that can increase acceptance (28). Only one study in Spain explores patients’ expectations about an IBI for depression in primary care (32) through a qualitative research design. The results showed good general acceptance of the IBI, detecting possible advantages and suggested improvements, such as the need for individualized and personalized interaction (32). To the best of our knowledge, no study in Spain has examined participants’ expectations and satisfaction with an IBI delivered in primary care, and how these variables are related to sociodemographic variables, clinical outcomes, and dropout rates.
The objectives of the present study are: a) to analyze participants’ expectations and satisfaction with an IBI for depression in primary care in Spain and offer data about the effects of some sociodemographic characteristics on these variables; b) to analyze whether the expectations are related to finalizing the intervention program; c) to analyze the relationship between expectations and satisfaction and the primary outcome measure (depression) in patients receiving this intervention; and d) to study the relationship of these variables (expectations and satisfaction) with other relevant clinical variables assessed through the intervention program (depression and anxiety, impairment and positive and negative affect). We believe this work is important because Spain is a country with a low presence of IBIs, and the study was conducted in the National Health System in primary care, where many patients do not have a high level of expertise in the use of ICTs.
Methods
Design and Setting
The current study is a secondary analysis of a multicenter, three-arm, parallel, RCT conducted with depressive patients recruited in primary care (registered under ClinicalTrials.gov NCT01611818). Patients were randomized to receive an IBI program (Smiling is Fun) for depression with low-intensity therapist-guided (LITG), completely self-guided (CSG), or an improved treatment based on the usual care from their GP improved treatment as usual (iTAU). It is important to mention that participants in LITG did not receive human support by phone or in person. They were offered the opportunity to request support by email (29). Patients could ask the psychotherapists questions or advice via email messages, with a maximum of three contacts during the treatment period. Four trained psychotherapists randomly contacted the patients by email to offer help with any difficulties or problems encountered when using the program. Patients could also ask a technician for help to resolve problems of a technical nature. In CSG, there was no contact with any therapist, and only technical questions about the computer program could be posed. Only 17 email contacts were made with 13 participants in the condition with support, which represented only 11.9% of the patients in this group (29). Patients did not make much use of the support offered. Thus, the low number of email contacts requesting assistance from the psychotherapists and, therefore, the results for the supported and unsupported groups were similar in adherence and efficacy. There were no significant differences between the two groups at any time (29). Because no differences were found in the two intervention conditions in the RCT, the analyses in the present work were performed with the entire sample (LITG and CSG). In this study, we present the data on expectations and satisfaction of all the participants in the IBI groups. The research protocol and data collection procedure have been described elsewhere (29, 53).
Participants
In the original study, a total of 296 depressive patients were recruited in primary care settings between November 2012 and January 2014 in the Spanish regions of Aragón, Andalucía, Baleares, and Valencia. Recruited participants had mild or moderately severe symptoms according to the Spanish Beck Depression Inventory-II (BDI-II) (54) (14–19: mild depression; 20–28: moderate depression). They were 18–65 years old, had the ability to use a computer, had Internet at home and an email account, and were able to read and understand Spanish. The Mini International Neuropsychiatric Interview (MINI) 5.0 (55) was used to assess different psychological disorders and establish the diagnosis. Patients were excluded from the study if they had severe depression (score ≥29 on the BDI-II), had a severe Axis I psychiatric disorder (e.g., psychotic disorders, presence of suicidal ideation or plan, alcohol/substance abuse or dependence, dementia), or were currently receiving or have received psychological treatment in the previous year. For the present study, the 198 participants who completed the Internet-based program were selected.
Intervention and Protocol Modules
The protocol used for the treatment of depression is a completely self-help (unguided) Internet-based program called “Smiling is Fun,” developed within the framework of the European project OPTIMI: Online Predictive Tools for Intervention in Mental Illness (grant agreement number 248544). This program is based on CBT, including behavioral activation (56, 57), and it combines strategies to promote emotion regulation, coping capacity, and resilience. At the beginning of the program, there are two initial modules (“Home” and “Welcome”). The “Home” module explains the goals of the intervention, who can benefit from it (with case examples), the terms and conditions, and who we are; and the “Welcome” module informs the participant about the contents and benefits of each module. Regarding the therapeutic content, Smiling is Fun is composed of 10 intervention modules with different psychological techniques, as Table 1 shows. Smiling is Fun has three complementary transversal tools that can be found in the main menu of the intervention program. The participants can use them every day to receive helpful feedback that is important for their reinforcement and motivation: 1) The “activity report” is where participants first have to specify their mood state, coping ability, and stress on a scale from 0 to 10. Then, they have to rate the degree of satisfaction with each activity in the past 24 h, and how much these activities are related to their own goals and values in life. Moreover, participants have to indicate what percentage of the day they have been active or involved in their life. 2) The “calendar” provides participants with information about homework and tasks already completed. Moreover, it indicates the days that participants have completed the activity report. The participants filled out the activity report every time they accessed the program. 3) “How am I?” offers a set of graphs and feedback on the participants’ progress, including activity level, emotional distress (anxiety and sadness), positive emotionality (active, energetic, enthusiastic, etc)., and negative emotionality (angry, stressed, tense, etc). Patients should work on each module in a sequential way for at least a week. For more information about this Internet-based program, see Refs. (29, 43, 53, 58).
Table 1 Intervention modules and main objectives.
Measures
The study measures and area and time of assessment are summarized in Table 2. Because we had already published a paper with all the efficacy data on the Internet-based treatment program (29), in the present work, we focused only on the primary outcome measure (BDI-II) and on the measurements obtained throughout the entire treatment, that is, measures that were assessed after each intervention module. Furthermore, we decided to report the sociodemographic measures that are commonly included in clinical trials: gender, age, marital status, and level of education.
Table 2 Study measures, assessment area, and time of assessment.
Sociodemographic Variables
The following sociodemographic variables were collected: gender, age, relationship status, and level of education.
Diagnostic Interview
Mini International Neuropsychiatric Interview 5.0 (MINI)
It is a structured diagnostic psychiatric interview based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) and the International Classification of Diseases. The MINI can be administered in a short period of time by (55) a clinician with brief training. In this study, the Spanish validated version was used (59).
Primary Outcome Measure (Pre- and Postassessment, Over the Internet)
Beck Depression Inventory-II, Spanish Validated Version
The Beck Depression Inventory-II (BDI-II), Spanish validated version (60), assesses the presence and severity of depressive symptoms. It is a self-report measure with 21 items rated on a 4-point scale ranging from 0 (not present) to 3 (severe), and total scores can range from 0 to 63. It contains emotional, cognitive, and somatic symptoms of depression, and it is based on the diagnosis of major depressive disorders described in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV). The Spanish version of the BDI-II (60), as a continuous variable, was used as the primary outcome measure. The BDI-II is one of the most widely used instruments to measure depression severity, and the published studies show strong agreement between the BDI-II and the clinical diagnosis of depression, as well as good psychometric properties for the scale (60, 61). In the current study, Cronbach’s alpha reliability coefficient was very large (α = .91).
Secondary Outcome Measures (Postmodule Assessment, Over the Internet)
Overall Depression Severity and Impairment Scale (ODSIS)
The Overall Depression Severity and Impairment Scale (ODSIS) is a self-report measure with five items that evaluate experiences related to depression (62). It measures the frequency and severity of depression, as well as the level of avoidance, work/school/home interference, and social interference associated with depression. It was found to have good convergent and discriminant validity and excellent internal consistency (α = 0.94 in the outpatient sample, 0.91 in the student sample, and 0.92 in the community sample) (62). The ODSIS was translated into Spanish, and a validation process was carried out (63). The validation data confirmed the factorial structure and reliability and validity data obtained by the original authors (62). In the current study, Cronbach’s alpha reliability coefficient was very large (α = .92).
Overall Anxiety Severity and Impairment Scale (OASIS)
The Overall Anxiety Severity and Impairment Scale (OASIS) consists of a five-item questionnaire, rated from 0 to 4, that assesses the frequency and severity of anxiety symptoms (64). The instrument also provides measures of avoidance, as well as work, academic, social, and everyday-life impairment related to anxiety symptoms. A psychometric analysis of the OASIS scale found good internal consistency (α = .80), test–retest reliability (k = .82), and convergent validity for this instrument. The Spanish version of the OASIS showed good internal consistency (α = .86), convergent and discriminant validity, as well as sensitivity to change (65). In the current study, the alpha coefficient was very satisfactory (α = .85).
Positive and Negative Affect Scale
The Positive and Negative Affect Scale (PANAS) consists of 20 items that evaluate two independent dimensions: positive affect (PA) and negative affect (NA). The range for each scale (10 items on each) is from 10 to 50 (66). It is a brief, reliable, and valid self-report measure, and it has shown excellent convergent and divergent validity (54). The Spanish version has demonstrated high internal consistency (α = 0.89 and 0.91 for PA and NA in women, respectively, and α = 0.87 and 0.89 for PA and NA in men, respectively) in college students (67). In the current study, alpha coefficients were very satisfactory for both scales (αs = .92 and .89 for positive and negative subscales, respectively).
Treatment Acceptance Measures
Expectation and Satisfaction Scales (Expectations at the End of Intervention Module 2 and Satisfaction at Postassessment, Over the Internet)
These scales are based on the Borkovec and Nau questionnaires (68). Each scale has six items, with responses ranging from 0 (“not at all”) to 10 (“very much”). The maximum of the scores in both scales is 60. More punctuation means more expectation and satisfaction. The questions assess how logical the intervention seemed, to what extent it satisfied the patient, whether the intervention would be recommended to other patients, whether it would be useful to treat other problems, its usefulness for the patient’s specific problem, and to what extent it could be unpleasant. Item 6, “unpleasantness,” is answered in reverse. Patients completed the expectation scale at the end of the second module, once the intervention rationale had been explained. The satisfaction scale was completed at the end of the program. These scales have been used in several studies (42–44, 69, 70). In the current study, Cronbach’s alpha reliability coefficients were very satisfactory for both scales (αs = .88 and .87 for expectation and satisfaction scales, respectively).
Procedure
Between November 2012 and January 2014, patients were recruited in primary care settings. General practitioners from the Spanish regions of Andalusia, Aragon, Valencia, and the Balearic Islands detected possible participants using a brief questionnaire. After a few days, an independent researcher used the MINI and other questionnaires to assess the participants, taking into account inclusion and exclusion criteria. Later, participants signed written informed consent to be part of the study. Then, a randomization was carried out by another independent researcher. The ethics committee of the regional health authority accepted the study on April 7, 2010 (ref: PI10/01083). All participants completed the pretreatment evaluation integrated in the web system. When they finished each of the10 treatment modules, they performed the postmodule evaluation, also through the web system. At the end of treatment, they also completed the posttreatment evaluation through the web site. More details about the design, procedure, therapists, and recruitment methods are included in the main outcome study (29).
Statistical Analysis
Means (M) and standard deviations (SD) for the expectation and satisfaction measures were analyzed. Pearson correlation coefficients, t-tests, and one-way ANOVAs were conducted to find out the effect of some sociodemographic variables on the expectation and satisfaction measures. T-tests were performed comparing expectations between dropouts and completers. For two-group comparisons, effect sizes were estimated by means of the standardized mean difference (d) and interpreted as low, moderate, and large magnitude for d values of 0.2, 0.5, and 0.8, respectively (71). For ANOVAs, the effect size was approached with the proportion of variance accounted for Eta2. Moreover, multiple regression analyses (by means of stepwise and hierarchical models) were applied to examine whether expectations and satisfaction predicted the improvement in the primary outcome measure. The improvement was defined as the difference between the pretest and posttest scores in the BDI-II. Finally, a canonical correlation analysis was performed with the six expectation items as the predictor variables and the 10 postmodule evaluations of the secondary outcome measures as the dependent variables. Variables with structure coefficients (rs) larger than .45 (in absolute value) were considered relevant for interpreting the results of these analyses (72). All the statistical analyses were conducted with IBM startical product and service solutions (SPSS) Statistics 20 (IBM Corporation, Armonk, NY, USA).
Results
Sample Description
Final analyses were performed on 198 subjects. The sample was mainly composed of women (76.4%). The age range was between 25 and 69 years, with an M of 48.33 (SD: 9.99). In the case of the educational level, 12.12% had primary studies, 68.8% had secondary studies, and 18.18% had university studies. Regarding marital status, 12.3% were single, 69.2% were married or had a partner, and 16.9% were separated or divorced. Regarding depression severity at pretreatment, the average on the BDI-II was 23.50 with a standard deviation of 8.40. One hundred eight participants were dropouts (54.54% of the sample).
Participants’ Expectations and Satisfaction
Table 3 presents the mean (M) and standard deviation (SD) of each item on the expectation and satisfaction scales, as well as their total scores. Note that item 6 (“unpleasantness”) is answered in reverse (the higher the score, the lower the expectation or satisfaction), and so it has been recoded to follow the same scoring criteria as the other five items on the scale. Therefore, taking into account that the scale ranges from 0 to 10 points, the average levels of expectations seem high, as all of them are above or around 7. Moreover, the average levels of satisfaction seem high, as all of them are above 7.5 points. In addition, a positive, statistically significant relationship was found between expectations and satisfaction (r = .68, p < .001).
Table 3 Means and standard deviations for expectation and satisfaction scales.
Sociodemographic Characteristics of the Participants Related to the Expectations and Satisfaction With the Program
Regarding the relationship between the patients’ ages and their expectations and satisfaction with the treatment, results showed that the only statistically significant correlations were found between age and the items “treatment logic” (Pearson’s r = .217, p < .01), “treatment satisfaction” (Pearson’s r = .248, p < .001), and “recommending to others” (Pearson’s r = .209, p < .01) from the expectation scale. However, there were no statistically significant correlations between age and satisfaction.
Regarding the relationship between the patients’ sex and their expectations, the analysis revealed that there were no statistically significant differences between men and women on any items, or on the total score for expectations. In the case of satisfaction, there was greater satisfaction in women than in men on the items “usefulness for other disorders” (t(86) = 2.59, p = .011; d = 0.65) and “usefulness for the patient” (t(86) = 2.60, p = .011; d = 0.66), and on the total score (t(86) = 2.29, p = .024; d = 0.58). There was also a greater marginally significant satisfaction in women on the items “treatment logic” (t(86) = 1.88, p = .063; d = 0.47) and “recommending to others” (t(86) = 1.94, p = .055; d = 0.49).
The effect sizes (ds) were of medium-high magnitude. Regarding marital status, one-way ANOVAs were applied, with marital status as an independent variable and expectations and satisfaction as dependent variables, in order to verify their possible statistical relationship. There were no statistically significant relationships between civil status and expectation or satisfaction scales. The same thing occurred in the case of the educational level.
Are the Patients’ Expectations Related to Finalizing the Intervention Program?
This section examines whether the expectations of the patients who finish and do not finish the treatment are similar or if, on the contrary, differences between the two groups can be observed (dropouts and completers). Table 4 presents the results when comparing the M expectations of both groups. Thus, the results showed that, with the exception of item 6 (“unpleasantness”), statistically significant differences were observed in favor of completers on all the items and on the total score. The effect sizes (ds) indicated that these differences were of moderate practical relevance. There were no statistically significant differences between dropouts and completers on the BDI-II at preintervention (t (196) = −.47, p = .640).
Table 4 T-tests comparing expectations of dropouts and completers.
Regarding adherence, 90 participants completed all the modules (45.45% of the sample), and 108 were dropouts (54.54% of the sample).
Relation Between Expectations and Satisfaction and the Primary Outcome Measure: Beck Depression Inventory-II
For this purpose, we restricted the sample to the 90 patients in the database with BDI-II data on the pretest and posttest. A first analysis consisted in calculating the Pearson correlation coefficients between expectations and the BDI-II at the pretest, in order to investigate potential relations between expectations and BDI-II severity in the baseline. All correlation coefficients between each expectancy item (and its total score) and the BDI-II at the pretest were statistically nonsignificant, with correlations ranging between r = −.09 (p = .37) for “usefulness for other disorders” and r = −.18 (p = .09) for the expectation total score.
A stepwise linear regression was conducted taking the expectation items (and their total score) as predictors and the BDI-II pretest–posttest improvement as the dependent variable. In the set of potential predictors, we also included the BDI-II at the pretest in order to control its potential confounding effect on the expectations–depression improvement relationship. The final model included two predictors: the expectation item 5, “usefulness for the patient” (Beta = 0.364, t(87) = 3.77, p < .001), and the BDI-II at the pretest (Beta = 0.325, t(87) = 3.37, p = .001), with 18.7% of total variance explained (F(2, 87) = 11.25, p < .001). The remaining potential predictors did not reach statistical significance. Next, a hierarchical regression model by successively entering in the model the BDI-II at the pretest and item 5 revealed that the inclusion of the former explained 7.6% of the variance (F(1, 88) = 7.19, p = .009), and the inclusion of the latter in the model led to an increase of 13% in the percentage of variance explained (ΔF(1, 87) = 14.23, p < .001). The direction of the relationship between item 5 and the result on the BDI-II was positive: the higher the score on the expectations reflected in item 5, the greater the pretest–posttest change in the BDI-II (Beta = 0.364). It should be noted, however, that the proportion of variance explained by these two predictors was low (R2adj = .187), which limits the practical use of this model to predict treatment outcome.
In the case of the relation between satisfaction and treatment efficacy, item 1, “treatment logic” (Beta = 0.528, p < .001), and the BDI-II at the pretest (Beta = 0.339, t(87) = 3.90, p < .001) were selected to enter in the model, accounting for 33.5% of the variance (F(2, 87) = 23.47, p < .001). Next, a hierarchical regression by successively entering the BDI-II at the pretest and item 1 revealed an increase of 11.3% in the percentage of variance accounted for by the last variable (ΔF(1, 87) = 36.83, p < .001). The direction of the relationship between item 1 and the result on the BD-II was positive: the higher the satisfaction score reflected in item 1, the greater the pretest–posttest change in the BDI-II (regression coefficient, b1 = 2.55) (Beta = 0.528).
Relation Between Expectations and Evolution in the Clinical Variables Assessed Through the Intervention Program (Postmodule Assessment)
Depression Impairment
The canonical correlation analysis with the six expectation items as the predictor variables and the 10 depression measures as the dependent variables was statistically significant, χ2(60) = 85.656, p = .017, with 63.7% of the variance shared between the two variable sets. Only the first function was considered noteworthy in the context of this study (37.3% of shared variance). Table 1 in the Supplementary Material presents the results. With the exception of occasion 1, all of the depression measures significantly contributed to the canonical variable. In particular, depression on occasions 7, 10, 9, and 8 showed a larger shared variance with the canonical variable (73.8%, 68.4%, 58.8%, and 56.8%, respectively). Regarding the predictor variable set, with the exception of expectation 3 (“recommending to others”), all of them exhibited relevant structure coefficients. In particular, expectation 4 (“usefulness for other disorders”) exhibited the largest shared variance with the canonical variable (47.2%). A negative relationship between the canonical variable of depression and the expectation variable was found, indicating that the greater the expectations, the lower the depression exhibited by the patients on the 10 measurement occasions.
Anxiety Impairment
The canonical correlation analysis with the six expectations as the predictor variables and the 10 anxiety measures as the dependent variables reached statistical significance, χ2(60) = 90.055, p = .007, with 65.6% of the variance shared between the two variable sets. Only the first function was considered noteworthy in the context of this study (34.3% of shared variance). Table 2 in the Supplementary Material presents the main results of this canonical correlation analysis. The structure coefficients show that, with the exception of occasions 1 and 3, all of the anxiety measures significantly contributed to the canonical variable, with occasions 8, 7, and 10 exhibiting a larger shared variance with the canonical variable (51.8%, 50.1%, and 46.4%, respectively). Regarding the predictor variable set, with the exception of expectation 5 (“usefulness for the patient”), all of them exhibited relevant structure coefficients. In particular, expectations 4 (“usefulness for other disorders”) and 1 (“treatment logic”) exhibited the largest shared variances with the canonical variable (78.7% and 54.2%, respectively). Finally, a negative relationship was found between the canonical variable of anxiety and the expectation variable, indicating that the greater the expectations, the lower the anxiety exhibited by the patients on the 10 measurement occasions.
Negative Affect
The canonical correlation analysis with the six expectations as the predictor variables and the 10 measures of NA as the dependent variables did not reach statistical significance, χ2(60) = 72.960, p = .122, with 57% shared variance between the two variable sets. Although a statistically insignificant relationship was found between expectations and negative affect, the first canonical function reached a relevant percentage of shared variance between the two canonical variables (28%). Therefore, the first canonical function was considered noteworthy. Table 3 in the Supplementary Material presents the results. Only 3 out of the 10 measures of NA significantly contributed to the canonical variable: those of occasions 9, 10, and 5, with 27.7%, 21.2%, and 20.2% of shared variance with the canonical variable, respectively. Regarding the predictor variable set, only expectation 5 (“usefulness for the patient”) exhibited a relevant structure coefficient (36.4% shared variance with the canonical variable). An inverse relationship between expectations and NA was found, indicating that the greater the expectations, the lower the NA exhibited by the patients during the treatment.
Positive Affect
The canonical correlation analysis with the six expectations as the predictor variables and the 10 measures of PA as the dependent variables was also statistically significant, χ2(60) = 83.687, p = .023, with 62% of the variance shared between the two variable sets. Only the first function was considered noteworthy in the context of this study (32.9% of shared variance). Table 4 in the Supplementary Material presents all the measures of PA that significantly contributed to the canonical variable. In particular, PA on occasions 8, 10, and 9 exhibited a larger shared variance with the canonical variable (81%, 79.9%, and 62.4%, respectively). Regarding the predictor variable set, all of them exhibited relevant structure coefficients. In particular, expectations 1 (“treatment logic”) and 4 (“usefulness for other disorders”) showed the largest shared variances with the canonical variable (66.6% and 57.9%, respectively). A positive relationship was found between the canonical variable of PA and the expectation variable, indicating that the greater the expectations, the greater the PA exhibited by the patients throughout the 10 measurement occasions.
Discussion
The first objective of this work was to analyze participants’ expectations and satisfaction with an IBI for depression in primary care in Spain. Furthermore to offer data about the relationships among some sociodemographic characteristics and these variables; and moreover, to analyze whether the expectations were related to finalizing the intervention program. Furthermore, this study offers data on the associations between expectations and satisfaction and the primary outcome measure (depression) in patients receiving this intervention, as well as the relationship between the expectations and the other clinical variables assessed through the intervention program (depression and anxiety impairment and positive and negative affect).
The participants in this study reported positive expectations about the program, coinciding with other studies on Internet-based treatments (22, 43, 73, 74) and also in a primary care setting. The expectations were high, suggesting that participants found the treatment very logical and useful, even for other psychological problems, and they felt confident about recommending it to a friend. Participants’ satisfaction after using the program was also high. Our results coincide with the literature indicating that people treated with Internet-based programs report high levels of acceptability and high satisfaction (58, 73–78). In addition, this finding is consistent with the results obtained in the previous qualitative study, where participants showed a favorable opinion in the case where they had to use an online program (32). However, a recent study carried out in Germany showed that an IBI for depression in primary care settings obtained low acceptance; however, the authors explained that acceptance could be increased by displaying a brief informational video (35). Hence, a possible explanation for the high satisfaction with our IBI can be found in the richness and quality of the ICT support: the program offers information about the treatment and continued personalized feedback to users through various transversal tools for providing support, as described earlier (activity report, calendar, and how am I)?. These results are especially relevant in an ecological environment such as the primary care service in a national health system.
It is also important to mention that the results showed differences in expectations and satisfaction depending on some sociodemographic variables. Regarding the participants’ age, the results showed that the older they were, the higher their expectations (specifically in items of “treatment logic,” “treatment satisfaction,” and “recommending to others”). According to a recent individual patient data meta-analysis (IPDM) (78), younger ages were related to more dropouts. In the present study, older age was associated with better expectations; thus, the results in our study could be related with those found in that study. However, this is only a possible explanation, and it is not possible to have a firm conclusion about this aspect. With regard to the participants’ sex, the analysis showed that there was greater satisfaction in women than in men on the items “usefulness for other disorders” and “usefulness for the patient” and on the total score. The IDPM, mentioned above (78), revealed that men had a greater risk of dropping out. This result could indicate that men are less satisfied than women, as in our study. However, this is only a possible explanation since we don’t have information regarding the satisfaction with the program of the dropout participants. It should also be taken into account that women try harder to cope with depression than men (78), which would indicate a greater willingness to continue and complete IBIs and be more satisfied with them. Moreover, there is also evidence supporting the idea that women are more aware of health problems than men (78, 79). Furthermore, the prevalence data indicate that there are also more depressed women than men (2, 80), which may influence the difference in acceptance. Regarding marital status, the results revealed that there were no statistically significant differences among the different categories on the expectation and satisfaction scales. The same thing occurred in the case of the educational level.
Another important result is that patients who finished the intervention had higher expectations about it. Taking into account the significant dropout rate from IBIs (73, 81–83), this result shows the relevance of working to improve the participants’ expectations as a way to promote program adherence. The sociodemographic variables and the participants’ expectations could be understood as moderators.
The completion rate in the present study was 45.45% of the sample. Thus, the dropout rates in the present study are high (54.4%), but it is important to take into account that the context of the present study is the primary care setting. Thus, the sample had specific characteristics (with a lower educational level than in other contexts: only 18.18% had university studies). A recent IPDM showed that a lower educational level significantly increased the risk of dropping out (78). Furthermore, the literature shows that in unguided web-based interventions for depression, compared to guided web-based interventions, average levels of adherence were estimated at 26% in unguided interventions and 72% in guided interventions (22, 78). Dropout rates in Internet-based treatment programs range from 2% to 83%, with a median of 19% and a weighted average of 31%, due to various causes, one of which could be not providing human support (84). In the present paper, patients did not make much use of the support offered (only 11.9%). For this reason, almost all the participants received unguided intervention, which may have influenced the results regarding dropouts.
Regarding the relationship between the expectation and satisfaction measurements and the primary clinical variable (depression), and the relation between expectations and the clinical variables assessed through the intervention program (depression and anxiety impairment and positive and negative affect), the results showed that expectations related to “usefulness for the patient” were a predictor that was statistically related to the results on the BDI-II. Thus, positive expectations were associated with better outcomes on the primary outcome measure. Furthermore, data are presented for the relationship between expectations and improvements in the clinical variables assessed throughout the intervention program. The results for the negative emotionality variables (depression and anxiety impairment and negative affect) showed that the higher the expectations, the lower the depression and anxiety impairment and NA exhibited by the patients during the 10 intervention modules. In the case of positive emotionality, the results showed that the higher the expectations, the greater the PA exhibited by the patients on the 10 measurement occasions. These results agree with the literature (36, 39) and suggest the importance of considering patients’ expectations about treatment as a factor that explains part of the therapeutic efficacy. Therefore, it is important to inform the participants, before the intervention, about why the program will be useful and how it can help them. Thus, it is important to present the IBI in a way that allows patients to obtain adequate and relevant information that can activate positive but realistic expectations about the intervention. Furthermore, in IBIs where the therapist is less visible, it is important to spend time developing strategies (for example, via telephone or with manuals) to emphasize the usefulness of the online intervention.
The literature shows that, in online programs without support during the intervention, preceding human contact increases the effectiveness (20); therefore, it would be interesting during this contact to transmit the rationale and the usefulness of the program. In addition, another strategy would be to dedicate the first modules of the program to this topic. In the IBI used in the present study, the “Home” and “Welcome” modules explain the usefulness of the program by showing examples of people who have similar problems. Regarding satisfaction with the IBI, results showed that the higher the satisfaction score reflected on the “treatment logic” item, the greater the improvement on the BDI-II. Thus, the satisfaction of perceiving the intervention as a logical treatment is related to greater benefits in the clinical variable. This is an important result because satisfaction has been found to help to optimize intervention efficacy (10, 45), and the results in the present study show that the same thing might occur with an IBI program for depression in primary care settings. However, it is important to mention that in the absence of a controlled treatment condition, this submission is not about treatment efficacy. The efficacy of the intervention was shown in the RCT carried out (29).
Based on these results, it is important to take into account the benefit of developing interventions not only with effective treatment components but also with aspects that improve patients’ expectations and satisfaction with the intervention, because the treatments’ completion and effectiveness will improve with the acceptability of the intervention (19, 69, 83). As we pointed out in the introduction, although clinical effectiveness is important, an additional criterion that is likely to affect implementation is the acceptability (37). At this moment, we are working on an implementation study to try to introduce this type of intervention in the National Health System in Spain. The task is not at all simple, given that in primary care in Spain, there are no clinical psychologists. There are only clinical psychologists in specialized care. For psychological treatment, the primary care physician has to refer the patient to specialized care. One of the biggest barriers is clinical professionals’ resistance to adopting this type of program applied through the Internet. Therefore, it is essential to demonstrate with data that this type of treatment works (as seen in the RCT) at an affordable cost. We worked on this cost–benefit issue in another study (85). Furthermore, future research should study therapists’ attitudes toward IBIs in the primary care context to find out how to improve their expectations and satisfaction (24, 86).
Limitations
The present study has some limitations that should be taken into account. First, the sample size is small, and the dropout rates are high (54.4%). It is true that the real sample size is modest, which is reflected in the limitations and the difficulty of generalizing the results. However, the results are from the first RCT performed in Spain on an IBI for depression. For this reason, although the sample is modest, we think the results are important for future implementations of IBIs in our country or in other Spanish-speaking countries with a Latin culture. Second, the assessments were carried out through the online program. Previous studies indicate that if assessments are carried out through the web, psychometric properties may change (47). However, other studies have demonstrated the usefulness of Internet and telephone assessments, and their concordance with traditional face-to-face assessment (87–90). Third, the effect of the GPs or psychotherapists may be a source of variability, and in the current study, this was not taken into account (29). Fourth, the generalizability of this work is limited by evaluating the participants’ satisfaction with one intervention. Fifth, because we do not have data about the dropouts’ satisfaction with the intervention, we do not know whether satisfaction or other variables moderated the completion of the program. However, following the literature, there might be some reasons that affect the dropout rates, such as the amount of support (insufficient) and the lack of specificity of the intervention contents to the participants’ own psychological problems (91). Sixth, using individual items from the expectation and satisfaction questionnaires limited the psychometric properties of these variables included in our analyses. However, in the balance between psychometric validity and specificity of the information offered by the individual items, we considered it more useful and informative to analyze individual items. Another limitation of our study was the low percentage of variance explained by the multiple regression for expectations on treatment efficacy (18.7% only). This finding limits the usefulness of the predictive model for practical purposes. Finally, the acceptability assessment (expectations and satisfaction) was based on two questionnaires, rather than qualitative information that can provide a general impression of the program. Furthermore, it would have been interesting to have used other assessment measures of acceptance [i.e., the Client Satisfaction Questionnaire adapted to IBIs (CSQ-I)] (92). In addition, the regression models elaborated to find predictors of treatment efficacy were based on the individual items of the expectation and satisfaction scales, not on composite scores, so that the reliability of these indicators was of limited scope. In the future, it would be appropriate to include qualitative analyses to report complementary and detailed data on this topic. Furthermore, it would be interesting to compare the results obtained in the present work with previous studies that we have carried out using the expectation and satisfaction scales.
Conclusions
Despite its limitations, to the best of our knowledge, this is the first study in Spain to address this issue in the field of IBIs for depression in primary care. The IBI showed high acceptance (expectations and satisfaction) that is related to the intervention’s efficacy. Research on IBI acceptability in relation to completion and efficacy variables could help to implement the treatment offered, reaching more people and improving the outcome. Future research in this field is needed.
Ethics Statement
We confirm that any aspect of the work covered in this manuscript that has involved human patients has been conducted with the ethical approval of all relevant bodies and that such approvals are acknowledged within the manuscript. The study was approved by the ethics committee of the regional health authority on April 7, 2010 (ref: PI10/01083). All participants interested in participating signed an informed consent form.
Author Contributions
AM and CS drafted the manuscript, with important contributions from CB, AG-P, and RB. CB, in collaboration with JG-C, MG, FM, and JM-M, designed the study and participated in each of its phases. AM, CS, MA, AC, and MH collaborated in the manuscript development and participated in each study phase. AM, CS, CB, and DC carried out the Internet-based adaptation of the treatment protocol, with important contributions by RB and AG-P. All authors participated in the review and revision of the manuscript and have approved the final manuscript to be published.
Funding
This study was financed by the Instituto de Salud Carlos III of the Spanish Ministry of Economy and Competitiveness with the PI10/01083 grant and of the Ministry of Economy and Competitiveness (Plan Nacional I + D + I. PSI2014-54172-R). The project also received funding from the Network for Prevention and Health Promotion in Primary Care (RD12/0005) grant from the Instituto de Salud Carlos III of the Ministry of Economy and Competitiveness (Spain), co-financed with European Union ERDF funds, and the PSI2014-54172-R grant from the Ministry of Economy and Competitiveness (Spain).
The Instituto de Salud Carlos III of the Spanish Ministry of Economy and Competitiveness with the PI10/01083 grant (Eficacia y coste-efectividad de un programa de psicoterapia asistida por ordenador para el tratamiento de la depresión mayor en atención primaria: estudio controlado, randomizado y cualitativo); the Network for Prevention and Health Promotion in primary Care (RD12/0005) grant; the Institute of Health Carlos III (ISCiii) CIBERobn is an initiative of ISCIII; the Government of Aragón; the Department of Innovation, Research; the University and FEDER “Construyendo desde Aragón”.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Abbreviations
ANOVA, analysis of variance; BDI-II, Beck Depression Inventory-II; CBT, cognitive behavioral therapy; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition; GP, general practitioner; IBIs, Internet-based interventions; ICTs, information and communication technologies; ITT, intention-to-treat; IPDM, individual patient data meta-analysis; M, mean; MINI, Mini International Neuropsychiatric Interview; OASIS, Overall Anxiety Severity and Impairment Scale; ODSIS, Overall Depression Severity and Impairment Scale; PANAS, Positive and Negative Affect Scale; PA, positive affect; NA, negative affect; RCT, randomized control trial; SD, standard deviation.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00325/full#supplementary-material
Data Sheet 1 | Canonical correlation analysis.
References
1. Fernández FC, Martínez ÁM, Barragán AB, Martín M, del Mar Molero M. Prevalencia de la depresión en España: análisis de los últimos 15 años. Eur J Invest (2015) 5(2):267–79. doi: 10.30552/ejihpe.v5i2.118
2. Haro JM, Ayuso-Mateos JL, Bitter I, Demotes-Mainard J, Leboyer M, Lewis SW, et al. ROAMER: roadmap for mental health research in Europe. Int J Meth Psych Res (2014) 23(S1):1–14. doi: 10.1002/mpr.1406
3. Cuijpers P, Beekman AT, Reynolds CF. Preventing depression: a global priority. JAMA (2012) 307(10):1033–4. doi: 10.1001/jama.2012.271
4. Smit F, Cuijpers P, Oostenbrink J, Batelaan N, de Graaf R, Beekman A. Costs of nine common mental disorders: implications for curative and preventive psychiatry. J Ment Health Policy (2006) 9(4):193–200.
5. Cano-Vindel A, Salguero JM, Mae Wood C, Dongil E, Latorre JM. La depresión en atención primaria: prevalencia, diagnóstico y tratamiento. Papeles del psicólogo (2012) 33(1):2–11.
6. Üstün TB, Ayuso-Mateos JL, Chatterji S, Mathers C, Murray CJ. Global burden of depressive disorders in the year 2000. Br J Psychiatry (2004) 184(5):386–92. doi: 10.1192/bjp.184.5.386
7. World Health Organization. Depression and other common mental disorders. CC BY-NC-SA 30 IGO (2017) (1):1–22.
8. Mohr DC, Ho J, Duffecy J, Baron KG, Lehman KA, Jin L, et al. Perceived barriers to psychological treatments and their relationship to depression. J Clin Psychol (2010) 66(4):394–409. doi: 10.1002/jclp.20659
9. Lester H, Howe A. Depression in primary care: three key challenges. Postgrad med j (2008) 84(996):545–48. doi: 10.1136/pgmj.2008.068387
10. Marks IM, Cavanagh K, Gega L. Computer-aided psychotherapy: revolution or bubble? Br J Psychiatry (2007) 191(6):471–73. doi: 10.1192/bjp.bp.107.041152
11. Purves DG, Bennett M, Wellman N. An open trial in the NHS of Blues Begone®: a new home based computerized CBT program. Behav Cogn Psychother (2009) 37(5):541–51. doi: 10.1017/S1352465809990282
12. Cuijpers P, van Straten A, Warmerdam L, van Rooy MJ. Recruiting participants for interventions to prevent the onset of depressive disorders: possible ways to increase participation rates. BMC Health Serv Res (2010) 10(1):181. doi: 10.1186/1472-6963-10-181
13. Kazdin AE, Blasé SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspect Psychol Sci (2011) 6(1):21–37. doi: 10.1177/1745691610393527
14. Kazdin AE, Rabbitt SM. Novel models for delivering mental health services and reducing the burdens of mental illness. Clin Psychol Sci (2013) 1(2):170–91. doi: 10.1177/2167702612463566
15. Cuijpers P, van Straten A, Schuurmans J, van Oppen P, Hollon SD, Andersson G. Psychotherapy for chronic major depression and dysthymia: a meta-analysis. Clin Psychol Rev (2010) 30(1):51–62. doi: 10.1016/j.cpr.2009.09.003
16. Ellis LA, Collin P, Hurley PJ, Davenport TA, Burns JM, Hickie IB. Young men’s attitudes and behaviour in relation to mental health and technology: implications for the development of online mental health services. BMC Psychiatry (2013) 13(1):119. doi: 10.1186/1471-244X-13-119
17. Kazdin AE. Technology-based interventions and reducing the burdens of mental illness: perspectives and comments on the special series. Cogn Behav Pract (2015) 22(3):359–66. doi: 10.1016/j.cbpra.2015.04.004
18. Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, et al. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health Technol Asses (2006) 10(33):1–186. doi: 10.3310/hta10330
19. Kaltenthaler E, Parry G, Beverley C, Ferriter M. Computerised cognitive–behavioural therapy for depression: systematic review. Br J Psychiatry (2008) 193(3):181–4. doi: 10.1192/bjp.bp.106.025981
20. Johansson R, Andersson G. Internet-based psychological treatments for depression. Expert Rev Neurother (2012) 12(7):861–9. doi: 10.1586/ern.12.63
21. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided Internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA Psychiatry (2017) 74(4):351–9. doi: 10.1001/jamapsychiatry.2017.0044
22. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev (2012) 32(4):329–42. doi: 10.1016/j.cpr.2012.02.004
23. Paganini S, Teigelkötter W, Buntrock C, Baumeister H. Economic evaluations of internet-and mobile-based interventions for the treatment and prevention of depression: a systematic review. J Affect Disord (2018) 225:733–55. doi: 10.1016/j.jad.2017.07.018
24. Schröder J, Berger T, Westermann S, Klein JP, Moritz S. Internet interventions for depression: new developments. Dialogues Clin Neurosci (2016) 18(2):203.
25. Spek V, Cuijpers PIM, Nyklíček I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med (2007) 37(3):319–28. doi: 10.1017/S0033291706008944
26. Gun SY, Titov N, Andrews G. Acceptability of Internet treatment of anxiety and depression. Australas Psychiatry (2011) 19(3):259–64. doi: 10.3109/10398562.2011.562295
27. Quero S, Botella C, Bretón JM, Soler C, Mira A, Baños RM. Future clinical psychologists’ attitudes toward an Internet-based intervention for treating. 18th Annual CyberPsychology and Cybertherapy Conference: Proceedings of Experience the Future of Health & Well-Being. Brussels, Belgium: Virtual Reality Medical Institute & Interactive Media Institute (2013).
28. González-Peña P, Torres R, del Barrio V, Olmedo M. Uso de las nuevas tecnologías por parte de los psicólogos españoles y sus necesidades. Clínica y Salud (2017) 28(2):81–91. doi: 10.1016/j.clysa.2017.01.001
29. Montero-Marín J, Araya R, Pérez-Yus M, Mayoral F, Gili M, Botella C, et al. An internet-based intervention for depression in primary care in Spain: a randomized controlled trial. J Med Internet Res (2016) 18(8):e231. doi: 10.2196/jmir.5695
30. Clarke G, Eubanks D, Reid CK, O’Connor E, DeBar LL, Lynch F, et al. Overcoming depression on the Internet (ODIN)(2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res (2005) 7(2):e16. doi: 10.2196/jmir.7.2.e16
31. Learmonth D, Rai S. Taking computerized CBT beyond primary care. Brit J Clin Psychol (2008) 47(1):111–8. doi: 10.1348/014466507X248599
32. Montero-Marín J, Prado-Abril J, Botella C, Mayoral-Cleries F, Baños R, Herrera-Mercadal P, et al. Expectations among patients and health professionals regarding Web-based interventions for depression in primary care: a qualitative study. J Med Internet Res (2015) 17(3). doi: 10.1186/1471-244X-13-64
33. Whiteside U, Richards J, Steinfeld B, Simon G, Caka S, Tachibana C, et al. Online cognitive behavioral therapy for depressed primary care patients: a pilot feasibility project. Perm J (2014) 18(2):21. doi: 10.7812/TPP/13-155
34. De Graaf LE, Gerhards SAH, Arntz A, Riper H, Metsemakers JFM, Evers SMAA, et al. Clinical effectiveness of online computerised cognitive–behavioural therapy without support for depression in primary care: randomised trial. Brit J Psychiat (2009) 195(1):73–80. doi: 10.1192/bjp.bp.108.054429
35. Ebert DD, Berking M, Cuijpers P, Lehr D, Pörtner M, Baumeister H. Increasing the acceptance of Internet-based mental health interventions in primary care patients with depressive symptoms. J Affect Disord (2015) 176:9–17. doi: 10.1016/j.jad.2015.01.056
36. Schröder J, Berger T, Meyer B, Lutz W, Hautzinger M, Späth C, et al. Attitudes towards Internet interventions among psychotherapists and individuals with mild to moderate depression symptoms. Cognit Ther Res (2017) 41(5):745–56. doi: 10.1007/s10608-017-9850-0
37. Wallin EE, Mattsson S, Olsson EM. The preference for Internet-based psychological interventions by individuals without past or current use of mental health treatment delivered online: a survey study with mixed-methods analysis. JMIR Ment Health (2016) 3(2):e25. doi: 10.2196/mental.5324
38. Kazdin AE. Acceptability of alternative treatments for deviant child behavior. J Appl Behav Anal (1980) 13(2):259–73. doi: 10.1901/jaba.1980.13-259
39. Rush B, Scott RE. Approved telehealth outcome indicator guidelines: quality, access, acceptability and cost. Calgary, AB, Canada: Calgary Health Telematics Unit, University of Calgary (2004).
40. Swift JK, Callahan JL. The impact of client treatment preferences on outcome: a meta-analysis. J Clin Psychol (2009) 65(4):368–81. doi: 10.1016/j.cpr.2005.03.002
41. Peñate W, Fumero A. A meta-review of Internet computer-based psychological treatments for anxiety disorders. J Telemed Telecare (2016) 22(1):3–11. doi: 10.1177/1357633X15586491
42. Botella C, Gallego MJ, García-Palacios A, Baños RM, Quero S, Alcañiz M. The acceptability of an Internet-based self-help treatment for fear of public speaking. Brit J Guid Couns (2009) 37(3):297–311. doi: 10.1080/03069880902957023
43. Botella C, Mira A, Moragrega I, García-Palacios A, Bretón-López J, Castilla D, et al. An Internet-based program for depression using activity and physiological sensors: efficacy, expectations, satisfaction, and ease of use. Neuropsych Dis Treat (2016) 12:393. doi: 10.2147/NDT.S93315
44. Botella C, Pérez-Ara MÁ, Bretón-López J, Quero S, García-Palacios A, Baños RM. In vivo versus augmented reality exposure in the treatment of small animal phobia: a randomized controlled trial. PLoS One (2016) 11(2):e0148237. doi: 10.1371/journal.pone.0148237
45. De Graaf LE, Huibers MJ, Riper H, Gerhards SA, Arntz A. Use and acceptability of unsupported online computerized cognitive behavioral therapy for depression and associations with clinical outcome. J Affect Disord (2009) 116(3):227–31. doi: 10.1016/j.jad.2008.12.009
46. Greenberg RP, Constantino MJ, Bruce N. Are patient expectations still relevant for psychotherapy process and outcome? Clin Psychol Rev (2006) 26(6):657–78. doi: 10.1016/j.cpr.2005.03.002
47. Buchanan T, Johnson JA, Goldbergn LR. Implementing a five-factor personality inventory for use on the internet. Eur J Psychol Assess (2005) 21(2):115–27. doi: 10.1027/1015-5759.21.2.115
48. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Quarterly (2003) 27(3):425–78. doi: 10.2307/30036540
49. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly (1989) 13(3):319. doi: 10.2307/249008
50. Holden RJ, Karsh B. The technology acceptance model: its past and its future in health care. J Biomed Inform (2010) Feb 43(1):159–72. doi: 10.1016/j.jbi.2009.07.002
51. Dünnebeil S, Sunyaev A, Blohm I, Leimeister JM, Krcmar H. Determinants of physicians’ technology acceptance for e-health in ambulatory care. Int J Med Inform (2012) Nov 81(11):746–60. doi: 10.1016/j.ijmedinf.2012.02.002
52. Wootton BM, Titov N, Dear BF, Spence J, Kemp A. The acceptability of Internet-based treatment and characteristics of an adult sample with obsessive compulsive disorder: an Internet survey. PLoS One (2011) 6(6):e20548. doi: 10.1371/journal.pone.0020548
53. López-del-Hoyo Y, Olivan B, Luciano JV, Mayoral F, Roca M, Gili M, et al. Low intensity vs. BMC Psychiatry (2013) 13(1):21. doi: 10.1186/1471-244X-13-21
54. Sanz J, García-Vera MP, Espinosa R, Fortún M, Vázquez C. Adaptación española del Inventario para la Depresión de Beck-II (BDI-II): 3. Clínica y salud (2005) 16(2):249–80.
55. Lecrubier Y, Sheehan DV, Weiller E, Amorim P, Bonora I, Sheehan KH, et al. The Mini International Neuropsychiatric Interview (MINI). Eur Psychiat (1997) 12(5):224–31. doi: 10.1016/S0924-9338(97)83296-8
56. Algoe SB, Fredrickson BL. Emotional fitness and the movement of affective science from lab to field. Am Psychol (2011) 66(1):35. doi: 10.1037/a0021720
57. Ekers D, Richards D, McMillan D, Bland JM, Gilbody S. Behavioural activation delivered by the non-specialist: phase II randomised controlled trial. Brit J Psychiat (2011) 198(1):66–72. doi: 10.1192/bjp.bp.110.079111
58. Mira A, Bretón-López J, García-Palacios A, Quero S, Baños RM, Botella C. An Internet-based program for depressive symptoms using human and automated support: a randomized controlled trial. Neuropsych Dis Treat (2017) 13:987. doi: 10.2147/NDT.S130994
59. Ferrando L, Bobes J, Gibert J, Lecrubier Y. MINI International Neuropsychiatric Interview. Spanish version, 5(0). In: Bobes J, Bousoño M, González MP, editors. Management of mental and behavioral disorders in primary care. Gofer (1997).
60. Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological (1996) 78(2):490–8. doi: 10.1037/t00742-000
61. Schotte CKW, Maes M, Cluydts R, De Doncker D, Cosyns P. Construct validity of the Beck Depression Inventory in a depressive population. J Affect Disord (1997) 46(2):115–25. doi: 10.1016/S0165-0327(97)00094-3
62. Bentley KH, Gallagher MW, Carl JR, Barlow DH. Development and validation of the Overall Depression Severity and Impairment Scale. Psychol Assessment (2014) 26(3):815. doi: 10.1037/a0036216
63. González-Robles A, Mira A, Breton-Lopez J, Riera A, Raya D, Baños RM, et al. Psychometric properties of the Overall Depression Severity and Impairment Scale in Spanish sample. General and clinical populations. Proceedings of European Society for Research on Internet Intervention. Warsaw (2015). p. 205.
64. Norman SB, Hami Cissell S, Means-Christensen AJ, Stein MB. Development and validation of an overall anxiety severity and impairment scale (OASIS). J Depress Anxiety (2006) 23(4):245–9. doi: 10.1002/da.20182
65. González-Robles A, Mira A, Miguel C, Molinari G, Garcia-Palacios A, Bretón-López J, et al. A brief online transdiagnostic measure: psychometric properties of the Overall Anxiety Severity and Impairment Scale (OASIS) among Spanish patients with emotional disorders. PLoS One (2018) 13(11):e0206516. doi: 10.1371/journal.pone.0206516
66. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol (1988) 54(6):1063. doi: 10.1037//0022-3514.54.6.1063
67. Sandín B, Chorot P, Lostao L, Joiner TE, Santed MA, Valiente RM. Escalas PANAS de afecto positivo y negativo: validación factorial y convergencia transcultural. Psicothema (1999) 11(1):37–51.
68. Borkovec TD, Nau SD. Credibility of analogue therapy rationales. J Behav Ther Exp Psychiatry (1972) 3(4):257–60. doi: 10.1016/0005-7916(72)90045-6
69. Quero S, Campos D, Mira A, Castilla D, Botella C, Bretón-López J. Exploring the relationship between the acceptability of a flying phobia treatment delivered via the Internet and clinical outcomes. Proceedings of 22th Annual CyberPsychology, Cybertherapy & Social Networking Conference. England, UK:Virtual Reality Medical Institute & Interactive Media Institute (2017).
70. Tortella-Feliu M, Botella C, Llabrés J, Bretón-López JM, del Amo AR, Baños RM, et al. Virtual reality versus computer-aided exposure treatments for fear of flying. Behav Modif (2011) 35(1):3–30. doi: 10.1177/0145445510390801
71. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Erlbaum (1988).
72. Sherry A, Henson RK. Conducting and interpreting canonical correlation analysis in personality research: a user-friendly primer. J Pers Assess (2005) 84(1):37–48. doi: 10.1207/s15327752jpa8401_09
73. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One (2010) 5(10):e13196. doi: 10.1371/journal.pone.0013196
74. Kaltenthaler E, Sutcliffe P, Parry G, Beverley C, Rees A, Ferriter M. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychol Med (2008) 38(11):1521–30. doi: 10.1017/S0033291707002607
75. Montero-Marín J, Carrasco JM, Roca M, Serrano-Blanco A, Gili M, Mayoral F, et al. Expectations, experiences and attitudes of patients and primary care health professionals regarding online psychotherapeutic interventions for depression: protocol for a qualitative study. BMC Psychiatry (2013) 13(1):64. doi: 10.1186/1471-244X-13-64
76. Titov N, Dear BF, Schwencke G, Andrews G, Johnston L, Craske MG, et al. Transdiagnostic internet treatment for anxiety and depression: a randomised controlled trial. Behav Res Ther (2011) 49(8):441–52. doi: 10.1016/j.brat.2011.03.007
77. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med (2009) 39(5):705–12. doi: 10.1017/S0033291708004224
78. Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G, et al. Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data meta-analysis’. Psychol Med (2015) 45(13):2717–26. doi: 10.1017/S0033291715000665
79. Babwah F, Baksh S, Blake L, Cupid-Thuesday J, Hosein I, Sookhai A, et al. The role of gender in compliance and attendance at an outpatient clinic for type 2 diabetes mellitus in Trinidad. Revista Panamericana de SaludPública (2006) 19(2):79–84. doi: 10.1590/S1020-49892006000200002
80. Haro JM, Palacín C, Vilagut G, Martínez M, Bernal M, Luque I, et al. Prevalencia de los trastornos mentales y factores asociados: resultados del estudio ESEMeD-España. Med Clin (2006) 126(12):445–51. doi: 10.1157/13086324
81. Melville KM, Casey LM, Kavanagh DJ. Dropout from Internet-based treatment for psychological disorders. Br J Clin Psychol (2010) 49(4):455–71. doi: 10.1348/014466509X472138
82. Van Ballegooijen W, Cuijpers P, van Straten A, Karyotaki E, Andersson G, Smit JH, et al. Adherence to Internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One (2014) 9(7):e100674. doi: 10.1371/journal.pone.0100674
83. Santana L, Fontenelle LF. A review of studies concerning treatment adherence of patients with anxiety disorders. Patient Prefer Adherence (2011) 5:427. doi: 10.2147/PPA.S23439
84. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther (2009) 38(4):196–205. doi: 10.1080/16506070903318960
85. Romero-Sanchiz P, Nogueira-Arjona R, García-Ruiz A, Luciano JV, García-Campayo J, Gili M, et al. Economic evaluation of a guided and unguided internet-based CBT intervention for major depression: results from a multicenter, three-armed randomized controlled trial conducted in primary care. PLoS One (2017) 12(2):e0172741. doi: 10.1371/journal.pone.0172741
86. Becker EM, Jensen-Doss A. Computer-assisted therapies: examination of therapist-level barriers to their use. Behav Ther (2013) 44(4):614–24. doi: 10.1016/j.beth.2013.05.002
87. Campos D, Quero S, Bretón López J, Riera del Amo A, Mira A, Tortella Feliu M, et al. Concordancia entre la evaluación psicológica a través de Internet y la evaluación tradicional aplicada por el terapeuta para la fobia a volar. TesisPsicológica (2015) 10(2):52–67.
88. Carlbring P, Brunt S, Bohman S, Austin D, Richards J, Öst LG, et al. Internet vs. Comput Human Behav (2007) 23(3):1421–34. doi: 10.1016/j.chb.2005.05.002
89. Hedman E, Ljótsson B, Rück C, Furmark T, Carlbring P, Lindefors N, et al. Internet administration of self-report measures commonly used in research on social anxiety disorder: a psychometric evaluation. Comput Human Behav (2010) 26(4):736–40. doi: 10.1016/j.chb.2010.01.010
90. Hedman E, Ljótsson B, Blom K, El Alaoui S, Kraepelien M, Rück C, et al. Telephone versus internet administration of self-report measures of social anxiety, depressive symptoms, and insomnia: psychometric evaluation of a method to reduce the impact of missing data. J Med Internet Res (2013) 15(10):e229. doi: 10.2196/jmir.2818
91. Fernández-Álvarez J, Díaz-García A, González-Robles A, Baños R, García-Palacios A, Botella C. Dropping out of a transdiagnostic online intervention: a qualitative analysis of client’s experiences. Internet Interv (2017) 10:29–38. doi: 10.1016/j.invent.2017.09.001
Keywords: depression, Internet-based intervention, acceptability, primary care, expectations, satisfaction
Citation: Mira A, Soler C, Alda M, Baños R, Castilla D, Castro A, García-Campayo J, García-Palacios A, Gili M, Hurtado M, Mayoral F, Montero-Marín J and Botella C (2019) Exploring the Relationship Between the Acceptability of an Internet-Based Intervention for Depression in Primary Care and Clinical Outcomes: Secondary Analysis of a Randomized Controlled Trial. Front. Psychiatry 10:325. doi: 10.3389/fpsyt.2019.00325
Received: 15 November 2018; Accepted: 26 April 2019;
Published: 10 May 2019.
Edited by:
Yuan-Pang Wang, University of São Paulo, BrazilReviewed by:
Xavier Noel, Free University of Brussels, BelgiumScott Tyler Aaronson, Sheppard and Enoch Pratt Hospital, United States
Severin Hennemann, Johannes Gutenberg University Mainz, Germany
Copyright © 2019 Mira, Soler, Alda, Baños, Castilla, Castro, García-Campayo, García-Palacios, Gili, Hurtado, Mayoral, Montero-Marín and Botella. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Adriana Mira, miraa@unizar.es