Efficacy of a between-session homework component delivered digitally for the treatment of adjustment disorders: Results from a pilot randomized clinical trial

Background: Adjustment Disorder (AjD) is one of the most frequently diagnosed mental disorders worldwide. However, there is still no treatment of choice for this problem. One of the first treatment programs specifically designed for AjD is the one presented in this study, a CBT protocol that uses Virtual Reality (VR) to improve patients’ adherence. Another key aspect to improve treatment outcomes is the use of between-sessions homework. However, little is known about how to increase patient engagement with these tasks. The use of Information and Communication Technologies (ICTs) can help patients to see homework as less demanding and provides greater flexibility. A previous study tested the feasibility of TEO (Online Emotional Therapy), a program to implement online homework during treatment. The aim of this study is to present session-to-session efficacy data from the homework assignment component and the patients’ preferences and opinions about this component in both formats (Traditional vs. TEO).

Methods: A two-arm pilot randomised clinical trial (RCT) was conducted with 57 patients with AjD. The two groups received the same CBT intervention with VR support, but one group completed homework in the traditional way and the other using TEO. Mood, self-efficacy and positive and negative emotions were measured to test the effect of the homework component in each treatment session. Preferences about the homework format were also assessed before and after treatment.

Results: Both types of administration produced significant improvements in mood, self-efficacy and emotional variables after completion of the homework component and before and after each homework session. Effect sizes ranged from small to large, with no differences between groups. Regarding the acceptability of the program, most participants preferred the TEO condition both before starting the treatment and after completing the homework assignment. Overall, participants were satisfied with all homework sessions and found them useful. Again, no significant differences were found between groups.

Conclusion: This is the first time that the effect of an online homework program during AjD treatment has been tested. It is proposed that this way of implementing homework could be as effective as the traditional format, but would be preferred by patients.

Clinical Trial Registration: clinicaltrials.gov, identifier NCT02452411

1 Introduction

Adjustment disorder (AjD) is considered one of the most frequently diagnosed mental disorders worldwide (Reed et al., 2011). Prevalence rates vary considerably and range from 1 to 2% in the general population (Maercker et al., 2012; Zelviene and Kazlauskas, 2018) to 19% in high-risk or clinical samples (O'Donnell et al., 2016).

The inclusion of this diagnostic category in the latest version of the International Classification of Diseases (ICD-11) of the World Health Organization (World Health Organization, 2018) involves a considerable advance in its conceptualization as a mental disorder. AjD is included within disorders specifically related to stress, constituting for the first time a full syndrome with its own list of diagnostic criteria. However, it is necessary to point out that the ICD-11 was not yet available when this research was carried out, so we used the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) (American Psychiatric Association [APA], 2000) and ICD-10 (World Health Organization, 1992). The literature has not reported significant differences using one manual or another (Maercker et al., 2012; Perkonigg et al., 2018).

According to the WHO, the AjD diagnosis requires the presence of one or more identifiable stressful events, symptoms of worry related to the stressor, and a failure to adapt to it that causes significant interference. In addition, criteria for any other mental disorder must not be met and usually symptoms do not last more than 6 months after the stressor and its consequences have disappeared.

Previous research suggest that AjD can cause a high impairment in patients by increasing the risk of suicide ideation and behaviors and the likelihood of developing more serious psychiatric disorders (Casey et al., 2015; O’Donnell et al., 2016). Despite the clinical relevance of this disorder, few studies have focused on developing specific psychological interventions (Kazlauskas et al., 2017; Bachem & Casey, 2018; O’Donnell et al., 2018). Therefore, there is currently no gold standard treatment approach for AjD.

One of the first intervention protocols developed specifically for AjD is the one of Botella et al. (2008). It is a Cognitive Behavioral Therapy (CBT) that seeks to promote an adaptative change in the meaning of the stressor, enabling patients to learn from what has happened.

This program has shown to be effective (Baños et al., 2011) applied in a traditional face-to-face format and supported by a flexible Virtual Reality (VR) system called EMMA’s World (Baños et al., 2009; Quero et al., 2019a). Indeed, incorporating VR into treatment increased patient’s motivation and adherence (Guillén et al., 2018) by making the intervention less aversive (Freeman et al., 2017).

Another way to increase adherence and reduce dropouts during treatment is using homework (Benbow and Anderson, 2019). Performing tasks between sessions is an opportunity for patients to practice autonomously what they have learned in therapy and to generalize it to their daily lives. Previous studies suggest that homework completion can lead to faster and substantial changes in the symptoms of the disorder (Beck, 1979; Burns and Spangler, 2000; Kazantzis et al., 2000; Busch et al., 2010; Conklin & Strunk, 2015; Haller & Watzke, 2020) and therefore to a greater response to treatment (Neimeyer & Feixas, 1990; Bryant et al., 1999; Kazantzis et al., 2010; Stirman et al., 2018; Callan et al., 2019). In addition, authors such as Conklin and Strunk (2015) suggest that the effects of homework are often relatively immediate, so a session-by-session analysis of homework assignments could be useful to study changes in treatment outcomes and draw conclusions about what content or tasks might be related to patient improvement.

Nevertheless, very little is known about how to increase patients’ engagement with these tasks. If we consider that avoidance is a central symptom in AjD and in any stress-and-trauma-related disorders, it is reasonable that patients perceive the tasks related with the stressful event as too hard or complicated (Stirman et al., 2018). Thus, new ways of managing homework between-sessions must be developed to overcome these barriers. One of the proposals to increase patients’ interest with homework refers to the assignments of homework via the Internet. The use of Information and Communication Technologies (ICTs) helps patients to see activities as less demanding, provides greater flexibility to patient and therapist and facilitates the adaptation of tasks to each person. In addition, this type of format makes it possible to provide immediate feedback to patients, which can increase their motivation and engagement (Quero et al., 2019a).

So far, very few studies have used the Internet to homework assignments. One example is the research by Reger et al. (2013), who developed the cell-phone application “PE Coach” for patients with post-traumatic stress disorder. This platform includes, among other functions, the ability to record treatment sessions, build live hierarchies, view activities already completed, organize homework on a schedule, and check patient adherence to homework.

Another example is the research by Callan et al. (2020) in which a mobile-phone application called “CBT Mobile-Work” was developed. In this case the objective was to create a platform that allowed patients to practice CBT skills over the internet. The results of the study showed preliminary feasibility and high satisfaction of both patients and therapists. It is believed that this application can increase patients’ adherence to perform tasks at home.

Similarly, Michelle et al. (2014) developed an app called “CBT Assistant” for homework assignments in patients with social anxiety disorder (SAD). It helped patients to practice CBT activities between sessions. The authors suggest that CBT Assistant can improve treatment outcomes by increasing patient’s adherence to homework.

Finally, we find the study by Strickland (2019) in which the mobile application “Kip” was designed to be combined with psychotherapy. This platform is not aimed at treating any specific disorder, but is simply intended to complete homework between sessions, as well as to monitor adherence to these tasks. Clinician can observe patient data and discuss it in session. In short, the study suggests that carrying out activities through this mobile app can correlate with a more beneficial outcome in the treatment.

To the authors’ knowledge, no study has proposed the development of a web platform for homework assignment in patients with AjD. In a previous study (Quero et al., 2011) this research team presented the TEO program, an Online Emotional Therapy to apply homework assignments during the treatment. This was the first time that homework could be completely managed over the internet. The feasibility of TEO (doability, initial efficacy and acceptability) was tested in the pilot Randomized Clinical Trial (RCT) conducted by Quero et al. (2019b). However, so far we do not have session-by-session efficacy data that would allow us to analyse which tasks might be most related to changes in outcomes and we do not know which homework assignment format is preferred by patients. These data could help to improve future approaches aimed at increasing patients’ motivation and adherence to homework. Therefore, the aim of this study is to present session-by-session efficacy data on the homework assignment component and patients’ preferences and opinions about this component in both formats (Traditional vs. TEO).

2 Materials and methods

2.1 Design

A two-arm exploratory pilot Randomized Clinical Trial (RCT) was conducted with three assessment time points (pre-treatment, post-treatment and 12-month follow-up). Participants were randomized into two experimental conditions: 1) the traditional homework group and 2) the Online Emotional Therapy (TEO) system for homework assignment group. Both groups received the same VR-supported CBT treatment (Botella et al., 2008) but with one difference: the format of homework administration (Traditional vs. TEO).

This study was conducted according to the Consolidated Standards of Reporting Trials statement (CONSORT, http://www.consort-statement.org) and was registered at clinicaltrials.gov (NCT02452411). It was approved by the Ethics Committee of the Universitat Jaume I.

2.2 Participants

The recruitment, inclusion and exclusion criteria, and detailed sample characteristics for both experimental conditions have been described previously (Quero et al., 2019b) and are briefly presented here.

The study was conducted at the Emotional Disorders Clinic in Universitat Jaume I (Castellón, Spain). Participants were recruited through advertisements in different media (e-mails, posters, radio and press) and through referrals made directly from the clinic.

Potential participants were included in the study if they: 1) were between 18 and 65 years of age, 2) met the diagnostic criteria for AjD according to DSM-IV-TR and ICD-10 and 3) had access to a computer with Internet connection. Exclusion criteria were: 1) current dependence or abuse to alcohol or other drugs, 2) psychosis or severe personality disorder 3) severe organic pathology that prevents applying the treatment, 4) risk of suicide, 5) being receiving another psychological treatment for the same problem and 6) the presence of changes in medication during the time of the study in case the patient is receiving pharmacological treatment.

The flow diagram with all the evaluation moments can be consulted in Quero et al. (2019b). In this study, only data from the pre-post homework component and throughout each homework session are presented. The sample was at convenience, so a total of 46 participants who had completed the homework component were included in the analyses (24 patients in the Traditional condition and 22 in the TEO condition). See Table 1 for a completed description of participants’ characteristics.

TABLE 1

TABLE 1. Sociodemographic variables of the two experimental groups and the total sample at pre-treatment.

2.3 Measures

2.3.1 AjD diagnosis

The Diagnostic Interview for Adjustment Disorders (Rachyla, 2022) based on the diagnostic criteria of the DSM-IV-TR (APA, 2000), the ICD-10 (WHO, 1992) and the Structured Clinical Interview for DSM-IV (SCID-CV; First et al., 1996) were used for AjD diagnosis. This semi-structured interview consists of two parts: 1) first, a list of 62 stressful events is presented and the patients have to indicate whether they have experienced it, 2) secondly, a list of 28 AjD symptoms is exposed and patients have to indicate the presence and severity of each symptom using a scale from 0 (Not at all) to 8 (Extremely severe). Additional items were also included in order to make an adequate differential diagnosis with post-traumatic stress disorder and generalized anxiety disorder, and to rule out that the symptoms were not caused by a medical condition or substance use.

2.3.2 Homework assignment component efficacy measures

A pre-post assessment instrument developed by our research group was used to evaluate different clinical variables before and after each homework session. Patients in the TEO condition responded to this assessment online, while participants in the traditional condition used paper and pencil.

First, before starting the homework session, mood was assessed using a 7-point Likert scale with different faces representing different degrees of sadness, with 0 being the maximum level of sadness. Self-efficacy to cope with the problem was also measured using a scale from 0 (“Not at all capable”) to 10 (“Completely capable”). Once the homework session was over, participants again answered an assessment with the same variables (mood and self-efficacy) and also answered a question about the extent to which they had perceived a change in these variables. A Likert scale ranging from 1 (“Much worse/much less capable”) to 7 (“Much better/much more capable”) was used to assess this perceived change.

In addition, patients’ emotions were measured both before and after the homework session. Specifically, the negative emotions (sadness, anxiety and anger) and positive emotions (joy and relaxation) that the patient experienced in relation to the stressful event were evaluated using a scale from 1 (“Not at all”) to 7 (“Completely”).

2.3.3 Opinion about the homework assignment component measures

A Homework Component Preference Questionnaire developed by our research group was used to assess patients’ opinion of the homework format. This instrument was administered before starting treatment and once the therapist had provided patients with an explanation of the different formats of homework assignment (Traditional vs. TEO). It included the following questions: Which homework modality would you choose if you could? Which one do you think might be more effective in helping you overcome the problem? Which one do you think might be more aversive? What type of modality would you recommend to a friend who has the same problem as you? In the present study only the item about the preference about the homework component will be analyzed.

In order to analyze patients’ preferences regarding the homework format after the end of treatment, a Homework Component Choice Questionnaire developed by our research group was used. Once the therapist reminded the patients of the function and the different aspects of each homework administration format, they were shown the format they had not received during treatment (Traditional vs. TEO). Specifically, patients in the Traditional group were shown the session 3 using the TEO system and patients who had used the TEO system during the intervention were shown the reading manual for the same session. Once both groups knew both homework formats, they were asked the following question, “If you could have chosen between the two ways of doing homework, which one would you have chosen?"

Finally, at the end of the homework session, the degree of homework acceptance was measured, including satisfaction and perceived usefulness of the session in both homework formats (Traditional vs. TEO). For this purpose, patients were asked to answer the following questions using a scale from 0 to 10: To what extent did you like today’s activity? and To what extent do you think today’s session was useful or beneficial to you?

2.4 Intervention

The CBT protocol by Botella et al. (2008) was adapted for this study. The duration of the intervention was 6-8 sessions of 90 min that took place weekly. The aim of the treatment was: 1) to teach patients strategies to overcome and cope with the stressful event and its consequences and 2) to promote their coping skills and learning from the event. For a detailed explanation of the treatment components see Quero et al. (2019b).

In addition to these components, this protocol also includes a homework component to put into practice what was learned during the therapy sessions. The content of the homework for each session is described in Quero et al. (2019b).

In this study the homework was assigned using two different formats (Traditional vs. TEO). The Traditional condition included the reading of summary manuals on the therapeutic content, as well as the use of self-records, written exercises and audio recordings of the sessions to practice the elaboration/exposure from home. The TEO condition used a digital support system to manage homework assignments between treatment sessions. This open internet-based technology is divided into two different platforms: the therapist’s and the patient’s one.

The therapist platform allows administering the patients and creating customized materials for them. For this purpose, the program contains multimedia content that can be used to create homework sessions (e.g., text, images, audio, video). Additionally, the TEO System allows to assess different clinical variables both before and after each homework session (e.g., mood state, self-efficacy or positive and negative emotions). Furthermore, it includes reports of each patient’s activity (e.g., assessment, content, assigned sessions, completed sessions, dates) that can download in order to monitor each patient.

The patient platform allows to receive the homework session sent by the therapist. Once the homework has been assigned, the patient answers a pre-session assessment and chooses a virtual environment in which he/she would like to perform the homework (a forest or a beach). This virtual environment also appears at the end of each session so that the patient can use it as a space for reflection on what was learned in the session. Finally, the patient answers a post-session assessment. The task can be repeated as many times as the user wishes.

For this study a homework session component was designed using audios, videos, and images that were available in the TEO system database. Specifically, the TEO system allows access to all the material available in EMMA’s world (symbols, audios, and images) so that patients could continue practicing the elaboration of the event at home. More detailed information about the TEO system can be found in Quero et al., 2011; Quero et al., 2019b.

2.5 Procedure

Participants who were interested in the study were screened for eligibility and, after receiving information about the project, signed the informed consent form. Additionally, two assessment sessions were conducted to confirm the diagnosis of AjD and to establish the therapeutic goals. Then, an independent researcher was responsible for randomly assigning participants to one of the conditions (Traditional vs. TEO). A computer-generated list created by Random Allocation Sofware 1.0 was used for this purpose. Once the participants had been assigned to each group, they received the same CBT protocol for AjD. The only difference between the groups was the homework administration format. In the traditional condition, the manualized protocol mentioned above was followed and homework included different materials (audios, readings, writing materials...). In contrast, in the TEO condition participants completed the homework through the online TEO system.

All seven therapists who took part in the study had a PhD or a Master’s degree in Clinical Psychology. All were trained in the use of VR and CBT techniques for the treatment of emotional disorders. In addition, senior clinicians trained and supervised PhDs weekly during the implementation of this treatment protocol.

2.6 Data analysis

First, group differences in the demographic data at pre-treatment were analyzed with chi-squared tests for categorical variables and t-tests for independent samples for the age variable. In addition, clinical variables related to the homework assignment component were explored to rule out differences between the experimental conditions at pre-treatment using t-test statistics. Second, in order to test the homework components’ efficacy, a linear mixed model for each outcome measure was implemented, with group (Traditional vs. TEO) as between-group factor and time (session 1, session 2, session 3, session 4, session 5, and session 6) and pre-post (before and after each homework session) as within-group factors. When the sphericity assumption was not met, the Greenhouse-Geisser correction was applied on the F statistic. Participant’s' perception of change was also analysed. Effect sizes (Cohen’s d) were calculated for within- and between-group changes. Following Cohen’s (1988) conventions, proportions of variance accounted for of about 0.10 and 0.25 were considered of medium and large magnitude, respectively. Moreover, to specifically analyze the change in the scores on each measure included in the study, the ANOVAs were followed by Sidak’s corrected post-hoc tests. Third, chi-squared tests were performed with the objective of examining the participants’ preferences and opinions about the way of applying the homework assignment component, before and after completing the treatment protocol. Furthermore, for the measures included in the study (i.e., mood state, self-efficacy, and positive and negative emotions) an ANCOVA was performed to control for the possible differences between the two experimental conditions in the age at pre-treatment. Finally, satisfaction and usefulness variables were analyzed using mixed ANOVAS with group (Traditional versus TEO) as between-group factor and time (session 1, session 2, session 3, session 4, session 5, and session 6) as within-group factor. All statistical analyses were performed using the statistical programme IBM SPSS Statistics 25.0 and following the current SPIRIT (Chan et al., 2013a; Chan et al., 2013b) and CONSORT (Moher et al., 2010) guideline recommendations.

3 Results

3.1 Baseline data and participants’ characteristics

Table 1 shows participants’ sociodemographic data for each group and for the total sample. First, results indicated that there were no significant differences between the experimental groups before treatment on any of the sociodemographic variables, except marital status (X² (2) = 8,013; p = 0,018). Furthermore, no significant differences were found between the two experimental groups on any of the homework component efficacy measures (self-efficacy, mood state, and positive and negative emotions) before the treatment either. Table 2 shows the means and standard deviations obtained for these clinical variables in each experimental condition and in the total sample at pre-treatment.

TABLE 2

TABLE 2. Means and standard deviations obtained for mood, self-efficacy, and emotional variables in the two experimental conditions on the pre-homework time in the first therapy session.

3.2 Differential effectiveness of homework assignment components in two experimental conditions (traditional vs. TEO) throughout the therapy sessions

Table 3 shows the means and standard deviations for the clinical variables in both experimental conditions from pre-to post-homework time. The results for each of the homework component efficacy measures are presented below.

TABLE 3

TABLE 3. Means and standard deviations obtained for mood, self-efficacy, and emotional variables in the two experimental conditions from pre-to post-homework time.

3.2.1 Mood state and self-efficacy to overcome the stressor

The mixed ANOVAs showed significant time effects for mood state (F (3.5, 144.8) = 20.030, p < 0.001) and for self-efficacy (F (3, 122.6) = 30.335, p < 0.001), with large effect sizes (Eta²Partial = 0.33 for mood state and Eta²Partial = 0.42 for self-efficacy). Both experimental groups improved on these variables across the different sessions received. No statistically significant interaction effect (time x group) was found (mood state: F (1, 41) = 0.06, p = 0.799; Eta²Partial = 0.00; self-efficacy: F (1, 41) = 0.65, p = 0.426; Eta²Partial = 0.02). Sidak’s post-hoc tests indicated that, compared to the first homework session, a significant change in mood state was produced from the third homework session, whereas for self-efficacy to overcome the stressful event, the improvement occurred from the fourth homework session. Mean values obtained for these variables before and after each homework session are shown in Figures 1, 2. Regarding the change in these variables from pre to post homework session, as Figures 1, 2 show, participants in both groups improve significantly after practicing each homework session assigned by the therapist (mood state (F (1.41) = 25.64, p < 0.001); self-efficacy (F (1.41) = 56,01, p < 0.001), with large effect sizes (Eta²\Partial = 0.38 for mood state and Eta²Partial = 0.58 for self-efficacy). Finally, the interaction effect for time x pre-post session did not reach statistical significance, and so no specific homework session produced more change than the others (mood state: F (5, 205) = 0.98, p = 0.431; Eta²Partial = 0.02; self-efficacy: F (5, 205) = 1.65, p = 0.148; Eta² Partial = 0.04).

FIGURE 1

FIGURE 1. Mood means for both conditions assessed before and after each homework assignment session.* Significant change compared to the first session.

FIGURE 2

FIGURE 2. Self-efficacy means for both conditions assessed before and after each homework assignment session.* Significant change compared to the first session.

In addition, to assessing the variables mood state and self-efficacy before and after each homework session, it was also considered important to analyze to what extent the participants perceived the change experienced in these variables after performing each session. The mixed ANOVAs showed significant time effects for mood state (F (5,205) = 8.92, p < 0.001; Eta²Partial = 0.18) with an effect size of medium magnitude. That is, participants perceived that their mood state changed favorably after doing homework throughout the different sessions. Sidak’s post-hoc tests indicated that the change in mood state was statistically significant from session 3 onward. Regarding the interaction effect, this was not significant, so both experimental conditions managed to change participants’ mood state (F (5, 205) = 0.40, p = 0.815; Eta²Partial = 0.01). Figure 3A shows the progress of mood state change after performing each of the homework sessions.

FIGURE 3

FIGURE 3. Change in mood state (A) and self-efficacy (B) assessed after performing each session.

Regarding the change experienced in the perceived self-efficacy to face the problem after completing the assigned homework, the mixed ANOVA showed significant time effects (F (5, 205) = 15.45, p < 0.001; Eta² Partial = 0.27). That is, participants perceived that their self-efficacy to face the problem changed favorably throughout the homework sessions. Sidak’s post-hoc tests indicated that it was from session two that the change was statistically significant. Thereafter, participants continued to improve their scores, reaching statistical significance also for the rest of the sessions. Figure 3B shows the progress of self-efficacy change after performing each of the homework sessions. Regarding the sessionxcondition interaction, a non-statistically significant result was found (F (5, 205) = 0.27, p = 0.926; Eta²Partial = 0.01).

3.2.2 Positive and negative emotions

In order to simplify the data presentation, means were calculated for positive (relax and joy) and negative (sadness, anger and anxiety) emotions, obtaining two final mean scores. Previously, the different types of emotions were found to correlate with each other in order to confirm the adequacy of combining them.

ANOVA results showed a significant time effect for positive (F (3.7, 152.7) = 16.91, p < 0.001) and negative emotions (F (3.4, 136.3) = 21.59, p < 0.001), with large effect sizes (Eta²Partial = 0.292 for positive emotions and 0.351 for negative emotions). Sidak’s post-hoc tests indicated that the change in the emotions was statistically significant, compared to the first session, from the fourth homework session onwards for positive emotions, and from the third homework session for negative emotions. Participants in both conditions increased their positive emotions and decreased their negative emotions once the homework assignment component had been completed. Mean values obtained for these variables before and after each homework session are shown in Figures 4, 5. Again, no interaction effect (time x group) was found (positive emotions: F (1, 41) = 0.002, p = 0.968, Eta²Partial = 0; negative emotions: F (1, 40) = 0.04, p = 0.849, Eta²Partial = 0.001). As for the change in these variables from pre to post homework session, this effect was significant for both positive (F (1, 41) = 19.32, p < 0.001) and negative emotions (F (1,40) = 34.71, p < 0.001), with large effect sizes (Eta²Partial = 0.320 for positive emotions and 0.465 for negative emotions). Overall, an increase in positive emotions and a decrease in negative emotions occurred after each homework session (see Figures 4, 5). Finally, the interaction effect for time x pre-post session did not reach statistical significance for positive emotions, and so no specific homework session produced more change in emotions than the others (F (4.2, 173.8) = 1.79, p = 0.129, Eta²Partial = 0.042). However, with negative emotions a statistically significant result was found for this interaction effect (F (3.1, 124.2) = 2.75, p = 0.044, Eta²Partial = 0.064), although with a small proportion of variance explained.

FIGURE 4

FIGURE 4. Positive emotion for both conditions assessed before and after each homework assignment session. *Significant change compared to the first session.

FIGURE 5

FIGURE 5. Negative emotion for both conditions assessed before and after each homework assignment session. *Significant change compared to the first session.

3.2.3 Preferences and opinions about homework assignment components (TEO vs. traditional)

Results revealed significant differences between the experimental conditions in the preference about the way the homework session component is administered. Before starting the treatment, and after receiving a brief explanation about each condition, most participants (71%) preferred the TEO condition (χ² = 4,874; p < 0.05).

Regarding the participants’ opinion once they completed the homework assignment component, the majority (70.5%) again revealed their preference for the TEO system for practicing the homework assignment component, with significant differences in choosing this type of homework (χ² = 5,350; p = 0.021). Furthermore, more than half of the participants in the traditional condition (54.54%) reported that they would have preferred the TEO condition.

Lastly, results showed a significant effect of session on satisfaction (F (3.8, 156.8) = 21,01, p < 0.001) and usefulness (F (3.4, 138) = 24.68 p < 0.001; Eta²Partial = 0.376), variables with large effect sizes (Eta²Partial = 0.339 for satisfaction and 0.376 for usefulness). Participants were satisfied with all the homework sessions, and they considered them increasingly useful. No significant differences were found between groups (satisfaction: F (1, 41) = 3.31, p = 0.076, Eta²Partial = 0.075; usefulness: F (1, 41) = 0.39, p = 0.537, Eta²Partial = 0.009) Therefore, both ways of applying the homework assignments were positively assessed (see Figures 6A,B).

FIGURE 6

FIGURE 6. Satisfaction (A) and usefulness (B) of each homework assignment session.

4 Discussion

The aim of this study was to present session-by-session efficacy data on the homework assignment component and patients’ preferences and opinions about this component in Traditional vs. TEO format. The findings suggest that: 1) both treatment formats were associated with improvements in the patients’ mood and self-efficacy and in their perception of improvement, 2) both conditions were associated with a decrease in negative affect and an increase in positive affect and, 3) both obtained good levels of satisfaction and usefulness, however, most participants preferred the TEO system condition.

Homework assignments administered both traditionally and with the TEO system produced significant improvements in AjD patients’ mood and self-efficacy. This is consistent with what has been observed in several meta-analyses suggesting the existence of a relationship between homework compliance and therapy outcomes (Kazantzis et al., 2010; Mausbach et al., 2010). Thus, it is proposed that treatments that include homework lead to greater treatment effects than those that do not. Experimental studies such as Stirman et al. (2018) or Callan et al. (2020) found this same relationship between homework completion and greater symptom change.

It was observed that the significant change in mood state was produced from the third homework session, while for self-efficacy to overcome the stressful event, the improvement occurred from the fourth homework session. This could be explained considering the content of the homework sessions. Thus, in session three, the acceptance of the event and its consequences was worked on in depth. Meanwhile, in session four the content focused on improving coping and developing the patients’ psychological strengths. Regarding the specific contribution of every homework session none of them produced more improvement than the others. As for the participants’ perception of change, they perceived that their mood state and self-efficacy improved after doing homework across the different sessions. Specifically, change in mood state was statistically significant from session 3 and change in self-efficacy started from session 2. That is, patients began to perceive an improvement in their sense of self-efficacy once they had been introduced to the exposure/elaboration technique and had to perform the stressful event acceptance task for the first time (session 2). On the other hand, the significant perception of improvement in the mood took place once the acceptance phase of the stressful event was completed (session 3).

In addition, an increase in positive emotions and a decrease in negative emotions occurred across treatment for both conditions, being significant from session 4 onwards. This change coincides with the homework session in which patients practice coping with the event, psychoeducation of emotions and a positive psychology exercise that aims to develop psychological strengths. Again, no specific homework session produced more change in emotions than the others.

Regarding the opinions and preferences of the participants, most of them preferred the TEO condition both before starting the treatment and after its completion. Even more than half of the patients in the traditional condition stated that they would have preferred the TEO format. There was no difference between the Traditional vs. TEO condition in terms of satisfaction. Participants were satisfied with the treatment sessions and rated them as useful. Furthermore, satisfaction and perceived usefulness increased over the course of the treatment. These results support the idea that complete homework digitally is preferred by patients. This finding is similar to that of Callan et al. (2020), in which the use of a mobile application to manage homework obtained good satisfaction results. One possible explanation is that the participants in this study constituted a young sample (mean age 29.98 years), so it is possible that this population is more accustomed to the use of technologies in their daily lives and this may have influenced their preferences when receiving therapy. In addition, during the development of TEO, we tried to adapt the content of the homework as much as possible to facilitate navigation, use and interactivity and to make the tasks enjoyable and easy to follow by using multimedia content. Despite these results, it is worth mentioning that the traditional format of assigning homework also was well-valued and therefore could be a suitable option for those patients who prefer not to use technologies during the intervention. Some authors suggest that the use of this type of technology could improve adherence to homework by making it more attractive (Michelle et al., 2014; Quero et al., 2019b). However, the specific attrition rate for the homework component was not analysed in this study. Future studies should test if a system like TEO could reduce dropouts and increase adherence to homework and to treatment.

It is worth mentioning that the present study has some limitations. The first one refers to the small sample size, which does not allow us to assure enough statistical power. Secondly, most of the sample were females and had a university education, which limits the generalization of the results. In addition, given that all participants agreed to participate in research that offered a VR-supported CBT treatment, it is possible that there was a prior preference for formats using ICTs over the general population. Therefore, again, the results cannot be necessarily generalised to other populations. Lastly, it should be noted that the Diagnostic Interview for AjD used in this study is still in process of validation.

In sum, both formats of homework assignment (Traditional vs. TEO) produced improvements in the patients’ outcomes and were positively assessed. However, the TEO system was preferred by the majority of AjD patients. It is important to take this finding into account given that these patients often perceive tasks related to the stressful event as too aversive (Stirman et al., 2018).

It is suggested that making use of technologies such as the Internet for homework assignment could increase patients’ motivation and facilitate symptoms change (Stirman et al., 2018). However, more research is needed to analyze the usefulness of new ways of assigning homework and their effects on patients’ engagement and adherence with both homework and the whole treatment.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Universitat Jaume I. The patients/participants provided their written informed consent to participate in this study.

Author contributions

SQ and CB contributed to conception and design of the study. MM, CT, and DC organized the database. AD-G and MM performed the statistical analysis. SF-B and AD-G wrote the first draft of the manuscript. SQ, SF-B, and CT wrote sections of the manuscript. CB, PC, and SQ contributed to manuscript revision. All the authors read, and approved the submitted version.

Funding

This research was supported by Spanish Ministerio de Ciencia, Innovación y Universidades (Programa Estatal I+D+i RTI 2018-100993-B-100), 2021 Research Promotion Plan at Universitat Jaume I (UJI-B2021-47), CIBEROBN, an initiative of the ISCIII (ISC III CB06 03/0052).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

American Psychological Association (Apa), (2000). Diagnostic and statistical manual for mental disorders. 4th ed. Washington: APA.

Google Scholar

Andreu-Mateu, S., Botella, C., Quero, S., Guillen, V., and Baños, R. (2012). La utilización de la realidad virtual y estrategias de psicología positiva en el tratamiento de los trastornos adaptativos. Behav. Psychol. 20 (2), 323–348. Available at: http://repositori.uji.es/xmlui/bitstream/handle/10234/67199/55033.pdf?sequence=1&isAllowed=y.

Google Scholar

Bachem, R., and Casey, P. (2018). Adjustment disorder: A diagnosis whose time has come. J. Affect. Disord. 227, 243–253. doi:10.1016/j.jad.2017.10.034

PubMed Abstract | CrossRef Full Text | Google Scholar

Baños, R. M., Botella, C., Guillen, V., García-Palacios, A., Quero, S., Bretón-López, J., et al. (2009). An adaptive display to treat stress-related disorders: EMMA's world. Br. J. Guid. Couns. 37 (3), 347–356. doi:10.1080/03069880902957064

CrossRef Full Text | Google Scholar

Baños, R. M., Guillen, V., Quero, S., Garcia-Palacios, A., Alcaniz, M., and Botella, C. (2011). A virtual reality system for the treatment of stress-related disorders: A preliminary analysis of efficacy compared to a standard cognitive behavioral program. Int. J. Human-Computer Stud. 69 (9), 602–613. doi:10.1016/j.ijhcs.2011.06.002

CrossRef Full Text | Google Scholar

Beck, A. T. (1979). Cognitive therapy of depression. New York: Guilford Press.

Google Scholar

Benbow, A. A., and Anderson, P. L. (2019). A meta-analytic examination of attrition in virtual reality exposure therapy for anxiety disorders. J. anxiety Disord. 61, 18–26. doi:10.1016/j.janxdis.2018.06.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Botella, C., Baños, R. M., and Guillén, V. (2008). “Creciendo en la adversidad. Una propuesta de tratamiento para los trastornos adaptativos,” in Psicología positiva aplicada. Editor C. Vázquez (Bilbao: DDB).

Google Scholar

Bryant, M. J., Simons, A. D., and Thase, M. E. (1999). Therapist skill and patient variables in homework compliance: Controlling an uncontrolled variable in cognitive therapy outcome research. Cogn. Ther. Res. 23 (4), 381–399. doi:10.1023/A:1018703901116

CrossRef Full Text | Google Scholar

Burns, D. D., and Spangler, D. L. (2000). Does psychotherapy homework lead to improvements in depression in cognitive–behavioral therapy or does improvement lead to increased homework compliance? J. Consult. Clin. Psychol. 68 (1), 46–56. doi:10.1037/0022-006X.68.1.46

PubMed Abstract | CrossRef Full Text | Google Scholar

Busch, A. M., Uebelacker, L. A., Kalibatseva, Z., and Miller, I. W. (2010). Measuring homework completion in behavioral activation. Behav. Modif. 34 (4), 310–329. doi:10.1177/0145445510373384

PubMed Abstract | CrossRef Full Text | Google Scholar

Callan, J. A., Jacob, J. D., Siegle, G. J., Dey, A., Thase, M. E., Dabbs, A. D., et al. (2020). CBT MobileWork©: User-centered development and testing of a mobile mental health application for depression. Cogn. Ther. Res. 45, 287–302. doi:10.1007/s10608-020-10159-4

CrossRef Full Text | Google Scholar

Callan, J. A., Kazantzis, N., Park, S. Y., Moore, C. G., Thase, M. E., Minhajuddin, A., et al. (2019). A propensity score analysis of homework adherence-outcome relations in cognitive behavioral therapy for depression. Behav. Ther. 50 (2), 285–299. doi:10.1016/j.beth.2018.05.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Casey, P., Jabbar, F., O’Leary, E., and Doherty, A. M. (2015). Suicidal behaviours in adjustment disorder and depressive episode. J. Affect. Disord. 174, 441–446. doi:10.1016/j.jad.2014.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Chan, A. W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., et al. (2013b). SPIRIT 2013 statement: Defining standard protocol items for clinical trials. Ann. Intern. Med. 158 (3), 200–207. doi:10.7326/0003-4819-158-3-201302050-00583

PubMed Abstract | CrossRef Full Text | Google Scholar

Chan, A. W., Tetzlaff, J. M., Gøtzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., et al. (2013a). SPIRIT 2013 explanation and elaboration: Guidance for protocols of clinical trials. Bmj 346, e7586. doi:10.1136/bmj.e7586

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Erlbaum.

Google Scholar

Conklin, L. R., and Strunk, D. R. (2015). A session-to-session examination of homework engagement in cognitive therapy for depression: Do patients experience immediate benefits? Behav. Res. Ther. 72, 56–62. doi:10.1016/j.brat.2015.06.011

PubMed Abstract | CrossRef Full Text | Google Scholar

First, M. B., Spitzer, R. L., Gibbon, M., and Williams, J. B. W. (1996). Structured clinical interview for DSM-IV axis I disorders, clinician version (SCID-CV). Washington, DC: American Psychiatric Press.

Google Scholar

Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., et al. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol. Med. 47 (14), 2393–2400. doi:10.1017/S003329171700040X

PubMed Abstract | CrossRef Full Text | Google Scholar

Guillén, V., Baños, R. M., and Botella, C. (2018). Users’ opinion about a virtual reality system as an adjunct to psychological treatment for stress-related disorders: A quantitative and qualitative mixed-methods study. Front. Psychol. 9, 1038. doi:10.3389/fpsyg.2018.01038

PubMed Abstract | CrossRef Full Text | Google Scholar

Haller, E., and Watzke, B. (2020). The role of homework engagement, homework-related therapist behaviors, and their association with depressive symptoms in telephone-based CBT for depression. Cogn. Ther. Res. 45, 224–235. doi:10.1007/s10608-020-10136-x

CrossRef Full Text | Google Scholar

Kazantzis, N., Deane, F. P., and Ronan, K. R. (2000). Homework assignments in cognitive and behavioral therapy: A meta-analysis. Clin. Psychol. Sci. Pract. 7 (2), 189–202. doi:10.1093/clipsy.7.2.189

CrossRef Full Text | Google Scholar

Kazantzis, N., Whittington, C., and Dattilio, F. (2010). Meta-analysis of homework effects in cognitive and behavioral therapy: A replication and extension. Clin. Psychol. Sci. Pract. 17 (2), 144–156. doi:10.1111/j.1468-2850.2010.01204.x

CrossRef Full Text | Google Scholar

Kazlauskas, E., Zelviene, P., Lorenz, L., Quero, S., and Maercker, A. (2017). A scoping review of ICD-11 adjustment disorder research. Eur. J. Psychotraumatology 8, 1421819. doi:10.1080/20008198.2017.1421819

PubMed Abstract | CrossRef Full Text | Google Scholar

Maercker, A., Forstmeier, S., Pielmaier, L., Spangenberg, L., Brähler, E., and Glaesmer, H. (2012). Adjustment disorders: Prevalence in a representative nationwide survey in Germany. Soc. Psychiatry Psychiatr. Epidemiol. 47 (11), 1745–1752. doi:10.1007/s00127-012-0493-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Mausbach, B. T., Moore, R., Roesch, S., Cardenas, V., and Patterson, T. L. (2010). The relationship between homework compliance and therapy outcomes: An updated meta-analysis. Cogn. Ther. Res. 34 (5), 429–438. doi:10.1007/s10608-010-9297-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Michelle, T. Q. Y., Jarzabek, S., and Wadhwa, B. (2014). “CBT assistant: MHealth app for psychotherapy,” in 2014 IEEE Global Humanitarian Technology Conference-South Asia Satellite (GHTC-SAS), September 2014 (IEEE), 135–140. doi:10.1109/GHTC-SAS.2014.6967572

CrossRef Full Text | Google Scholar

Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., et al. (2010). CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. Int. J. Surg. 10 (1), 28–55. doi:10.1016/j.ijsu.2011.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Neimeyer, R. A., and Feixas, G. (1990). The role of homework and skill acquisition in the outcome of group cognitive therapy for depression. Behav. Ther. 21 (3), 281–292. doi:10.1016/S0005-7894(05)80331-4

CrossRef Full Text | Google Scholar

O'Donnell, M. L., Metcalf, O., Watson, L., Phelps, A., and Varker, T. (2018). A systematic review of psychological and pharmacological treatments for adjustment disorder in adults. J. Trauma. stress 31 (3), 321–331. doi:10.1002/jts.22295

PubMed Abstract | CrossRef Full Text | Google Scholar

O’Donnell, M. L., Alkemade, N., Creamer, M., McFarlane, A., Silove, D., Bryant, R., et al. (2016). A longitudinal study of adjustment disorder after trauma exposure. Am. J. Psychiatry 173 (12), 1231–1238. doi:10.1176/appi.ajp.2016.16010071

PubMed Abstract | CrossRef Full Text | Google Scholar

Perkonigg, A., Lorenz, L., and Maercker, A. (2018). Prevalence and correlates of ICD-11 adjustment disorder: Findings from the zurich adjustment disorder study. Int. J. Clin. Health Psychol. 18 (3), 209–217. doi:10.1016/j.ijchp.2018.05.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Quero, S., Botella, C., Pérez Ara, M. Á., Andreu-Mateu, S., Baños, R., and Castilla, D. (2011). Teo: An online emotional therapy system to apply homework assignments in the treatment of adjustment disorders. J. Cybertherapy Rehabilitation 4 (2), 306–307. Available at: http://repositori.uji.es/xmlui/bitstream/handle/10234/73646/50077.pdf?sequence=1&isAllowed=y.

Google Scholar

Quero, S., Molés, M., Campos, D., Andreu-Mateu, S., Baños, R. M., and Botella, C. (2019a). An adaptive virtual reality system for the treatment of adjustment disorder and complicated grief: 1-year follow-up efficacy data. Clin. Psychol. Psychother. 26, 204–217. doi:10.1002/cpp.2342

PubMed Abstract | CrossRef Full Text | Google Scholar

Quero, S., Rachyla, I., Molés, M., Mor, S., Tur, C., Cuijpers, P., et al. (2019b). Can between-session homework be delivered digitally? A pilot randomized clinical trial of CBT for adjustment disorders. Int. J. Environ. Res. Public Health 1620, 3842. doi:10.3390/ijerph16203842

PubMed Abstract | CrossRef Full Text | Google Scholar

Rachyla, I. (2022). Development and validation of an online treatment program for adjustment disorders [Unpublished doctoral dissertation]. Castello de la Plana, Valencia, Spain: Universitat Jaume I.

Google Scholar

Reed, G. M., Correia, J. M., Esparza, P., Saxena, S., and Maj, M. (2011). The WPA-WHO global survey of psychiatrists’ attitudes towards mental disorders classification. World Psychiatry 10, 118–131. doi:10.1002/j.2051-5545.2011.tb00034.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Reger, G. M., Hoffman, J., Riggs, D., Rothbaum, B. O., Ruzek, J., Holloway, K. M., et al. (2013). The “PE coach” smartphone application: An innovative approach to improving implementation, fidelity, and homework adherence during prolonged exposure. Psychol. Serv. 10 (3), 342–349. doi:10.1037/a0032774

PubMed Abstract | CrossRef Full Text | Google Scholar

Stirman, S. W., Gutner, C. A., Suvak, M. K., Adler, A., Calloway, A., and Resick, P. (2018). Homework completion, patient characteristics, and symptom change in cognitive processing therapy for PTSD. Behav. Ther. 49 (5), 741–755. doi:10.1016/j.beth.2017.12.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Strickland, E. L. (2019). Mhealth progress monitoring and homework tracking as an adjunct to cognitive behavioral therapy treatment [doctoral dissertation]. California: The Wright Institute.An exploratory analysis of the smartphone application "kip

Google Scholar

World Health Organization (Who), (2018). International classification of Diseases for mortality and morbidity statistics. 11th ed. Geneva: WHO.

Google Scholar

World Health Organization (Who), (1992). The ICD-10 classification of mental and behavioural disorders: Clinical descriptions and diagnostic guidelines. Geneva: WHO.

Google Scholar

Zelviene, P., and Kazlauskas, E. (2018). Adjustment disorder: Current perspectives. Neuropsychiatr. Dis. Treat. 14, 375–381. doi:10.2147/NDT.S121072

PubMed Abstract | CrossRef Full Text | Google Scholar