Improving Acceptability and Uptake Behavior for Internet-Based Cognitive-Behavioral Therapy

Internet-based cognitive behavioral therapy (iCBT) programs have the potential to improve access to mental healthcare, but they are not viewed as acceptable nor widely utilized by the general public. This study tested whether two acceptance-facilitating interventions improved acceptability and uptake-related behavior for therapist assisted and self-guided iCBT. Participants were randomly assigned to read a treatment rationale for iCBT (vs. a brief definition) and to receive a small financial incentive (or not) for seeking more information about evidence-based iCBT programs. Participants (N = 662) were a diverse group recruited from a University participant pool and the surrounding community. Participants completed standardized measures of attitudes toward and outcome expectancy for iCBT and a single question about willingness to use it and were given the opportunity to get information about accessing evidence-based iCBT programs. A series of MANCOVAs showed small, positive effects of the treatment rationale on attitudes and outcome expectancy for both self-guided and therapist-assisted iCBT, but not for willingness to use it. A hierarchical logistic regression model found no effect of the treatment rationale or financial incentive on whether participants sought additional information about how to access iCBT, although psychopathology symptoms and identifying as White or multiracial were positively associated with information-seeking. Inconsistent with past research, participants rated therapist-assisted and self-guided iCBT as equally acceptable. Participants recruited from the community reported greater willingness to use iCBT than University students. These results underscore the urgent need for further research toward improving the acceptability and uptake of iCBT so that it may better fulfill its potential to fill the gap in unmet mental health need.

Introduction

Internet-based cognitive behavioral therapy (iCBT) programs are cognitive behavioral interventions that treat psychological problems via digital platforms. Internet-based cognitive behavioral therapy programs have been shown to reduce symptoms across a range of mental disorders, including post-traumatic stress disorder (1), social anxiety disorder (2), and panic disorder (3), among others. Internet-based cognitive behavioral therapy creates an opportunity to disseminate treatment to people who cannot access face-to-face therapy, as over half of the global population has access to the Internet (4). Additionally, iCBT programs maintain fidelity with treatment protocols in a way that face-to-face treatment delivery in community settings may not (5). Given the insufficient number of licensed mental healthcare providers in the U.S., particularly in areas like rural communities (6), iCBT represents an opportunity to substantially increase access to evidence-based treatment delivered as intended.

Internet-based cognitive behavioral therapy can include support from a therapist or be delivered in a self-guided format. Therapist-assisted iCBT is thought to increase client adherence and reduce attrition (7). An obvious advantage of self-guided iCBT is that a person does not need to find a therapist to access mental healthcare, but a trade-off is that people using self-guided formats may not engage long enough to benefit as much (or at all). One meta-analysis of 12 randomized controlled trials for iCBT for depression and anxiety found that whereas therapist-assisted iCBT programs demonstrated large effect sizes for treatment outcomes, iCBT programs without therapist guidance or support showed small to moderate effects (8). Overall, people benefit from iCBT when paired with therapist assistance or used alone, although the magnitude of effect is likely higher for programs with therapist assistance (9).

Despite its efficacy, iCBT is widely underutilized by the general public (10–12), perhaps because they do not view it as an acceptable form of mental health treatment. Research in this area has defined and operationalized the concept of “acceptability” for digital mental health interventions in a variety of ways. Overlapping constructs like satisfaction, feasibility, and usability are used interchangeably with acceptability (13). Operational definitions include single Likert scale items that assess participants' willingness to use iCBT (14, 15), longer questionnaires designed for individual studies (16), and one psychometrically validated questionnaire that assesses attitudes toward psychological interventions that are delivered online (17). Studies have also operationalized acceptability using validated self-report measures for other constructs, like outcome expectancy—the expectation that one will benefit from treatment (18). The lack of precision in the conceptualization and measurement of the acceptability of iCBT may explain why estimates of the acceptability of iCBT vary widely across research studies.

People who use either self-guided or therapist-assisted iCBT report a high degree of user satisfaction (19–21). However, large survey studies have found that most people are unfamiliar with digital mental health interventions such as iCBT (14) and that people prefer other forms of treatment over Internet-based therapy (22). Therapist-assisted iCBT programs are generally rated as more acceptable than self-guided programs (23, 24), but one survey study found that only 16% of non-treatment-seeking adults would consider using therapist-assisted iCBT to address a mental health concern (16). The significant contrast between high user satisfaction in treatment studies and low acceptability in the general population may be due to the “denominator problem” (25). This refers to a bias that can result when a large number of people are invited to participate in a treatment study, but only the small proportion of those who are motivated and interested volunteer and enroll. However, even large survey studies that recruit potentially biased samples, such as people seeking treatment on mental health clinic websites, have found low acceptability for iCBT (26, 27). This points to a clear need for strategies to increase iCBT's appeal to potential users.

Treatment rationales, which describe how specific therapy interventions work, have long been shown to improve outcome expectancy for face-to-face psychotherapy (28). A handful of studies have incorporated treatment rationales for digital mental health interventions into video or text-based materials designed to improve acceptability and related constructs. Studies generally find that these acceptability-facilitating interventions improve acceptability and intention to use digital mental health programs (23, 24, 29, 30) but not all (31). One limitation to this literature is that most studies used samples that were small or that may not be representative of the general population: Mitchell and Gordon (24) studied a small (N = 20) sample of undergraduate students, Ebert et al. (29) studied primary care patients, and Soucy et al. (30) recruited participants who had already demonstrated an interest in using iCBT. Only one study has examined the impact of an intervention to improve acceptability of both self-guided and therapist-assisted programs (23).

No study to date has examined the effect of treatment rationales and related strategies on behaviors related to the actual uptake of iCBT. A few studies have examined whether financial incentives (e.g., vouchers, nominal cash payments, or raffles) improve adherence to mental health treatment (32–34), but none have examined their impact on behaviors signaling a willingness to try iCBT. This leaves a notable gap in the literature regarding the potential benefit of providing a small monetary incentive to increase behaviors related to the uptake of iCBT.

The current experimental study examined the effect of a treatment rationale on self-reported acceptability and uptake-related behavior for iCBT among a non-treatment-seeking sample. Acceptability was defined as a set of cognitively based, positive attitudes toward these interventions (17). Given the wide variability in the ways that acceptability has previously been measured, three separate measures were drawn from the literature and analyzed together to measure this construct. The study also examined the effect of a financial incentive ($25 raffle) on seeking information about how to access iCBT programs. Given past research, the authors hypothesized the following: (1) a treatment rationale would increase acceptability for both therapist-assisted and self-guided iCBT, (2) participants would report higher acceptability for therapist-assisted iCBT as compared to self-guided iCBT, and (3) a treatment rationale and a financial incentive would increase behaviors related to the uptake of iCBT.

Materials and Methods

Participants

Participants were recruited from a large southeastern University in an urban setting and canvassed from public areas in the surrounding metropolitan area. University student participants (N = 403) were recruited online from a University-based research participant pool for psychology course credit. Community participants (N = 346) were recruited from public spaces and given the opportunity to enter a raffle with a 1 in 30 chance of winning a $25 gift card as compensation. To be included in the study, participants had to be aged 18 or over and literate in English.

Of the 749 individuals who expressed interest in the study, six respondents were excluded due to failure to meet inclusion criteria. Of the remaining participants (N = 743), 81 respondents were excluded from analyses because they took <5 min on the survey or failed the study's manipulation check (11%). In all, data from 662 participants (University N = 365; Community N = 297) were included for data analysis. Demographic data for these participants are presented in Table 1.

TABLE 1

Table 1. Participant characteristics.

Procedure

All study procedures were completed using Qualtrics, a survey-creation platform and secure hosting server. University student participants completed the study on their own personal web-enabled devices. Community members completed the study on a tablet computer (i.e., iPad) provided by a research assistant or received an email with instructions to complete the study online.

All participants were assigned a study identification number and completed informed consent procedures prior to starting the study. Upon enrollment, participants were immediately randomized to receive a treatment rationale for iCBT (or a brief definition of iCBT) and a financial incentive to seek information about how to access evidence-based iCBT programs (or none) in a 2 × 2 experimental design. Experimenters were blinded to study condition. Participants first completed questionnaires assessing demographic information and symptoms of depression, anxiety, and stress. Next, depending on experimental condition, participants received a treatment rationale for iCBT or a brief definition of self-guided and therapist-assisted iCBT. Participants then answered questions about their history using and familiarity with online mental health interventions and completed measures of acceptability for self-guided and therapist-assisted iCBT. After completing these measures, participants were informed that they would receive an email within 24 h with a link to access and download iCBT programs, if interested. This link connected participants to a brief online survey in which they could select iCBT programs with empirical support from randomized clinical trials and receive information about how to access them. Depending on experimental condition, participants were also told they would receive a small financial incentive for completing this survey, or not.

Experimental Conditions

Treatment Rationale

Participants assigned to the treatment rationale condition read an in-depth description of iCBT, including rates of usage, research basis, and accessibility. The rationale used persuasion techniques that have been proposed to increase outcome expectancy for psychotherapy, including an authoritative speaker (a University professor and licensed clinical psychologist) and emphasis on empirical support (35). The rationale ended with a “frequently asked questions” section that specifically addressed the most commonly perceived advantages and disadvantages of therapist-assisted iCBT (16). The treatment rationale was ~800 words in length. As a manipulation check, participants who received the treatment rationale then answered three true or false questions about iCBT (see Appendix A.1 for full details).

Participants assigned to the brief definition condition did not receive the treatment rationale. Instead, these participants read a one-paragraph definition of iCBT, which described the difference between self-guided and therapist-assisted iCBT, so that they would have enough information to answer questions assessing their attitudes about these two modalities (see Appendix A.2). The brief definition of iCBT was 130 words in length.

Financial Incentive

Participants in the financial incentive condition were offered entry into a raffle with a 1 in 30 chance to a win a $25 e-gift card for completing a survey that included a list of iCBT programs with empirical support from randomized clinical trials about which they would receive information about how to access and download, if interested. Participants assigned to the no financial incentive condition were not offered a financial incentive to complete the survey.

Measures

Demographics and History of Psychotherapy

A 22-item demographics questionnaire was developed for the current study using items from the Standardized Data Set from the Center for Collegiate Mental Health at Penn State University (36). In addition, past and current experience using both face-to-face and Internet-based mental health services was measured using a series of Likert-type self-report items developed for the study (e.g., “Have you ever received face-to-face psychotherapy or counseling?”, “If so, how helpful were these services”).

Acceptability of iCBT

Attitudes Toward Psychological Online Interventions Scale (APOI)

The APOI is a 16-item validated measure of general attitudes toward online psychological interventions (17). Although many questionnaires have been developed to evaluate acceptability toward Internet-based mental health programs, the APOI is the only psychometrically validated questionnaire to specifically examine this construct. Accordingly, it was selected for the current study over other non-validated questionnaires. Although not indicated in original paper (17), positively valenced items were reverse-coded (Schröder, personal communication, February 12, 2020). Total scores range from 16 to 80 with higher scores indicating more positive attitudes toward iCBT. The APOI demonstrated strong overall internal consistency (α = 0.77) in a sample of 1,013 participants (17) and demonstrated good internal consistency in the present sample for both self-guided iCBT (α = 0.83) and therapist-assisted iCBT (α = 0.82).

Credibility/Expectancy Questionnaire (CEQ)

The expectancy subscale of the CEQ (37) consists of 3 items assessing expectations about efficacy for psychological treatments (0–100%), with higher scores indicating higher expectancy of efficacy. It was included in the current study because of its previous use as a measure of iCBT acceptability (18) and to evaluate the effect of outcome expectancy persuasion techniques that were included in the treatment rationale. The CEQ has demonstrated high internal consistency for the overall scale (α = 0.84–0.85), fair to excellent internal consistency for the expectancy subscale (α = 0.79–0.9), and good test-retest reliability [r = 0.83; (37)]. The internal consistency of the expectancy subscale in the present sample was excellent for both self-guided iCBT (α = 0.91) and therapist-assisted iCBT (α = 0.90).

Single Item

A single Likert scale item assessing willingness to use iCBT, “Would you use a (self-guided/therapist-assisted) iCBT program to improve your life (e.g., reduce stress, anxiety, depression)?” was used as a measure of acceptability based on use of similar items in past research (38). Response choices were scored on a 5-point Likert scale and comprised the following: “definitely would use,” “would likely use,” “unsure,” “unlikely to use,” and “definitely would not use,” with higher scores indicating greater willingness to use iCBT.

Psychopathology

Depression, Anxiety, and Stress Scale–21 Item (DASS-21)

The DASS-21 is a 21-item validated measure of mental illness symptoms that yields three subscales: depression, anxiety, and stress (39). Scores for the total DASS-21 scale range between 0 and 126, with higher scores indicating more distress or impairment. The DASS-21 demonstrates strong convergent validity with both the Beck Anxiety Inventory (BAI; r = 0.81) and Beck Depression Inventory (BDI; r = 0.74) indicating satisfactory ability to discriminate between both anxiety and depressive symptoms (40). The DASS-21 demonstrated excellent internal consistency in the present sample (α = 0.90).

Uptake Behavior for iCBT

Participants were classified as having engaged in behavior related to the uptake of ICBT (or not) if they completed the survey that included a list of iCBT programs about which they would receive information about how to access and download (or not).

Statistical Analyses

Acceptability of iCBT

Age and psychopathology were included as covariates in all models examining acceptability of iCBT, given their association with interest in Internet-based behavioral and psychological treatment (22, 26). A two-way MANCOVA was used to evaluate the effects of rationale condition and recruitment source (community, University) on acceptability of self-guided and therapist-assisted iCBT. A two-way mixed-design MANCOVA was used to evaluate differences in acceptability between self-guided and therapist-assisted iCBT. The three dependent variables within each MANCOVA model included general attitudes (as measured by the APOI), outcome expectancy (as measured by the expectancy subscale of the CEQ), and a single item assessing willingness to use iCBT. For each model, recruitment source was included as an independent variable to test for a two-way interaction. In the absence of an interaction, recruitment source was collapsed and main effects were interpreted across all participants. Listwise deletion was used for participants with missing data, which created variation in sample sizes across models.

iCBT Uptake Behavior

A two-step hierarchical logistic regression was performed to test the hypothesis that a treatment rationale (vs. brief definition of iCBT) and a financial incentive (vs. none) would improve participants' likelihood of seeking out information about how to access and download iCBT programs. Uptake behavior was classified as a binary dependent variable (yes vs. no). In step one, four variables were entered to control for participant characteristics that have previously been shown to relate to uptake of iCBT or use of other mental health services: age, psychopathology (DASS-21 total score), gender, and race/ethnicity. Age was included due to evidence that older age is negatively related to use of health-related technologies (41). Psychopathology was included to account for current need for mental health treatment. Race was included due to research showing that African Americans are less likely to initiate iCBT treatment than Whites (42) and U.S. national data demonstrating that White and multiracial individuals seek mental health treatment at higher rates than other racial groups (43). Accordingly, this variable was dummy coded to compare racial identities associated with higher and lower levels of mental health service utilization (White, multiracial vs. Black/African-American, Hispanic/Latinx, Asian). Gender was included due to U.S. national data demonstrating that women seek mental health treatment at higher rates than men (43) and was dummy coded to compare men and women. In step two of the model, treatment rationale and financial incentive conditions were entered to assess the influence of these experimental manipulations while controlling for participant characteristics. All analyses were conducted using Statistical Package for the Social Sciences (SPSS) version 25.0.

Results

Table 2 shows descriptive statistics and intercorrelations for key variables. Whereas 37.5% of participants reported a history of face-to-face psychotherapy, only 2.1% of participants reported using an online mental health program. Responses to the Depression, Anxiety, and Stress Scale–21 indicated that, on average, participants did not endorse severe levels of psychopathology (M = 31.90, SD = 21.80) based on the suggested cutoff of 60 for severe mental illness (39). However, many participants met or exceeded clinical cutoffs suggested by Lovibond and Lovibond (39) for mild depression (Cutoff: 10; N = 285, 43.1%), mild anxiety (Cutoff: 8; N = 335, 50.6%), or mild stress (Cutoff: 15; N = 220, 33.2%), with a total of 411 participants (60.2%) meeting the cutoff for mild symptoms on at least one of these three subscales.

TABLE 2

Table 2. Means, standard deviations, and correlations between acceptability of iCBT and indicators of mental health symptomatology.

Acceptability of iCBT

Assumptions

The three dependent variables within each MANCOVA model were moderately correlated and there was no multicollinearity. Normal distribution of dependent variables was assessed visually and using the Shapiro-Wilk-test. Several dependent variables were significant (p < 0.05), however, visual inspection of Q-Q plots revealed that dependent variables were approximately normal. Given that MANCOVA is robust to minor violations of normality (44), the authors proceeded with analyses. Relationships between dependent variables and covariates were linear with homogeneous regression slopes, as determined by visual inspection of scatterplots. Residuals were normally distributed, as assessed by visual inspection of Q-Q plots. To determine the influence of outliers, each model was run with and without univariate and multivariate outliers. All results are reported with outliers included, as removal of outliers did not cause meaningful differences, with one exception (discussed below). Homogeneity of covariance matrices varied across models and is discussed below.

Rationale and Self-Guided iCBT

For the two-way MANCOVA (rationale ^* recruitment source with age and psychopathology as covariates) examining acceptability for self-guided iCBT, there was homogeneity of covariance matrices, as assessed by Box's M-test (p = 0.093). The multivariate main effect of recruitment source on the combined dependent variables was significant with seven univariate outliers (standardized residual > 3.0) included in the model (p = 0.048), but fell to non-significance when these outliers were removed (p = 0.051). Because the outliers appeared to be valid observations, results for this parameter with and without inclusion of outliers are reported.

See Table 3 for multivariate effects. There was no statistically significant interaction effect between rationale condition and recruitment source on the combined dependent variables, F_(3,587) = 0.762, p = 0.516, Wilks' Λ = 0.996, partial η² = 0.004. The main effect of rationale condition on the combined dependent variables was statistically significant, F_(3,587) = 3.617, p = 0.013, Wilks' Λ = 0.982, partial η² = 0.018. There was a statistically significant univariate effect of rationale condition for general attitudes, F_(1,589) = 9.382, p = 0.002, partial η² = 0.016, and for outcome expectancy, F_(1,589) = 5.886, p = 0.016, partial η² = 0.010, such that these two variables were higher for participants who received the treatment rationale. There was no statistically significant univariate effect of rationale condition for the single-item rating of willingness to use iCBT (p = 0.133). The main effect of recruitment source on the combined dependent variables was statistically significant with outliers included in the model, F_(3,587) = 2.657, p = 0.048, Wilks' Λ = 0.987, partial η² = 0.013. There was a statistically significant univariate effect of recruitment source on willingness to use iCBT, F_(1,589) = 7.033, p = 0.008, partial η² = 0.012, such that community participants reported greater willingness to use self-guided iCBT. When outliers were removed from this model, the multivariate effect of recruitment source on the combined dependent variables fell to non-significance (p = 0.051).

TABLE 3

Table 3. Multivariate effects for MANCOVA models examining the impact of a treatment rationale on attitudes toward iCBT.

Rationale and Therapist-Assisted iCBT

For the two-way MANCOVA (rationale ^* recruitment source with age and psychopathology as covariates) examining acceptability for therapist-assisted iCBT, Box's M-test was significant, indicating a violation of homogeneity of covariance matrices (p < 0.001). Accordingly, Pillai's Trace was used as a multivariate test statistic to control for inflation in Type I error rate (45).

See Table 3 for multivariate effects. There was no statistically significant interaction effect between rationale condition and recruitment source on the combined dependent variables, F_(3,571) = 0.227, p = 0.878, Pillai's Trace = 0.001, partial η² = 0.001. The main effect of rationale condition on the combined dependent variables was statistically significant, F_(3,571) = 7.421, p < 0.001, Pillai's Trace = 0.038, partial η² = 0.038. The main effect of recruitment source on the combined dependent variables was not statistically significant, F_(3,571) = 1.829, p = 0.141, Pillai's Trace = 0.010, partial η² = 0.010. There was a statistically significant univariate effect of rationale condition for general attitudes, F_(1,573) = 12.814, p < 0.001, partial η² = 0.022, and outcome expectancy, F_(1,573) = 6.045, p = 0.014, partial η² = 0.010, such that these two variables were higher for participants who received the treatment rationale. There was no statistically significant univariate effect of rationale condition for willingness to use iCBT (p = 0.578).

Type of iCBT

For the mixed design two-way MANCOVA (type of iCBT ^* recruitment source with age and psychopathology as covariates) comparing acceptability for self-guided and therapist-assisted iCBT, there was homogeneity of covariance matrices, as assessed by Box's M-test (p = 0.053).

See Table 4 for multivariate effects. There was no statistically significant interaction effect between type of iCBT and recruitment source on the combined dependent variables, F_(3,558) = 0.527, p = 0.664, Wilks' Λ = 0.997, partial η² = 0.003. The main effect of type of iCBT on the combined dependent variables was not statistically significant, F_(3,558) = 2.293, p = 0.077, Wilks' Λ = 0.988, partial η² = 0.012. The main effect of recruitment source on the combined dependent variables was statistically significant, F_(3,558) = 2.650, p = 0.048, Wilks' Λ = 0.986, partial η² = 0.014. There was a statistically significant univariate effect of recruitment source on willingness to use iCBT, F_(1,560) = 7.582, p = 0.006, partial η² = 0.013, such that community participants reported greater willingness to use iCBT.

TABLE 4

Table 4. Multivariate effects for MANCOVA comparing attitudes toward self-guided and therapist-assisted iCBT.

iCBT Uptake Behavior

See Table 5 for results of regression analysis. Participants were excluded from the analysis if they did not fit into the coding scheme for gender (N = 13, 2.0%) or race/ethnicity (N = 14, 2.1%), did not receive a timely follow-up email with a list of iCBT programs due to experimenter error (N = 28, 4.2%), or did not have complete data for variables included in the analysis (N = 22, 3.3%). Of the 662 total eligible participants, 588 participants were eligible for regression analysis. Out of these participants, 47 (8.0%) sought out information about how to access and download iCBT programs and 541 (92.0%) did not. Step one of the model, which included participant characteristics, significantly predicted uptake behavior, χ²(4) = 12.172, p = 0.016. Step one explained 4.8% (Nagelkerke R²) of the variance in uptake behavior and correctly classified 92.0% of cases, although it should be noted that the model predicted that 100% of participants would not engage in uptake behavior. In this step, psychopathology was positively related to uptake behavior for iCBT (OR = 1.026, p = 0.046) and identifying as Black/African-American, Hispanic/Latinx, or Asian was negatively associated with uptake behavior compared to identifying as White or multiracial (OR = 0.509, p = 0.029). Step two of the model, which added the rationale and financial incentive conditions as regressors, did not explain significantly greater variability in uptake behavior than step one, ΔR² = 0.003, p = 0.703. Rationale condition (OR = 0.893, p = 0.717) and financial incentive condition (OR = 1.264, p = 0.452) did not significantly predict uptake behavior, although the full model remained significant, χ²(6) = 12.876, p = 0.045.

TABLE 5

Table 5. Hierarchical logistic regression model predicting uptake-related behavior.

Discussion

Consistent with hypotheses, participants who read a treatment rationale reported significant increases in acceptability as measured by general attitudes and outcome expectancy for self-guided and therapist-assisted iCBT across a community and University student sample. Inconsistent with hypotheses, the treatment rationale had no influence on participants' willingness to use either self-guided or therapist-assisted iCBT. Surprisingly, participants' ratings of acceptability (across all three measures) did not significantly differ between self-guided and therapist-assisted iCBT; this finding is inconsistent with prior research, which has generally found that people prefer therapist-assisted over self-guided iCBT. This is the first study to examine the effect of an acceptability-facilitating intervention on behavior related to the uptake of iCBT. Neither the rationale nor the financial incentive influenced uptake behavior, which was very low.

Although the effects of the treatment rationale on acceptability were significant, they were small compared to similar controlled studies of acceptability-facilitating interventions for Internet-based mental health treatment. These interventions, which include treatment rationales, have produced medium-sized increases in acceptability (23, 29). Differences in the effects of acceptability-facilitating interventions between studies may be driven by variations in intervention content, overall length, and method of operationalizing acceptability. For example, past studies examining acceptability-facilitating interventions have used information about iCBT and techniques to increase outcome expectancy, much like in the current study. However, they have also used psychoeducation on specific mental disorders, personalized symptom assessments with feedback, patient testimonials, and appeals to participants' self-efficacy to use a specific program (29, 31, 46).

The smaller effect of the rationale on acceptability of iCBT in the current study relative to past studies may also be related to length. The current study's rationale was ~800 words in length. Previous research has found that treatment rationales of ~250 words may be the optimal length for enhancing outcome expectancy (47). Additionally, Casey et al. (23) found that an acceptability-facilitating intervention of ~400 words caused a medium-sized increase in acceptability for Internet-based mental health treatment. For the current treatment rationale, the authors prioritized describing iCBT in depth, incorporating outcome expectancy persuasion techniques, and addressing perceived advantages and disadvantages of iCBT that have been reported in previous research. The greater length may have caused fatigue or failed to hold participants' attention, which could have prevented participants from fully processing all of the information, thereby reducing its effect. Researchers constructing treatment rationales and other interventions to improve acceptability for iCBT in the future should be aware that acceptability-facilitating interventions which require longer reading times may reduce their impact.

The current study is the first to examine the effects of a treatment rationale and financial incentive on behavior related to the uptake of iCBT. Contrary to hypotheses, neither intervention significantly affected uptake-related behavior. Psychopathology symptom severity and race/ethnicity were associated with uptake of iCBT, although it should be noted that total regression model accounted for a very small proportion of variance (~5%). Participants who reported higher psychopathology were more likely to seek out information about how to access and download iCBT programs. Unlike some prior research on acceptability-facilitating interventions, participants from this study were not drawn from a treatment-seeking sample. It is possible that participants did not believe that they needed iCBT. Lack of a perceived need for mental health treatment is a widely documented barrier to seeking mental health services, particularly among people with mild to moderate symptoms (48). Given that over half of the participants in this study reported at least mild mental health symptoms, interventions designed to increase uptake of iCBT in the general population might have greater success using materials that emphasize the benefit of iCBT as a low-intensity intervention for individuals with mild to moderate symptoms. Personalized feedback about mental health symptoms may be particularly helpful for individuals who are unaware that they may benefit from iCBT. Conversely, people experiencing mild mental health symptoms may believe that they could benefit from iCBT, but be uninterested in making efforts to improve their mental health because their distress is relatively low. Future research on iCBT uptake could evaluate participants' readiness for change and tailor acceptability-facilitating interventions to increase motivation for change if this is a common barrier.

Participants identifying as “White” or “multiracial” were more likely to engage in behavior related to the uptake of iCBT than participants who self-identified as “Black/African-American,” “Hispanic/Latinx,” or “Asian.” Given that people who identify as racial/ethnic minorities are less likely to have access to and to use mental health services (49), this is a sobering finding. Digital mental health interventions, like iCBT, have the potential to overcome practical barriers to mental health treatment that disproportionately affect minority groups, such as cost and transportation (50, 51). The results from this study suggest that these communities may not be inclined to seek out such treatments, simply because they circumvent such practical barriers. Although a small number of studies have examined perceptions and interest in iCBT within specific racial and cultural minority communities (42, 52), many more are needed. It is also critical that future research identify the extent to which acceptability-facilitating interventions need to be culturally tailored to increase uptake in minority communities.

There are meaningful distinctions to be made between general appraisals toward an intervention, personal expectations of efficacy, and a willingness to engage with an intervention—the three dependent measures of acceptability in this study. Our results indicate that interventions which improve general attitudes and outcome expectancy for iCBT programs do not cause corresponding increases in willingness to use them. This may be due to method-variance, given that willingness to use iCBT was assessed using a single item. If, however, the finding is replicated and valid, it is concerning, because it suggests that attitudinal changes caused by treatment rationales and other interventions do not lead to greater self-reported willingness to use iCBT. This is reinforced by this study's finding that the treatment rationale did not increase uptake-related behavior for iCBT. Interestingly, community adults reported slightly greater willingness to use iCBT than University students, an effect that was not attributable to differences in age or psychopathology between samples. This may be due to disparities in access to face-to-face mental health treatment—the University students recruited for this study have access to no-cost counseling services, whereas most community participants likely do not.

Strengths and Limitations

This is the first experimental study to measure the effects of a treatment rationale on acceptability and uptake-related behavior for iCBT. It is also the first to examine the effect of a financial incentive on uptake-related behavior for iCBT. To date, this is the largest study to examine an acceptability-facilitating intervention for Internet-based mental health treatment. Furthermore, this study operationalized acceptability in a robust way by using three widely used measures of this construct, including a psychometrically validated measure of acceptability toward online mental health interventions. This is important because much of the existing literature that has examined acceptability toward iCBT has used heterogeneous measures of this construct. The need to increase the diversity and inclusion of minority and underrepresented populations in the literature concerning attitudes and utilization of iCBT is paramount. The study used a robust sampling method, recruited a diverse sample of urban community adults and University students, and reported participant characteristics that are associated with underutilization of mental health services. This is a major contribution to the literature; the majority of studies (97%) in a widely cited meta-analysis of randomized controlled trials supporting the efficacy and acceptability of iCBT (19) did not report the racial/ethnic make-up of their sample.

Despite the study's strengths, there are limitations that warrant attention. The small differences between the two rationale conditions in the current study may be due to the nature of the “brief definition” control condition. The authors determined that it was important to define self-guided and therapist-assisted iCBT for participants assigned to the control condition because iCBT is a relatively nascent technology and most people are unfamiliar with Internet-based mental health treatment (14). The brief definition, however, may have functioned like an active control and reduced the comparative effect of the treatment rationale. The use of a survey-based metric for examining uptake-related behavior may have limited our ability to detect true iCBT uptake, as it was insensitive to actual usage of programs. The results for uptake-related behavior cannot be generalized to gender non-conforming people and people outside of the specific racial identities that were predominant in our sample, as we did not have enough of these participants to examine them in our regression model. Additionally, although research has generally supported the use of raffles for incentivizing behavior change, it is possible that the ratio of financial incentive to odds of winning (1:30 chance for $25) was too weak to influence uptake-related behavior. Lastly, the majority of participants were college-educated, which may have implications for measuring attitudes toward Internet-based mental health treatments as educational attainment has been linked to mental health treatment-seeking (53).

Future Directions

More research is needed to systematically investigate differences in acceptability-facilitating interventions for iCBT that use different types of content. Studies should also examine whether interventions that cause significant improvements in acceptability also lead to measurable increases in uptake-related behavior. Studies examining financial incentives should evaluate the impact of different “doses” of incentive and their cost-effectiveness in healthcare delivery systems. Future research should investigate the relationship between acceptability for iCBT and access to other forms of care across different populations. Studies that recruit diverse samples across different demographic characteristics are vital for understanding the effect of individual characteristics on acceptability and uptake-related behavior for iCBT, as well as other relevant constructs. For example, certain minority racial identities are associated with lower levels of mental health service utilization (43) and racial disparities in trust and experiences with healthcare institutions may play a role in acceptability of digital forms of treatment in comparison to face-to-face care (54). It will be necessary to identify how iCBT appeals differently across racial groups and other demographics to maximize its delivery to those who can most benefit.

Conclusion

Internet-based cognitive behavioral therapy is well-positioned to leverage its intrinsic benefits of standardization, cost-effectiveness, and ease of access to help fill the gap in unmet mental health need. However, iCBT will be unable to fulfill these goals if acceptability toward these interventions is not significantly improved for the average consumer. The authors hope that future research will build on the findings of the current study to develop effective methods of improving acceptability and uptake-related behavior for iCBT programs in order to fully realize their potential.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Georgia State University Institutional Review Board. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

AM and PA devised the project, the main conceptual ideas, and protocol outline. AM, DE, and LS coordinated data collection for the study and wrote the manuscript. AM conducted all statistical analyses. AM and DE designed the figures and tables. PA supervised the project. All authors contributed to the final version of the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank the Anxiety Research and Treatment Lab and every undergraduate assistant who worked tirelessly in their efforts to canvass the community and collect data for this project. The authors affirm that this was an IRB approved protocol.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fdgth.2021.653686/full#supplementary-material

References

1. Hobfoll SE, Blais RK, Stevens NR, Walt L, and Gengler R. Vets prevail online intervention reduces PTSD and depression in veterans with mild-to-moderate symptoms. J Consult Clin Psychol. (2016) 84:31–42. doi: 10.1037/ccp0000041

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Gershkovich M, Herbert JD, Forman EM, and Glassman L. Guided internet-based self-help intervention for social anxiety disorder with videoconferenced therapist support. Cogn Behav Pract. (2016) 23:239–55. doi: 10.1016/j.cbpra.2015.08.005

CrossRef Full Text | Google Scholar

3. Fogliati VJ, Dear BF, Staples LG, Terides MD, Sheehan J, Johnston L, et al. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided internet-delivered treatment for panic disorder and comorbid disorders: a randomized controlled trial. J Anxiety Disord. (2016) 39:88–102. doi: 10.1016/j.janxdis.2016.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

4. International Telecommunication Union. Measuring Digital Development Facts and Figure 2019. (2019). Available online at: https://www.itu.int/en/ITU-D/Statistics/Documents/facts/FactsFigures2019.pdf (accessed January 15, 2021).

5. Wolitzky-Taylor K, Chung B, Bearman SK, Arch J, Grossman J, Fenwick K, et al. Stakeholder perceptions of the barriers to receiving and delivering exposure-based cognitive behavioral therapy for anxiety disorders in adult community mental health settings. Community Mental Health J. (2019) 55:83–99. doi: 10.1007/s10597-018-0250-z

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Substance Abuse and Mental Health Services Administration. Rural Behavioral Health: Telehealth Challenges and Opportunities. (2016). Available online at: https://store.samhsa.gov/sites/default/files/d7/priv/sma16-4989.pdf (accessed January 15, 2021).

7. Mohr DC, Cuijpers P, and Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res. (2011) 13:1–16. doi: 10.2196/jmir.1602

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Spek V, Cuijpers P, Nyklicek I, Riper H, Keyzer J, and Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. (2007) 37:319–28. doi: 10.1017/S0033291706008944

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Johansson R, and Andersson G. Internet-based psychological treatments for depression. Expert Rev Neurother. (2012) 12:861–70. doi: 10.1586/ern.12.63

CrossRef Full Text | Google Scholar

10. Carper MM, McHugh RK, and Barlow DH. The dissemination of computer-based psychological treatment: a preliminary analysis of patient and clinician perceptions. Adm Policy Mental Health Mental Health Serv Res. (2013) 40:87–95. doi: 10.1007/s10488-011-0377-5

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Hennemann S, Beutel ME, and Zwerenz R. Ready for eHealth? Health professionals' acceptance and adoption of eHealth interventions in inpatient routine care. J Health Commun. (2017) 22:274–84. doi: 10.1080/10810730.2017.1284286

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Waller R, and Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med. (2009) 39:705–12. doi: 10.1017/S0033291708004224

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Ng MM, Firth J, Minen M, and Torous J. User engagement in mental health apps: a review of measurement, reporting, and validity. Psychiatr Serv. (2019) 70:538–44. doi: 10.1176/appi.ps.201800519

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Handley TE, Perkins D, Kay-Lambkin F, Lewin T, and Kelly B. Familiarity with and intentions to use Internet-delivered mental health treatments among older rural adults. Aging Mental Health. (2015) 19:989–96. doi: 10.1080/13607863.2014.981744

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Wootton BM, Titov N, Dear BF, Spence J, Andrews G, Johnston L, et al. An internet administered treatment program for obsessive-compulsive disorder: a feasibility study. J Anxiety Disord. (2011) 25:1102–7. doi: 10.1016/j.janxdis.2011.07.009

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Travers MF, and Benton SA. The acceptability of therapist-assisted, internet-delivered treatment for college students. J Coll Stud Psychother. (2014) 28:35–46. doi: 10.1080/87568225.2014.854676

CrossRef Full Text | Google Scholar

17. Schröder J, Sautier L, Kriston L, Berger T, Meyer B, Späth C, et al. Development of a questionnaire measuring attitudes towards psychological online interventions-the APOI. J Affect Disord. (2015) 187:136–41. doi: 10.1016/j.jad.2015.08.044

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Titov N, Andrews G, Schwencke G, Robinson E, Peters L, and Spence J. Randomized controlled trial of internet cognitive behavioural treatment for social phobia with and without motivational enhancement strategies. Aust N Z J Psychiatry. (2010) 44:938–45. doi: 10.3109/00048674.2010.493859

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord. (2018) 55:70–8. doi: 10.1016/j.janxdis.2018.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Hedman E, Ljótsson B, and Lindefors N. Cognitive behavior therapy via the internet: a systematic review of applications, clinical efficacy and cost—effectiveness. Expert Rev Pharmacoecon Outcome Res. (2012) 12:745–64. doi: 10.1586/erp.12.67

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Van Ballegooijen W, Cuijpers P, Van Straten A, Karyotaki E, Andersson G, Smit JH, et al. Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS ONE. (2014) 9:e100674. doi: 10.1371/journal.pone.0100674

CrossRef Full Text | Google Scholar

22. Mohr DC, Siddique J, Ho J, Duffecy J, Jin L, and Fokuo JK. Interest in behavioral and psychological treatments delivered face-to-face, by telephone, and by internet. Ann Behav Med. (2010) 40:89–98. doi: 10.1007/s12160-010-9203-7

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Casey L, Joy A, and Clough BA. The impact of information on attitudes toward e-mental health services. Cyberpsychol Behav Soc Netw. (2013) 16:593–8. doi: 10.1089/cyber.2012.0515

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Mitchell N, and Gordon PK. Attitudes towards computerized CBT for depression amongst a student population. Behav Cogn Psychother. (2007) 35:421–30. doi: 10.1017/S1352465807003700

CrossRef Full Text | Google Scholar

25. Mohr DC, Lyon AR, Lattie EG, Reddy M, and Schueller SM. Accelerating digital mental health research from early design and creation to successful implementation and sustainment. J Med Internet Res. (2017) 19:1–22. doi: 10.2196/jmir.7725

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Gun SY, Titov N, and Andrews G. Acceptability of internet treatment of anxiety and depression. Aust Psychiatry. (2011) 19:259–64. doi: 10.3109/10398562.2011.562295

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Wootton BM, Titov N, Dear BF, Spence J, and Kemp A. The acceptability of internet-based treatment and characteristics of an adult sample with obsessive compulsive disorder: an internet survey. PLoS ONE. (2011) 6:1–6. doi: 10.1371/journal.pone.0020548

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Ahmed M, and Westra HA. Impact of a treatment rationale on expectancy and engagement in cognitive behavioral therapy for social anxiety. Cogn Ther Res. (2009) 33:314–22. doi: 10.1007/s10608-008-9182-1

CrossRef Full Text | Google Scholar

29. Ebert DD, Berking M, Cuijpers P, Lehr D, Pörtner M, and Baumeister H. Increasing the acceptance of internet-based mental health interventions in primary care patients with depressive symptoms: a randomized controlled trial. J Affect Disord. (2015) 176:9–17. doi: 10.1016/j.jad.2015.01.056

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Soucy JN, Owens VAM, Hadjistavropoulos HD, Dirkse DA, and Dear BF. Educating patients about Internet-delivered cognitive behaviour therapy: perceptions among treatment seekers and non-treatment seekers before and after viewing an educational video. Internet Interv. (2016) 6:57–63. doi: 10.1016/j.invent.2016.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Baumeister H, Reichler L, Munzinger M, and Lin J. The impact of guidance on Internet-based mental health interventions—a systematic review. Internet Interv. (2014) 1:205–15. doi: 10.1016/j.invent.2014.08.003

CrossRef Full Text | Google Scholar

32. Burton A, Marougka S, and Priebe S. Do financial incentives increase treatment adherence in people with severe mental illness? A systematic review. Epidemiologia e Psichiatria Sociale. (2010) 19:233–42. doi: 10.1017/S1121189X00001160

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Post EP, Cruz M, and Harman J. Incentive payments for attendance at appointments for depression among low-income African Americans. Psychiatr Serv. (2006) 57:414–6. doi: 10.1176/appi.ps.57.3.414

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Stanley IH, Chu C, Brown TA, Sawyer KA, and Joiner TE. Improved clinical functioning for patients receiving fee discounts that reward treatment engagement. J Clin Psychol. (2016) 72:15–21. doi: 10.1002/jclp.22236

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Ametrano RM, Constantino MJ, and Nalven T. The influence of expectancy persuasion techniques on socially anxious analogue patients' treatment beliefs and therapeutic actions. Int J Cogn Ther. (2017) 10:187. doi: 10.1521/ijct.2017.10.3.187

CrossRef Full Text | Google Scholar

36. Center for Collegiate Mental Health. Center for Collegiate Mental Health Standardized Data Set (SDS). University Park, PA: Pennsylvania State University (2017). p. 1–34.

37. Devilly GJ, and Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry. (2000) 31:73–86. doi: 10.1016/S0005-7916(00)00012-4

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Handley TE, Kay-Lambkin FJ, Inder KJ, Attia JR, Lewin TJ, and Kelly BJ. Feasibility of internet-delivered mental health treatments for rural populations. Soc Psychiatry Psychiatr Epidemiol. (2014) 49:275–82. doi: 10.1007/s00127-013-0708-9

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Lovibond SH, and Lovibond PF. Manual for the Depression Anxiety Stress Scales (DASS). Sydney, NSW: Psychology Foundation Monograph (1993).

Google Scholar

40. Lovibond PF, and Lovibond SH. The structure of negative emotional states: comparison of the depression anxiety stress scales (DASS) with the beck depression and anxiety inventories. Behav Res Ther. (1995) 33:335–43. doi: 10.1016/0005-7967(94)00075-U

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Or CKL, and Karsh BT. A systematic review of patient acceptance of consumer health information technology. J Am Med Inform Assoc. (2009) 16:550–60. doi: 10.1197/jamia.M2888

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Jonassaint CR, Gibbs P, Belnap BH, Karp JF, Abebe KK, and Rollman BL. Engagement and outcomes for a computerised cognitive-behavioural therapy intervention for anxiety and depression in African Americans. Br J Psychiatry Open. (2017) 3:1–5. doi: 10.1192/bjpo.bp.116.003657

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Substance Abuse and Mental Health Services Administration. National Survey on Drug Use and Health, 2002-2019. (2020). Available online at: https://www.samhsa.gov/data/report/2019-nsduh-detailed-tables (accessed January 15, 2021).

Google Scholar

44. Verma JP. Repeated Measures Design for Empirical Researchers. Hoboken, NJ: John Wiley and Sons (2016).

Google Scholar

45. Olson CL. On choosing a test statistic in multivariate analysis of variance. Psychol Bull. (1976) 83:579–86. doi: 10.1037/0033-2909.83.4.579

CrossRef Full Text | Google Scholar

46. Ebert DD, Franke M, Kählke F, Küchler AM, Bruffaerts R, Mortier P, et al. Increasing intentions to use mental health services among University students. Results of a pilot randomized controlled trial within the World Health Organization's World Mental Health International College Student Initiative. Int J Methods Psychiatr Res. (2019) 28:1–13. doi: 10.1002/mpr.1754

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Horvath P. Treatment expectancy as a function of the amount of information presented in therapeutic rationales. J Clin Psychol. (1990) 46:636–42. doi: 10.1002/1097-4679(199009)46:5<636::aid-jclp2270460516>3.0.co;2-u

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Andrade LH, Alonso J, Mneimneh Z, Wells JE, Al-Hamzawi A, Borges G, et al. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychol Med. (2014) 44:1303–17. doi: 10.1017/S0033291713001943

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Stockdale SE, Lagomasino IT, Siddique J, McGuire T, and Miranda J. Racial and ethnic disparities in detection and treatment of depression and anxiety among psychiatric and primary health care visits, 1995-2005. Med Care. (2008) 46:668–77. doi: 10.1097/MLR.0b013e3181789496

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Alegria M, Lin J, Chen C-N, Duan N, Cook B, and Meng X-L. The impact of insurance coverage in diminishing racial and ethnic disparities in behavioral health services. Health Serv Res. (2012) 47:1322–44. doi: 10.1111/j.1475-6773.2012.01403.x

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Snell-Johns J, Mendez JL, and Smith BH. Evidence-based solutions for overcoming access barriers, decreasing attrition, and promoting change with underserved families. J Fam Psychol. (2004) 18:19–35. doi: 10.1037/0893-3200.18.1.19

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Choi I, Andrews G, Sharpe L, and Hunt C. Help-seeking characteristics of Chinese- and English-speaking Australians accessing Internet-delivered cognitive behavioural therapy for depression. Soc Psychiatry Psychiatr Epidemiol. (2015) 50:89–97. doi: 10.1007/s00127-014-0956-3

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Steele L, Dewa C, Lin E, and Lee K. Education level, income level and mental health services use in Canada: associations and policy implications. Healthcare Policy. (2007) 3:96–106. doi: 10.12927/hcpol.2007.19177

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Boulware LE, Cooper LA, Ratner LE, Laveist TA, and Powe NR. Race and trust in the Health Care System. Public Health Rep. (2003) 118:358–65. doi: 10.1016/S0033-3549(04)50262-5

PubMed Abstract | CrossRef Full Text | Google Scholar