The effectiveness of CBT-based daily supportive text messages in improving female mental health during COVID-19 pandemic: results from the Text4Hope program

Introduction: The COVID-19 pandemic has significantly exacerbated gender disparities in mental health, particularly impacting women. To address this, Alberta, Canada, launched Text4Hope, a Cognitive Behaviour Therapy-based text messaging intervention, to provide support and resources for psychological challenges during the pandemic. This study aimed to assess the effectiveness of Text4Hope in reducing stress, anxiety, depression, sleeping disturbances, and suicidal ideation among female subscribers during the COVID-19 pandemic.

Methods: The study employed both an uncontrolled longitudinal design and a controlled cohort design. The uncontrolled longitudinal study analyzed a one-year dataset (n = 9,545) of clinical outcomes, comparing mean differences in mental health symptoms from baseline to 6 weeks after subscription. The controlled cohort design compared two groups, with (n = 1,763) and without (n = 567) intervention exposure during the same period. Data were collected through self-administered online surveys completed at baseline and six weeks after subscription. Sociodemographic information and validated scales (e.g., 10-item Perceived Stress Scale (PSS-10), Generalized Anxiety Disorder (GAD-7), and Patient Health Questionnaire (PHQ-9)) were used to assess mental health outcomes.

Results: The results from the longitudinal study indicated a significant reduction in anxiety prevalence and anxiety symptoms, with a 19.63% decrease in GAD-7 mean score and a 32.02% decrease in likely anxiety from baseline to six weeks. Depressive symptoms and perceived stress also showed a significant reduction (p < 0.001), albeit to a lesser extent. In the controlled cohort study, the intervention group had significantly (p < 0.001) lower PHQ-9 [19.5 (SD 7.05)], GAD-7 [7.5 (SD 5.27)], and CMH [35.53 (SD 18.45)] scores. Additionally, the study found substantial differences (p < 0.001) in suicidal ideation (26.1 vs. 15.7) between groups but no significant differences in sleep disruption.

Discussion: These findings suggest that Text4Hope could be an effective intervention for reducing stress, depression, suicidal ideation, and particularly anxiety symptoms among women during public emergencies. The study provides valuable insights into the potential benefits of text messaging interventions in supporting mental health during crisis situations.

1. Introduction

The COVID-19 pandemic has profoundly impacted women's mental health since it has brought to light pre-existing inequities and exacerbated disparities surrounding the psychological burden experienced by the female population (1). In pre-pandemic times, women were more likely to experience mental health problems. The prevalence of anxiety disorders in Canadian women is 6.3%, while in men, these figures reach 3.7% (2). Depression, which is a major risk factor for suicide, is two times more present in females than males (3), and although the prevalence of deaths caused by suicide is three times higher in males than in females (4), suicide attempts are 1.5 to 2 times more frequent in women and girls than men and boys (5). Women also report higher levels of perceived stress than men, which is generally attributable to their multiple roles and responsibilities in their careers and family (6). This is supported by data from the Canadian Community Health Survey, which has demonstrated a rising trend of high stress in the female population since 2003, with a rate of 23.7% for women in 2014 compared to 22.2% for males (7). High levels of stress can cause sleep problems, which have been found to be more common in women, particularly those with lower education and income, with a 10% higher prevalence than the male population (8).

Some of the factors that contribute to women's higher risk of mental health symptoms include biological and hormonal changes throughout life, particularly during pregnancy, the post-partum period, and menopause, which are considered critical periods (9, 10). Social and cultural pressures imposed on women, multiple roles and responsibilities within work and family, and a higher likelihood of experiencing poverty, traumatic events, discrimination, harassment, and gender-based violence can all play a role in the increased psychological burden on the female population (11). The gender disparities also appear to be influenced by women's greater natural tendency to report emotional symptoms and seek help than men, making them more likely to be diagnosed with mental health problems (12).

The uncertainty of the pandemic, public health measures (i.e., quarantine and lockdowns), and social disruptions of this unprecedented time have burdened everyone's mental health (13). Females were again at a higher risk of developing psychiatric symptoms during the COVID-19 outbreak (14). Amidst this public health emergency context, being a woman at a younger age, having student status and past psychiatric history, experiencing quarantine and being a COVID-19-positive patient (15) were all identified as risk factors for stress, anxiety, and depression (15, 16). Job status was also related to increased mental health symptoms, particularly for unemployed and female healthcare workers (17, 18, 19). Environmental aspects such as frequent social media use, pandemic news exposure, and an unhealthy diet were positively associated with greater odds of anxiety during the pandemic (20, 21). Furthermore, as observed in previous crises, the pandemic led to a surge in intimate partner violence and mothers witnessing violence against their children, which increased gender-based psychological issues (22–24).

To ease the strain on women's mental health, text messaging interventions are being proposed as a cost-effective program to provide support and resources to help them cope with psychological challenges, particularly those who may have limited access to traditional mental health services (25, 26). Text message interventions, also known as SMS interventions, are a type of digital mental health support that can be used for a variety of purposes, including supportive text messages, social support engagement, care team contact capabilities, data feedback, psychoeducation, adherence-based psychotherapy, remote care delivery, secure medication storage, and contingency planning (27, 28). Cognitive Behavioural Therapy (CBT), a psychotherapy approach that has been found effective for a wide range of mental health conditions, can be delivered in a variety of modalities (29). High-intensity CBT is generally provided by trained health professionals and commonly delivered in a face-to-face setting, individually or in a group format. Conversely, low-intensity CBT primarily focuses on self-help approaches and can be delivered through various platforms, including the Internet, telephone, and text messaging (30, 31). CBT modalities appear to be equally effective, and although the evidence on low-intense CBT interventions is not as extensive as for traditional CBT, the literature suggests that they can be just as effective as traditional CBT, particularly for individuals who may have limited access to traditional mental health services (30). Text message interventions have been found to be effective in reducing symptoms of anxiety, depression and perceived stress in women (32). Although highly beneficial, text message interventions cannot replace professional help but serve as a complementary tool to support mental health (33).

Text4Hope is a CBT-based text message intervention targeting individuals experiencing stress, anxiety and depression as a result of the COVID-19 pandemic (34). The service was launched in March 2020 in Alberta, Canada, and as of July 2021, over 54,000 Canadians had signed up for the Text4Hope program (35). Previous studies (36–38) have shown that the Text4Hope intervention can reduce symptoms of anxiety, depression, and perceived stress in the general population, with consistent results across different evaluation methodologies. A recent study using a two-pronged design to evaluate the Text4Hope database focused on youth mental health and confirmed the positive impact of the intervention by comparing outcomes over time and between groups (39). However, the specific impact of the program on female subscribers is not well understood, despite the majority of Text4Hope subscribers being women and the higher likelihood of women experiencing mental health issues. Therefore, this study aims to analyze a subset of the Text4Hope database to evaluate the intervention's efficacy in reducing symptoms of stress, anxiety, depression, sleep disturbances, and suicidal thoughts among female subscribers. By utilizing a longitudinal and a controlled cohort evaluation approach, this research provides insight into the potential benefits of text messaging interventions targeting female populations and their impact on mental wellbeing (33).

2. Methodology

2.1. Study design

This study utilized a specific dataset of Text4Hope female subscribers to evaluate the impact of the intervention on mental health outcomes of this particular population. It employed an uncontrolled within-subject comparison (longitudinal study) and a controlled between-subject comparison (cohort study) to analyze the available data. The longitudinal study examined clinical outcomes from a one-year dataset of female subscribers who completed both the baseline and the follow-up survey at 6 weeks after subscription. This group of Text4Hope subscribers corresponds to 24.99% (n = 9,475) of the 53,309 female individuals who subscribed to the program between March 2020 and March 2021. (Figure 1).

FIGURE 1

Figure 1. Longitudinal study flowchart and context of the female population dataset within the Text4Hope initiative.

The cohort study compared two distinct study populations of Text4Hope female subscribers (Figure 2). The first group, called the intervention group (IG, n = 1,723), comprised female subscribers who received the intervention for six weeks and completed the follow-up assessment survey six weeks after enrolment, between April 26 and July 12, 2020. The second group, referred to as the control group (CG, n = 567), consisted of female subscribers who joined the program in the same period, completed the baseline evaluation only, and had not yet received any text messages, as they were at the beginning of the program. Data from these subscribers (CG) were used solely to establish a comparison group of individuals who had not received the intervention at that point in time. By comparing participants with different levels of exposure to the intervention (IG with intervention exposure and CG without intervention exposure) during the same time period, we were able to evaluate the additional impact of the intervention on symptom levels beyond what would be expected from the natural course of the pandemic. This study design allowed us to examine the natural evolution of the outcomes of interest by observing individuals and comparing outcomes between groups. Both IG and CG were also part of the longitudinal study since a one-year dataset was included in the longitudinal analysis. However, the two studies are separate and distinct, each serving different purposes in evaluating the intervention's impact on mental health outcomes. The research protocol (32) was approved by the University of Alberta Research and Ethics Board (Pro00086163).

FIGURE 2

Figure 2. Controlled cohort study flowchart context of the female population dataset within the Text4Hope initiative.

It is important to note that the primary goal of the Text4Hope program was to provide mental health support to individuals during the challenging times of the COVID-19 pandemic. As a result, the flowchart presented in this study reflects the natural progression of participants who voluntarily engaged with the program and completed the assessment surveys.

2.2. Study intervention

The intervention tested in this study was Text4Hope, a CBT-based text messaging program implemented in Alberta, Canada, during the COVID-19 pandemic. During this challenging time, the Text4Hope program emerged as a unique initiative, potentially contributing to reducing the gap to access mental health support since it was launched in March 2020. To the best of our knowledge, no other program of its kind was implemented in such a timely manner worldwide at that time.

The text messages provided by this program are based on CBT principles and were created by psychiatrists, psychologists, and mental health therapists to provide subscribers with bite-sized pieces of information about the cognitive triangle (thoughts, feelings, and actions) to achieve therapeutic benefit in the long run. The content of the text messages was designed to promote self-care, social support, hope, and affirmation, as well as to assist subscribers in managing symptoms of stress, anxiety and depression. Examples of text messages sent were:

• Self-care is always important. During times of stress, it becomes even more important as stress can take a toll on our mental and physical well-being. Hydrate, make a plan, prioritize sleep, meditate, unload and say no, connect with friends and family on phone even as you social distance.

• Notice when you're feeling sad, angry, lost or overwhelmed about life changes. Don't push the feelings away -acknowledge these feelings and let yourself grieve.

• When your emotions are high, calm yourself down by taking slow, deep breaths in through your nose and out through your mouth.

Individuals from the general population could self-subscribe to receive SMS text messages from the program for three months by texting the word “COVID19HOPE” to a short code number. The one-way (no-reply) daily supportive text messages were set to be automatically sent at 9 a.m. (Mountain Time) daily for the length of the program (3 months). After subscribing to the program, users were greeted with a welcome message and directed to a link where they completed the baseline assessment survey. Six weeks after enrollment, the link to the follow-up survey was also sent via SMS. Subscribers received the daily text messages regardless of whether or not they answered the surveys since participation was entirely voluntary.

2.3. Data collection

Study data were collected from self-completed online surveys powered by REDCap, including demographic information and mental health symptoms, by Text4Hope's subscribers. Demographic information, including age, ethnicity, educational level, employment, relationship and housing status, were collected at baseline. Validated scales that reflect the mental health outcomes of interest in the general population were used to assess subscribers' mental health symptoms at baseline and six weeks after enrollment. Originally, Text4Hope follow-up assessments were conducted six weeks and three months after the baseline assessment, covering the entire program length. However, in this paper, we decided to focus on the six-week follow-up only, aiming to capture initial outcomes into the early impact of the Text4Hope program on the mental health outcomes of female subscribers.

Data for the observational controlled study were collected from April 26 to July 12, 2020. This period was chosen because the majority of subscribers joined Text4Hope in the first three months of the program. Having a large number of individuals who subscribed to the program in the same time frame allowed investigators to ensure that both the IG and CG were exposed to similar external factors and contextual influences. In addition, this duration represents the timeframe of the follow-up period for the first study cohort, who were recruited in the initial phase (from March 23 and April 25, 2020), and it is worth noting that the only difference between the IG and CG was that the CG had subscribed to the Text4Hope program and completed the baseline assessment, without receiving yet any intervention. This period also coincided with the first wave of the pandemic, and with the first lockdown in Alberta that began on March 17th, 2020. In contrast, the longitudinal study dataset was collected over a year, starting from March 20, 2020, and ending on March 20, 2021, covering the entire first year of the pandemic. During this period, the lockdown restrictions in Alberta were gradually lifted between May and July 2020, although some limitations remained in place until July 12, 2020. The province experienced a second wave in the fall of 2020, which prompted a new round of restrictions and closures (40).

2.4. Outcome measures

Study outcomes were measured through quantitative data from self-completed online surveys. Perceived stress, symptoms of anxiety and depression were assessed by using the 10-item Perceived Stress Scale (PSS-10) (41), the Generalized Anxiety Disorder (GAD-7) scale (42), and the Patient Health Questionnaire (PHQ-9) (43), respectively. The primary outcome of the study was determined by analyzing the total scores of each scale. This involved assessing the mean changes between baseline and 6 weeks in the longitudinal study, and calculating the mean difference (95% confidence interval) between the IG and CG groups in the controlled study. Secondary outcomes were differences between baseline and 6 weeks and between groups IG and CG in self-reported prevalence rates of moderate/high stress, likely generalized anxiety disorder GAD, and likely major depressive disorder (MDD). The criteria for moderate/high stress was established as having a PSS-10 score of 14 or higher (44). The presence of a GAD-7 score of 10 or higher was indicative of GAD (42), while a PHQ-9 score of 10 or higher was suggestive of MDD (43). The Composite Mental Health (CMH) score, which is the sum of all scores from the previously mentioned scales, was also assessed. The third and ninth questions of PHQ-9 were used to evaluate sleep disturbances and suicidal ideation, respectively.

2.5. Hypothesis

The hypothesis for this study is that after six weeks of receiving the daily supportive text messages (IG), female participants would have at least 25% lower scores on the CMH score, PSS-10, PHQ-9, and GAD-7 scales, and the prevalence for each of moderate/high stress, likely GAD, likely MDD, disturbed sleep and suicidal ideation and thoughts of self-harm would also be lower for IG participants than for CG participants. This conservative forecast is based on previous randomized controlled trials (44, 45) utilizing supportive text messages for patients with major depressive disorder and alcohol use disorder. These studies found a 25% and a 50%, respectively, greater reduction in their depressive symptom scores compared to patients who did not receive the daily supportive messages.

2.6. Statistical analysis

The longitudinal study compared sociodemographic and clinical factors between participants who completed both baseline and sixth-week surveys and those who completed only the baseline survey using the Chi-square. Descriptive statistics were used to analyze the distribution of demographic characteristics and isolation conditions of the studied population. We employed a paired sample t-test and the McNemar test to compare mean PSS-10, GAD-7, PHQ-9, and CMH scores and the prevalence of moderate to high stress, likely GAD, and likely MDD from baseline to six weeks for individuals who completed questionnaires at both periods. The controlled study compared IG with CG demographics. A Chi-Square test was used to compare the prevalence of moderate to high stress, likely depression, likely anxiety, having suicidal ideation and sleep disturbance symptoms between the IG at 6 weeks and CG at baseline. In addition, mean scores were compared using independent t-tests, and Welch t-test was used when equality of variance assumption was violated as indicated by Leven's test. To address the issue of multiple testing and potential inflation of type I error, the Bonferroni correction was applied. Finally, multivariate logistic regression models were performed to assess the impact of the intervention on the clinical variables while controlling for demographic variables. There were no imputations for missing values and no matching techniques for the demographic characteristics between the two study groups was performed. Participants were included in the analysis even with only one outcome variable. In other words, individuals who had missing data for one outcome measure but had data available for other outcome measures were still included in the analysis for the respective measures they had completed. This accounts for the differences in the sample size (n) for each variable, as it may vary from the initial sample size included in the sociodemographic analysis. However, we controlled for these characteristics in a logistic regression model, as previously stated. Data analysis was performed using SPSS for Windows version 25 (IBM Corporation) (46).

3. Results

3.1. Longitudinal study results

The longitudinal study illustrates results from participants who completed the baseline and 6-week surveys. Sociodemographic characteristics of the longitudinal study participants are shown in Table 1. A total of 9,475 female participants subscribed to Text4Hope during a one-year timeframe of the study. The majority of the studied population was between 26 and 60 years old (7,499, 79.1%), white (7,825, 83.1%), had postsecondary education (7,752, 86.7%), was employed (6,021, 67.3%), was married, cohabiting or partnered (6,337, 70.9%), and owned a house (5,894, 66.8%). Among females aged 25 years or less, we found a higher frequency of students (349, 36.2%), singles (440, 45.8%) and individuals living with family (553, 58.3%).

TABLE 1

Table 1. Distribution of demographic characteristics of female subscribers who completed baseline and 6-week surveys, based on the age category of the participants.

Table 2 demonstrates the changes in baseline mean scores for the primary outcome variables after the introduction of Text4Hope. The table results indicate that mean scores for PSS-10, PHQ-9, and GAD-7 were lower after six weeks of receiving text messages. Specifically, there was a reduction of 6.24%, 8.80%, and 19.63% in mean scores, respectively (relative measure of change). This difference was significant for all three variables (p < 0.001), with a stronger effect size for anxiety.

TABLE 2

Table 2. Changes in baseline mean scores of PSS-10, PHQ-9, and GAD-7 after the introduction of Text4Hope.

Table 3 shows the percentage of female subscribers who reported moderate-to-high stress, likely depression, and likely anxiety at baseline and 6 weeks. A significant reduction (p < 0.001) in the prevalence of moderate-to-high stress (86.5 vs. 79.5), likely depression (42.5 vs. 36.4), and likely anxiety (45.6 vs. 31.0) were found when comparing data from both time points.

TABLE 3

Table 3. Prevalence of moderate-to-high stress, likely GAD, and likely MDD at baseline and 6-weeks (McNemar test).

3.2. Controlled cohort study results

The results of the controlled trial exhibit data from two distinct groups of Text4Hope female subscribers (IG with intervention exposure and CG without intervention exposure). Table 4 presents the demographic characteristics of IG (n = 1,763) and CG (n = 567) at the time of their enrollment in the program. Similar majorities (41–60 years old, white, post-secondary education, employed, married, and homeowners) were found in both groups. However, the IG and CG had significant variations (p < 0.001) in age, educational level, employment status, and housing status.

TABLE 4

Table 4. Sociodemographic distribution characteristics of female subscribers of the IG and CG at baseline.

Table 5 summarizes the differences between the intervention and control groups in the three primary outcome variables and the Composite Mental Health (CMH) score. With the exception of PSS-10, the intervention group had significantly (p < 0.001) lower PHQ-9 [19.5 (SD 7.05)], GAD-7 [7.5 (SD 5.27)], and CMH [35.53 (SD 18.45)] scores. Bonferroni correction was applied for multiple testing, demonstrating a significance level of p 0.012, which maintains the significance level for the CMH score.

TABLE 5

Table 5. Independent sample t-test comparing the mean scores for IG and CG on PSS-10, the GAD-7, and PHQ-9 scales and the Composite Mental Health (CMH) score.

Table 6 demonstrates the difference in the prevalence of moderate to high stress, likely depression, likely anxiety,sleep disturbance, and suicidal ideation/thoughts of self-hams between the IG and CG. The intervention group had significantly (p < 0.001) lower moderate to high stress (88.6 vs. 79.0), likely depression (51.4 vs. 36.2), likely anxiety (47.0 vs. 31.0), and suicidal thoughts (26.1 vs. 15.7). No differences were detected in the prevalence of sleep disturbance (p > 0.005) between the two groups.

TABLE 6

Table 6. Chi-square test of association between prevalence of clinical parameters and study arm.

Table 7 explores the predictive value of each group (with or without intervention) on the likelihood of the psychological conditions of interest when controlling for demographic (age, ethnicity, education level, relationship, employment and housing status). Statistical significance was observed in the models predicting the moderate/high stress (Χ² (df = 21; n = 1,896) = 149.36, p < .001, accounting for 7.6% (Cox and Snell R²) to 12.3% (Nagelkerke R²) of the variance; and correctly classified 81.5% of the cases), likelihood of MDD (Χ² (df = 21; n = 1,813) = 165.11, p < .001, accounting for 8.7% (Cox and Snell R²) to 11.7% (Nagelkerke R²) of the variance; and correctly classified 65.3% of the cases), likelihood of GAD (Χ² (df = 21; n = 1,774) = 190.78, p < .001, accounting for 10.2% (Cox and Snell R2) to 14% (Nagelkerke R²) of the variance; and correctly classified 69.1% of the cases), and suicidal ideation/self-harm thoughts (Χ² (df = 21; n = 1,813) = 190.80, p < .001, accounting for 10% (Cox and Snell R²) to 16.2% (Nagelkerke R²) of the variance; and correctly classified 83.1% of the cases). The model predicting sleep disruption (Χ² (df = 21; n = 1,813) = 30.30, p.086, accounting for 1.7% (Cox and Snell R²) to 2.6% (Nagelkerke R²) of the variance; and correctly classified 78.6% of the cases) was not statistically significant. Results from the table show that the intervention group had lower odds of having the psychological symptoms (p < 0.001 and OR < 1), meaning at the Text4Hope intervention was a significant predictor for lower odds of moderate to high stress (OR 0.54), likely anxiety (OR 0.55), likely depression (OR 0.57), and suicidal ideation (OR 0.55). The text messaging intervention was not found to be a predictor for lower sleep disturbances (p > 0.05 and OR 0.80).

TABLE 7

Table 7. Summary from five multivariate logistic regression models for respondents’ likelihood to present with moderate to high stress, likely anxiety, likely depression, suicidal ideation or self-harm thoughts, and sleep disturbance symptoms, with a focus on the type of subscribers (intervention or control), while controlling for other demographic variables.

4. Discussion

This observational study examined the effectiveness of Text4Hope in improving mental health among female subscribers during the COVID-19 pandemic. The overall results of both longitudinal and controlled studies suggest that Text4Hope intervention may be effective in reducing symptoms of stress, depression, and anxiety among female subscribers who engaged in the program and completed the assessment surveys.

Females are more likely to engage in different e-health behaviours than males (47), and they usually present higher satisfaction levels as text messaging program users than their male counterparts (48, 49). This partially explains why most Text4Hope's subscribers are female, accounting for approximately 88% of the total subscribers according to data from previous studies (36–38). The longitudinal study revealed demographic characteristics of a privileged segment of the population: they are between the ages of 26 and 60, white, have completed some college-level education, are currently employed, are in a relationship, and own their own home. The low prevalence of marginalized women groups, such as women from ethnic or racial minorities in the study population, may be due to the trend of this group having less access to mobile technology than the general population, suggesting that access to the Text4Hope program may also be limited (50). Age, level of education, income, and geographic location are all factors that contribute to less mobile device accessibility. Nonetheless, these disadvantaged women groups may encounter additional barriers to mobile technology access, such as a lack of digital literacy, infrastructure, or discrimination (51, 52). Inequalities in the use of e-health services are likely to persist unless efforts are made to increase access to mobile phones, especially among the underserved female population, as part of implementing population-level digital health programs.

The sociodemographic characteristics of female subscribers were analyzed in a controlled trial, revealing significant differences in age, educational level, employment status, and housing status between the Intervention Group (IG) and Control Group (CG). These differences might be related to the time taken by each group to seek mental health support during the pandemic. Specifically, individuals in the IG (who had completed the 6-week survey during the data collection period) began seeking mental health support through Text4Hope just one week after the World Health Organization (WHO) officially announced the pandemic on March 11, 2020. In contrast, the CG (who had enrolled in the program during the data collection period) began seeking mental health support almost two months later. Our findings suggest that subscribers younger than 25 years old, had lower education levels, and were living with family tended to take longer to seek help. On the other hand, women between the ages of 41 and 60, with higher education levels, who were unemployed and owned a home tended to seek help faster during this public emergency. These findings highlight the importance of considering sociodemographic factors when designing interventions for mental health support during crises to ensure that vulnerable populations receive timely support.

Although the results of this study did not entirely corroborate the expected changes stated in the researchers' hypothesis, the intervention had a significant (p < 0.001) influence on the primary outcomes of the study, reducing perceived stress, anxiety, and depressive symptoms. Two randomized controlled trials using CBT-based text messaging programs as the intervention to be tested have yielded comparable results (44, 45). From baseline to six weeks, there was a reduction of −19.63% in the GAD-7 mean score, indicating a greater change in anxiety symptoms. Changes related to depressive symptoms and perceived stress were less expressive but still significant. The same trend was found in the prevalence of moderate-to-high stress, likely depression, and likely anxiety and in the mean scores of the three primary outcome variables at enrolment (BS) and six weeks (6 W) in the study. A significant (p < 0.001) decrease in the prevalence of these symptoms, especially in likely anxiety, and lower scores in the related assessment scales were found. It is worthy to note that the results were presented in both mean scores and prevalence rates of symptom categories to provide a comprehensive understanding of the impact of the intervention on mental health outcomes. By reporting mean scores, the magnitude of change in symptom severity can be assessed, while the prevalence rates enable the examination of the proportion of participants who experienced clinically significant symptom reduction. Together, these results do not provide additional evidence of the intervention's effectiveness but rather offer an alternative perspective on the outcomes. However, the differences encountered suggest that Text4Hope intervention successfully improved mental health outcomes among female subscribers in a short time frame of 6 weeks. Similar results were found in previous studies evaluating male population. We have previously evaluated the impact of the Text4Hope intervention in men and the results showed a positive impact of the intervention in this gender group.

CBT-based text messages from the Text4Hope program effectively deliver a low-intensity CBT intervention and promote mental well-being among a large population. The impact of these bite-sized CBT pieces may be related to the “just-in-time” nature of the intervention, delivering support at the moment and in the context the person needs (53). This is especially important during times of high vulnerability or periods of increased susceptibility to negative emotions and thoughts (54), such as the ones the population experienced during the pandemic's social isolation. The “time and place” component of daily supportive text messages appears to be critical in determining the beneficial support in this situation. Furthermore, the portable and ubiquitous nature of mobile phones allows individuals to continuously access mental health support, even in the face of numerous restrictions and other safety measures imposed on in-person services, which disrupted or halted critical mental health services in 93% of countries worldwide (55).

However, the unprecedented and uncertain nature of the pandemic context might have hindered the effectiveness of these CBT chunks in reducing perceived stress. Individuals' responses to stress and coping mechanisms influence the difficulty of addressing specific stressors during public health emergencies (56); therefore, a one-size-fits-all approach may not be effective for everyone. Tailoring text messages to the recipient's needs and preferences could increase their efficacy (57). Text4Support is an example of a CBT-based text messaging program tailored to address specific psychological problems through theme-specific content focused on the management of symptoms of specific mental health concerns (e.g., depression, anxiety, bipolar disorders) and messages of self-care, social support, hope, affirmation, and recovery. Moreover, it is important to consider that the engagement in e-Mental Health interventions (i.e., text messaging, mobile apps, etc) tends to decrease over time (58, 59). The high attrition rates and poor rates of sustained engagement might be seen as a limitation—and it is, considering long-term impacts or severe mental health issues (60)—but during emergencies like pandemics, such interventions can be promptly delivered, significantly reducing mental health symptoms for many individuals (37, 61). As such, they may serve as a valuable low-intensity intervention tool, acting as a first step in a stepped-care approach to service delivery (62), until other mental health interventions can be implemented to address the broader population's needs. Conversely, it is noteworthy to mention the findings from a study on Text4Hope subscribers' satisfaction (48). The study revealed that the majority of respondents (1,531/1,716, 89.2%) always read the text messages after six weeks of receiving them, with about 23.4% (401/1,716) of respondents indicating that they always or often returned to read the text messages.

It is essential to consider that the results associated with the Text4Hope intervention may not be exclusively attributed to the CBT-based text message intervention. The Hawthorne Effect, a psychological phenomenon where individuals alter their behavior or performance when aware of being observed (63), may also have played a role. However, researchers can employ strategies to assess whether the Hawthorne Effect influenced a study. In our study, we have incorporated specific features in our design that allow us to make informed inferences about the potential impact of the Hawthorne Effect on the findings. For example, in the controlled cohort study, we compared data from individuals who were at the baseline stage (not yet exposed to the intervention) with the changes observed in the intervention group. The presence of significant differences in outcomes indicates that the CBT-based text messages likely had a distinct impact on the subscribers. Furthermore, it is important to note that the Hawthorne Effect often diminishes over time as individuals become accustomed to being observed (64). In our longitudinal analysis, we did not observe this diminishing effect, as the results remained consistent after 6 weeks. By incorporating multiple study designs, our aim was to provide a comprehensive view of the intervention's effects, taking into account both immediate impacts and potential influences, such as the Hawthorne Effect, over an extended period. This approach enhances our understanding of the intervention's impact and its underlying mechanisms.

Finally, statistically significant differences were found in the prevalence of sleep disturbance between the IG and the CG, although CBT is a safe and highly effective treatment for sleep problems (65). Interestingly, the results showed significant differences between groups for suicidal ideation (p < 0.001), as well as a protective impact against the perceived stress, likely depression and anxiety, and suicidal thoughts, despite the sociodemographic characteristics of the subjects at baseline. Studies have shown that text-based interventions can be effective in reducing suicidal ideation and increasing positive thinking, particularly for the highly distressed, young female population, including typically underserved minorities and a substantial percentage of individuals who do not receive help elsewhere (66, 67). While text-based crisis hotlines can provide immediate support to individuals in crisis, Text4Hope can provide accessible and convenient ongoing support to individuals who are at risk for suicidal thoughts and can reduce the likelihood of certain psychological symptoms (68).

Several limitations must be acknowledged in this study. Self-selection and dropouts could bias the study results if they relate to the examined outcomes. Text4Hope subscribers may have been more motivated to improve their mental health and more likely to respond positively to the intervention, (i.e., reading the text messages and responding to the follow-up surveys) than those who did not enroll. This may have led to an overestimation of the effect of Text4Hope on mental health outcomes. On the other hand, severe mental health symptoms could be a reason for dropout, leading to an underestimation of Text4Hope's impact on mental health outcomes. However, the reasons for dropouts are unknown, and it is possible that those who did not complete the surveys may also have benefited from the intervention. The high rate of dropouts resulted in a reduced sample size, raising valid concerns about the potential impact on the study results. However, it is worth noting that we used all available data from those who voluntarily participated in the assessment surveys, which provides valuable insights into the effectiveness of the intervention among those who engaged with the program. It should be noted that the results may not be generalizable to the broader population who did not respond to the evaluation surveys. Seasonality might also be affected by the results of this study. The lifting of lockdown measures and the decrease in case numbers during the summer of 2020, followed by a decline in mental health symptoms during the same period, may have influenced the impact of the intervention.

The controlled cohort design also has some limitations due to the real-world nature of the Text4Hope program, which made it impractical to have a separate group of individuals without the intervention. Randomization and withholding the intervention were ethically inappropriate, as it could harm those in need of support during the pandemic. Despite potential bias, this design reflects the real-world setting and offers valuable insights into the intervention's effectiveness. The data collected during the same two-month period for both groups allow us to reasonably attribute any observed score differences to the intervention. Lastly, the corrections employed in the data analysis may not have adequately addressed violations of the normality assumption. However, this violation apparently has not impacted the produced results upon using parametric analysis (t-test). This is due to the large sample size included in this study which yields no problem when using this type of test. This implies that parametric procedures can be used even when the data are not normally distributed (69, 70). The major strength of this study was the robustness of its design, which included a controlled cohort that allowed the investigators to control for the impact of the natural course of the pandemic on mental health outcomes, and for self-selection bias, demonstrating that the intervention was effective regardless of these limitations.

In considering particularities of women's health, we acknowledge the potential impact of sensitive periods, such as pregnancy, puerperium, or menopause on mental health symptoms in the female population. Therefore, further research is needed to explore the interplay between women's health and mental health outcomes during sensitive periods.

In conclusion, the findings of this study demonstrate the effectiveness of Text4Hope, a CBT-based daily supportive text messaging program, in improving mental health outcomes among women during the COVID-19 pandemic. E-mental health interventions, including text messaging programs like Text4Hope, have the potential to address the unique challenges and disparities in women's mental health by providing accessible mental health support. However, it is important to consider the gender gaps in technology usage and women's specific usage patterns before implementing strategies to increase uptake and maximize the benefits of such interventions.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by University of Alberta Health Research Ethics Board (protocol code Pro00086163 approved on March 18, 2020). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

Conceptualization, funding, acquisition, investigation, project administration, and supervision: VA. Data curation: RS, WV, AG, SS and VA. Formal analysis and methodology: RS and VA. Writing original draft: RLD. Writing review and editing: RLD, RS, BA, WV, AG, SS, AJG and VA. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by grants from the Mental Health Foundation, the Calgary Health Trust, the University Hospital Foundation, the Alberta Children’s Hospital Foundation, the Royal Alexandra Hospital Foundation, the Alberta Cancer Foundation, Alberta Health Services, and the University of Alberta. The funders had no role in the design and conduct of the study, collection, management, analysis, interpretation of the data, preparation, review, and approval of the manuscript, or the decision to submit the results for publication.

Acknowledgment

Marianne Hrabok contributed to the drafting of Text4Hope program messages.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Thibaut F, van Wijngaarden-Cremers PJ. Women’s mental health in the time of COVID-19 pandemic. Front Global Women’s Health. (2020) 1:588372. doi: 10.3389/fgwh.2020.588372

CrossRef Full Text | Google Scholar

2. Dobson KG, Vigod SN, Mustard C, Smith PM. Trends in the prevalence of depression and anxiety disorders among working-age Canadian adults between 2000 and 2016. Health Rep. (2020) 31(12):12–23. doi: 10.25318/82-003-x202001200002-eng

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Canadian Mental Health Association (British Columbia). Depression (2013). Available at: https://www.cmha.bc.ca/documents/depression-2/

4. Statistics Canada. Canadian Vital Statistics—Death Database. 2017–2019. Released in January, 2022.

5. American Foundation for Suicide Prevention. Suicide among women: a critical review. Women’s Health. (2022) 59(2):125–34.

Google Scholar

6. American Psychological Association. Stress in America: Gender and stress. American Psychological Association. Available at: http://www. apa. org/news/press/releases/stress/2010/gender-stress.aspx (Accessed on March 31, 2023).

7. Statistics Canada. Table 13-10-0451-01 Health indicators, annual estimates, 2003–2014. doi: 10.25318/1310045101-eng.

8. Chaput JP, Yau J, Rao DP, Morin CM. Prevalence of insomnia for Canadians aged 6 to 79. Health Rep. (2018) 29(12):16–21.30566205

PubMed Abstract | Google Scholar

9. Schweizer-Schubert S, Gordon JL, Eisenlohr-Moul TA, Meltzer-Brody S, Schmalenberger KM, Slopien R, et al. Steroid hormone sensitivity in reproductive mood disorders: on the role of the GABAA receptor complex and stress during hormonal transitions. Front Med (Lausanne). (2021) 7:479646. doi: 10.3389/fmed.2020.479646

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Schweizer-Schubert S. Editorial: endocrinological and social moderators of emotional well-being during perimenstrual, perinatal and perimenopausal transitions: what women want for sexual health and smooth hormonal changes. Front Psychol. (2022) 12:813291. doi: 10.3389/fpsyg.2021.813291

CrossRef Full Text | Google Scholar

11. Capaldi DM, Knoble NB, Wu Shortt J, Kim HK. A systematic review of risk factors for intimate partner violence. Partner Abuse. (2012) 3:231–80. doi: 10.1891/1946-6560.3.2.231

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Turcotte M. Women in Canada: A gender-based statistical report: women and health. Canada: Statistics Canada (2011).

13. Galea S, Merchant RM, Lurie N. The mental health consequences of COVID-19 and physical distancing: the need for prevention and early intervention. JAMA Intern Med. (2020) 180(6):817–8. doi: 10.1001/jamainternmed.2020.1562

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Liu N, Zhang F, Wei C, Jia Y, Shang Z, Sun L, et al. Prevalence and predictors of PTSS during COVID-19 outbreak in China hardest-hit areas: gender differences matter. Psychiatry Res. (2020) 287:112921. doi: 10.1016/j.psychres.2020.112921

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Guo Q, Zheng Y, Shi J, Wang J, Li G, Li C, et al. Immediate psychological distress in quarantined patients with COVID-19 and its association with peripheral inflammation: a mixed-method study. Brain Behav Immun. (2020) 88:17–27. doi: 10.1016/j.bbi.2020.05.038

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Solomou I, Constantinidou F. Prevalence and predictors of anxiety and depression symptoms during the COVID-19 pandemic and compliance with precautionary measures: age and sex matter. Int J Environ Res Public Health. (2020) 17(14):4924. doi: 10.3390/ijerph17144924

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Boniol M, McIsaac M, Xu L, Wuliji T, Diallo K, Campbell J. Gender equity in the health workforce: analysis of 104 countries. Working paper 1. Geneva: World Health Organization (2019).

18. Seedat S, Rondon M. Women’s wellbeing and the burden of unpaid work. Br Med J. (2021) 374:n1972. doi: 10.1136/bmj.n1972

CrossRef Full Text | Google Scholar

19. Lin SL. Generalized anxiety disorder during COVID-19 in Canada: gender-specific association of COVID-19 misinformation exposure, precarious employment, and health behavior change. J Affect Disord. (2022) 302:280–92. doi: 10.1016/j.jad.2022.01.100

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Gao J, Zheng P, Jia Y, Chen H, Mao Y, Chen S, et al. Mental health problems and social media exposure during COVID-19 outbreak. PLOS ONE. (2020) 15(4):e0231924. doi: 10.1371/journal.pone.0231924

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Fullana MA, Hidalgo-Mazzei D, Vieta E, Radua J. Coping behaviors associated with decreased anxiety and depressive symptoms during the COVID-19 pandemic and lockdown. J Affect Disord. (2020) 275:80–1. doi: 10.1016/j.jad.2020.06.027

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Mazza M, Marano G, Lai C, Janiri L, Sani G. Danger in danger: interpersonal violence during COVID-19 quarantine. Psychiatry Res. (2020) 289:113046. doi: 10.1016/j.psychres.2020.1130463046

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Roesch E, Amin A, Gupta J, García-Moreno C. Violence against women during COVID-19 pandemic restrictions. Br Med J. (2020) 369:m1712. doi: 10.1136/bmj.m1712

CrossRef Full Text | Google Scholar

24. Cerna-Turoff I, Fischer H-T, Mayhew S, Devries K. Violence against children and natural disasters: a systematic review and meta-analysis of quantitative evidence. PLoS One. (2019) 14(5). doi: 10.1371/journal.pone.0217719

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Shalaby R, Adu MK, El Gindi HM, Agyapong VIO. Text messages in the field of mental health: rapid review of the reviews. Front Psychiatry. (2022) 13:921982. doi: 10.3389/fpsyt.2022.921982

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Lattie EG, Stiles-Shields C, Graham AK. An overview of and recommendations for more accessible digital mental health services. Nat Rev Psychol. (2022) 1(2):87–100. doi: 10.1038/s44159-021-00003-1

CrossRef Full Text | Google Scholar

27. Berrouiguet S, Baca-García E, Brandt S, Walter M, Courtet P. Fundamentals for future mobile-health (mHealth): a systematic review of mobile phone and web-based text messaging in mental health. J Med Internet Res. (2016) 18:e135. doi: 10.2196/jmir.5066

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Liverpool S, Mota CP, Sales CMD, Čuš A, Carletto S, Hancheva C, et al. Engaging children and young people in digital mental health interventions: systematic review of modes of delivery, facilitators, and barriers. J Med Internet Res. (2020) 22:e16317. doi: 10.2196/16317

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Musiat P, Tarrier N. Collateral outcomes in e-mental health: a systematic review of the evidence for added benefits of computerized cognitive behavior therapy interventions for mental health. Psychol Med. (2014) 44(15):3137–50. doi: 10.1017/S0033291714000245

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Fordham B, Sugavanam T, Edwards K, Hemming K, Howick J, Copsey B, et al. Cognitive–behavioural therapy for a variety of conditions: an overview of systematic reviews and panoramic meta-analysis. Health Technol Assess (Rockv). (2021) 25:1–378. doi: 10.3310/hta25090

CrossRef Full Text | Google Scholar

31. Komariah M, Amirah S, Faisal EG, Prayogo SA, Maulana S, Platini H, et al. Efficacy of internet-based cognitive behavioral therapy for depression and anxiety among global population during the COVID-19 pandemic: a systematic review and meta-analysis of a randomized controlled trial study. Healthcare. (2022) 10:1224. doi: 10.3390/healthcare10071224

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Mrklas K, Shalaby R, Hrabok M, Gusnowski A, Vuong W, Surood S, et al. Prevalence of perceived stress, anxiety, depression, and obsessive-compulsive symptoms in health care workers and other workers in Alberta during the COVID-19 pandemic: cross-sectional survey. JMIR Ment Health. (2020) 7(9):e22408. doi: 10.2196/22408

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Garrido S, Millington C, Cheers D, Boydell K, Schubert E, Meade T, et al. What works and what doesn’t work? A systematic review of digital mental health interventions for depression and anxiety in young people. Front Psychiatry. (2019) 10:759. doi: 10.3389/fpsyt.2019.00759

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Agyapong VI, Hrabok M, Vuong W, Gusnowski A, Shalaby R, Mrklas K, et al. Closing the psychological treatment gap during the COVID-19 pandemic with a supportive text messaging program: protocol for implementation and evaluation. JMIR Res Protoc. (2020) 9(6). doi: 10.2196/19292

CrossRef Full Text | Google Scholar

35. ResilienceNHope: ResilienceNHope Programs. Available at: https://www.resiliencenhope.org/programs (Accessed 18 January 2023).

36. Agyapong VIO, Hrabok M, Vuong W, Shalaby R, Noble JM, Gusnowski A, et al. Changes in stress, anxiety, and depression levels of subscribers to a daily supportive text message program (Text4Hope) during the COVID-19 pandemic: cross-sectional survey study. JMIR Ment Health. (2020) 7:e22423. doi: 10.2196/22423

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Agyapong VIO, Shalaby R, Hrabok M, Vuong W, Noble JM, Gusnowski A, et al. Mental health outreach via supportive text messages during the COVID-19 pandemic: improved mental health and reduced suicidal ideation after six weeks in subscribers of Text4Hope compared to a control population. Int J Environ Res Public Health. (2021) 18:2157. doi: 10.3390/ijerph18042157

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Agyapong VIO, Hrabok M, Shalaby R, Vuong W, Noble JM, Gusnowski A, et al. Text4Hope: receiving daily supportive text messages for 3 months during the COVID-19 pandemic reduces stress, anxiety, and depression. Disaster Med Public Health Prep. (2022) 16:1326–30. doi: 10.1017/dmp.2021.27

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Agyapong B, Shalaby R, Vuong W, Gusnowski A, Surood S, Greenshaw AJ, et al. Text4Hope effectiveness in reducing psychological symptoms among young adults in Canada: longitudinal and naturalistic controlled program evaluation. J Clin Med. (2023) 12:1942. doi: 10.3390/jcm12051942

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Government of Alberta. COVID-19 Alberta statistics. Retrieved April 28, 2023, from Available at: https://www.alberta.ca/stats/covid-19-alberta-statistics.htm

41. Cohen S, Williamson G. Perceived stress in a probability sample of the United States. In: Spacapam S., Oskamp S., editors. The social psychology of health. Newbury Park, CA: Sage (1988). p. 31–67.

42. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. (2006) 16:1092–7. doi: 10.1001/archinte.166.10.1092

CrossRef Full Text | Google Scholar

43. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J General Int Med. (2001) 16:606–13. doi: 10.1046/j.1525-1497.2001.016009606.x

CrossRef Full Text | Google Scholar

44. Agyapong VIO, Juhás M, Ohinmaa A, Omeje J, Mrklas K, Suen VYM, et al. Randomized controlled pilot trial of supportive text messages for patients with depression. BMC Psychiatry. (2017) 17:286. doi: 10.1186/s12888-017-1448-2

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Agyapong VIO, Juhás M, Mrklas K, Hrabok M, Omeje J, Gladue I, et al. Randomized controlled pilot trial of supportive text messaging for alcohol use disorder patients. J Subst Abuse Treat. (2018) 94:74–80. doi: 10.1016/j.jsat.2018.08.014

PubMed Abstract | CrossRef Full Text | Google Scholar

46. IBM Corp. IBM SPSS statistics for windows (version 28.0) [computer software]. Armonk, NY: IBM Corp (2020).

47. Escoffery C. Gender similarities and differences for e-health behaviors among U.S. adults. Telemed J E Health. (2018) 24(5):335–43. doi: 10.1089/tmj.2017.0136

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Shalaby R, Vuong W, Hrabok M, Gusnowski A, Mrklas K, Li D, et al. Gender differences in satisfaction with a text messaging program (Text4Hope) and anticipated receptivity to technology-based health support during the COVID-19 pandemic: cross-sectional survey study. JMIR Mhealth Uhealth. (2021) 9(4):e24184. doi: 10.2196/24184

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Sugarman DE, Meyer LE, Reilly ME, Greenfield SF. Feasibility and acceptability of a web-based, gender-specific intervention for women with substance use disorders. J Women’s Health. (2020) 29:636–46. doi: 10.1089/jwh.2018.7519

CrossRef Full Text | Google Scholar

50. George AS, Morgan R, Larson E, LeFevre A. Gender dynamics in digital health: overcoming blind spots and biases to seize opportunities and responsibilities for transformative health systems. J Public Health (Bangkok). (2018) 40(suppl_2):ii6–ii11. doi: 10.1093/pubmed/fdy180

CrossRef Full Text | Google Scholar

51. Rowntree O. Connected women: the mobile gender gap report. London: GSMA (2018).

52. Mohan D, Bashingwa JJH, Tiffin N, Dhar D, Mulder N, George A, et al. Does having a mobile phone matter? Linking phone access among women to health in India: an exploratory analysis of the national family health survey. PLoS One. (2020) 15(7):e0236078. doi: 10.1371/journal.pone.0236078

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Belalcázar C. Mobile phones & literacy: empowerment in women’s hands: a cross-case analysis of nine experiences. Paris, France: Unesco/Ed Sector (2015).

54. Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, et al. Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med. (2018) 52(6):446–62. doi: 10.1007/s12160-016-9830-8

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Witkiewitz K, Marlatt GA. Relapse prevention for alcohol and drug problems: that was zen, this is tao. Am Psychol. (2004) 59(4):224–35. doi: 10.1037/0003-066X.59.4.224

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Brunier A, Drysdale C. COVID-19 disrupting mental health services in most countries, WHO survey. World Health Organ. (2020):2021–06.

Google Scholar

57. Wasil AR, Franzen RE, Gillespie S, Steinberg JS, Malhotra T, DeRubeis RJ. Commonly reported problems and coping strategies during the COVID-19 crisis: a survey of graduate and professional students. Front Psychol. (2021) 12:598557. doi: 10.3389/fpsyg.2021.598557

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Lecomte T, Potvin S, Corbière M, Guay S, Samson C, Cloutier B, et al. Mobile apps for mental health issues: meta-review of meta-analyses. JMIR Mhealth AUhealth. (2020) 8(5):e17458. doi: 10.2196/17458

CrossRef Full Text | Google Scholar

59. Kannisto KA, Korhonen J, Adams CE, Koivunen MH, Vahlberg T, Välimäki MA. Factors associated with dropout during recruitment and follow-up periods of a mHealth-based randomized controlled trial for mobile.net to encourage treatment adherence for people with serious mental health problems. J Med Internet Res. (2017) 19(2):e46. doi: 10.2196/jmir.6417

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Koh J, Tng GYQ, Hartanto A. Potential and pitfalls of mobile mental health apps in traditional treatment: an umbrella review. J Pers Med. (2022) 12(9):1376. doi: 10.3390/jpm12091376

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Staeck R, Drüge M, Albisser S, Watzke B. Acceptance of E-mental health interventions and its determinants among psychotherapists-in-training during the first phase of COVID-19. Internet Interv. (2022) 29:100555. doi: 10.1016/j.invent.2022.100555

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Marzano L, Bardill A, Fields B, Herd K, Veale D, Grey N, et al. The application of mHealth to mental health: opportunities and challenges. Lancet Psychiatry. (2015) 2(10):942–8. doi: 10.1016/S2215-0366(15)00268-0

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Parsons H. What happened at hawthorne? Science. (1974) 183:922–32. doi: 10.1126/science.183.4128.922

PubMed Abstract | CrossRef Full Text | Google Scholar

64. McCambridge J, Witton J, Elbourne DR. Systematic review of the hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. (2014) 67(3):267–77. doi: 10.1016/j.jclinepi.2013.08.015

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Noar SM, Harrington NG, Aldrich RS. The role of message tailoring in the development of persuasive health communication messages. Ann Int Commun Assoc. (2009) 33(1):73–133. doi: 10.1080/23808985.2009.11679085

CrossRef Full Text | Google Scholar

66. Trauer JM, Qian MY, Doyle JS, Rajaratnam SM, Cunnington D. Cognitive behavioral therapy for chronic insomnia: a systematic review and meta-analysis. Ann Intern Med. (2015) 163:191–204. doi: 10.7326/M14-2841

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Pisani AR, Gould MS, Gallo C, Ertefaie A, Kelberman C, Harrington D, et al. Individuals who text crisis text line: key characteristics and opportunities for suicide prevention. Suicide Life Threat Behav. (2022) 52(3):567–82. doi: 10.1111/sltb.12872

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Abdullah H, Lynch S, Aftab S, Shahar S, Klepacz L, Cristofano P, et al. Characteristics of calls to a COVID-19 mental health hotline in the first wave of the pandemic in New York. Community Ment Health J. (2021) 57(7):1252–4. doi: 10.1007/s10597-021-00868-9

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Ghasemi A, Zahediasl S. Normality tests for statistical analysis: a guide for non-statisticians. Int J Endocrinol Metab. (2012) 10(2):486–9. doi: 10.5812/ijem.3505

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Elliott AC, Woodward WA. Statistical analysis quick reference guidebook. SAGE Publications, Inc. (2007). doi: 10.4135/9781412985949