Marketplace and Literature Review of Spanish Language Mental Health Apps

Language differences between patients and providers remains a barrier to accessing health care, especially mental health services. One potential solution to reduce inequities for patients that speak different languages and improve their access to care is through the delivery of healthcare through mobile technology. Given that the Latinx community serves as the largest ethnic minority in the United States, this two-phased review examines Spanish app development, feasibility and efficacy. Phase 1 explored the commercial marketplace for apps available in Spanish, while phase 2 involved a literature review of published research centered around the creation, functions, and usability of these apps using the PubMed and Google Scholar electronic databases. Of the apps available on the database, only 14.5% of them had Spanish operability. The literature search uncovered 629 results, of which 12 research articles that tested or described 10 apps met the inclusion criteria. Of the 10 apps studied in this literature review, only four apps were translated to Spanish. Our study reveals that despite increasing interest in Spanish-language apps to address mental health, the commercial marketplace is not currently meeting the demand.

Introduction

Latinos are the largest ethnic minority in the United States and currently account for 18.5% of the US population (1). It is estimated that 21.3% or nearly 13 million Latinx individuals have at least one mental illness (1, 2). Yet, despite the great need for assessment and treatment of mental health illness in the Latinx population, data shows that only 9.6% of them accessed any mental health services (2). There is a growing body of literature that highlights the barriers to accessing mental health care, which includes low rates of insurance coverage, legal status, stigma, and socioeconomic factors. Language is a salient factor in accessing mental health care services. A study in 2013 showed that 32% or 15.7 million Latinx individuals report speaking English less than “very well.” (3) Language differences between patients and providers have been shown to be detrimental for effective communication, which can lead to lower quality of care, and poorer outcomes (4, 5). The imbalance between demand and supply is aggravated by the fact that only an estimated 4.0% of psychiatrists are Latinx (6).

One potential solution to reduce inequities for Spanish-speaking patients and improve access to care is through the delivery of healthcare through mobile health care (mHealth). Through the use of apps via mobile devices, patients can access a wealth of resources to improve their mental health. This change requires both access to a smartphone with internet capabilities and a desire of Latinx to use mHealth; both are true today. Research shows that 80% of Latinx individuals have access to the internet via a mobile device, thus making delivery of mHealth feasible (7, 8). Latinx individuals want mHealth as demonstrated in a 2016 study where over 85% of participants reported interest in using mobile apps to improve their health and at least a quarter stated they would use a mobile app for mental wellness (9).

Despite the demand for more mHealth for Spanish-speaking populations, it is unclear whether the commercial marketplace has met the increasing need and the extent of published literature on Spanish app development, feasibility and efficacy are unknown. The objectives of this U.S. app store marketplace and literature review are to in phase (1) review the number and characteristics of Spanish mental health apps available on the commercial market; in phase (2) review existing literature on the development, translation, and cultural adaptation of mental health apps to Spanish-language/culture as well as the feasibility and efficacy of these apps in the Spanish-speaking population; (3) synthesize the results from the first two phases to identify key challenges, opportunities, and recommendations for development, translation, and cultural adaptation of Spanish-language mental health apps.

Methods

Phase 1

This review was conducted in two phases. Phase 1 involved searching a database of commercial market for apps available in Spanish, while phase 2 consisted of a literature review of published research centered around the creation, functions, and usability of these apps. To determine the commercial market's availability of mental health apps in Spanish, we utilized the Mobile Health Index and Navigation (MIND) database published by The Division of Digital Psychiatry at BIDMC (available at apps.digitalpsych.org). The database currently reports the existence of 220 mental health apps in the commercial market and collects data on 105 objective questions set forth by the APA for each registered mental health app. It is a useful resource for this phase as it is the largest database of mental health apps supported by peer reviewed evidence, publicly accessible, and allows users to filter through apps based on personal preferences and priorities (10). We utilized this database and filtered results by “Spanish Functionality” to find all the apps that are currently available in Spanish and collected basic characteristics of these apps, including platform availabilities, supported conditions, engagement features, connection to other services, and supporting study availability, see Table 1.

TABLE 1

Table 1. Characteristics of mental health mobile apps in Spanish.

Phase 2

To understand the current state of mental health apps in Spanish, we created a searchable question to identify Spanish-language app development, feasibility, and efficacy. The developed search strategy was conducted on two electronic databases: PubMed and Google Scholar. Major themes searched included Spanish, Spanish-language, Spanish-speaking, Hispanic, and Latinx combined with health and mental health synonyms. This was joined to a list of applicable terms for the type of technology utilized, including mobile devices, smartphones, and apps. Given the fast-paced changes in technology, only articles published after January 2015 were included. To be included, papers had to meet the following criteria: (1) feature a Spanish-language mobile app, (2) describe the development, feasibility, or interventional approach of a mobile app, and (3) the app must address at least one mental health disorder including, but not limited to depression, anxiety, substance use disorder, and/or eating disorders. Articles were excluded if (1) published before January 2015, and (2) app does not address any mental health disorders.

The search revealed 629 articles. A review of the 629 citations/abstracts was conducted manually. The full text was considered for papers with abstracts unavailable. This initial screening resulted in 49 articles that met the preliminary inclusion criteria. A snowball approach was used to ensure the literature search was comprehensive. This involved reviewing papers that cited the 49 articles, searching for previous papers written by the lead authors, and utilizing the “related articles” feature on each database. Following a full-text review of the final set of papers, 12 met the comprehensive criteria and are reviewed here. There are no conflicts of interest identified by the authors of this study. All authors certify responsibility for the manuscript.

Results

Phase 1

A review of the Division of Digital Psychiatry's mHealtgh App Navigation Database (MIND) revealed that of 220 mental health apps available on the commercial market, 32 (14.5%) offer Spanish operability (10). Of these, 31 (96.9%) are available on iOS and 27 (84.4%) are available on Android. The most common supported conditions were stress and anxiety (n = 16, 50.0%) and mood disorders (n = 14, 43.8%). Significant overlap was noted with 12 (37.5%) apps supporting both mood and stress and anxiety conditions. Of the 32 apps, 10 (31.3%) have supporting studies published (11–32). However, a review of these studies revealed that none of them focused on Spanish-speaking populations. Further, none of the apps on the marketplace found in Phase 1 were found in the app mentioned in the literature review of Phase 2 as outlined below. Table 1 summarizes the most salient characteristics of these apps.

Phase 2

The literature search revealed 12 research articles that tested or described 10 apps. Among the 10 apps, only four distinct mental health disorders were addressed. Of the 12 articles, six described app development and/or cultural adaptation, three studied feasibility, and three described interventional studies. Most articles (n = 8, 66.7%) focused on Spanish-speaking Latino adults in the United States. The rest of the articles focused on Spanish-speaking individuals in Peru, Colombia, and Australia. Of the 10 apps described or studied, half (n = 5, 50.0%) aimed to address depressive symptoms. Table 2 displays the article and app characteristics including articles' research design and study population, as well as the apps' supported conditions. Of note, control apps not studied or reported on were not analyzed in Table 2.

TABLE 2

Table 2. Summary metrics of studies on mental health apps in Spanish.

App Development Protocol

The ¡Aptívate! app utilizes Brief Behavioral Activation (BBA) to address depressive symptoms by emphasizing that behaviors can influence mood and encourages participants to complete activities that align with self-selected values. To create this app, researchers started with the English app “Moodivate” and translated the app interface into Spanish utilizing a back-translation approach with the help of bilingual translators. Given that the app has patients self-select values and activities, cultural adaptation was not deemed necessary. However, the psychoeducation component of the app was carefully designed to avoid stigma in this population by de-emphasizing depressive symptoms as internal flaws and emphasizing the lack of environmental rewards as the etiology (33).

The Mental Health eClinic (MHeC) app was developed to address a broad range of mental health concerns and includes a triage system for those needing urgent help. The app was developed utilizing a participatory design methodology for Spanish-speaking youth in two different settings: international students in Australia and native youth in Colombia (34, 35). In a participatory design, stakeholders including patients, supportive others, and healthcare professionals provide their input in all six phases of app development. In both settings, all developmental phases that directly involved stakeholders were conducted in Spanish, eliminating the need for translation. In Australia, cultural adaptation included changing of the language question to specify Spanish dialects, changing the ethnicity question to reflect indigenous populations, and after initial disagreement, adoption the informal “tú” throughout the app (35). In Colombia, cultural adaptation included incorporating family structure and support networks, establishing credibility through university, health service provider, and community organization collaboration, and given the country's characteristics screening for economic stability, food security, and violence exposure (34).

The Automated Bilingual Computerized Alcohol Screening and Intervention (AB-CASI) mobile app was developed as the Spanish version of the emergency department-based alcohol screening, brief intervention, and referral to treatment (ED-SBIRT) program (36). The goal of the app is to address alcohol use disorders in Spanish without requiring extensive human resources, such as translators, in the emergency department. To culturally adapt this app, the researchers utilized user-centered design through design, development, and evaluation of app prototypes. This methodology ensures that stakeholders, namely patients and professionals, are involved in the development process. This app also addresses literacy issues through text-to-speech, which was found to be crucial for culturally adapting the app. Text to speech apps transform written text into audio. Researchers ultimately chose a Text-To-Speech app to help with this process. Additionally, this work emphasizes that beyond translation and cultural adaptation, it is necessary to adapt health apps to the context in which they will be used. In this instance, the app had to be designed for use in an emergency department, therefore the app must have capability to save progress and start/stop/pause.

The Ecological Momentary Assessment (EMA) app was created to understand unhealthy eating and weight control behaviors of Mexican American women with low literacy (37). The app aims to collect information on these activities repeatedly in its natural context without having to rely on memory. Original studies of this app were geared toward college-enrolled women, but in contrast this study focused on women with low health literacy, requiring a shift from written components to pictures, icons, and sound features. App development was achieved by utilizing a user-centered methodology which involved the end-users at all four stages of development. Through this work, researchers found that: (1) text-to-icon translation (words are translated into images) was more complex than anticipated given the discrepancy in definitions of unhealthy eating behaviors between participants and researchers; (2) participants described forms of weight control products as opposed to their intended effects (ex: diet pill as opposed to laxative); (3) icons were found to be too complex to use to collect context and mood, therefore this component was not included in the final app.

Finally, Muñoz et al. submitted a study protocol for the design, development, and evaluation of the San Francisco Stop Smoking app which aims to help Latinos stop smoking (38). Although results of this study are pending, the protocol describes the human-centered methodology utilized to create this app. To achieve cultural adaptation, the first phase of development will require field observations of potential app users. These observations will give researchers first-hand information about how Latinos use their phones, how they interact with apps, and ultimately help researchers understand what app features might best serve the end-users. Researchers will also conduct workshops with Spanish speaking patient in which they will design an ideal app. This will also give researchers direct feedback on specific features that might be needed for the app to work for Spanish-speakers trying to quit smoking.

Feasibility

CONEMO is a nurse-supported app that utilizes behavioral activation to reduce depressive symptoms in patients with diabetes, hypertension, or both (39). The study by Lena Brandt et al. combines two feasibility pilot studies conducted in Lima, Peru to test the feasibility of: (1) implementing the app in two healthcare systems in Peru; (2) scaling up the app-based intervention. The study had 29 participants (mean age 60) utilize the app for 6 weeks, receiving three sessions per week. Semi-structured interviews revealed the app provided several health benefits, including reduction of stress and increase in motivation, and the majority were satisfied with the app. One major barrier was usability, with at least 72% of participants reporting some difficulties in using the app, though these subsided with longer use, and self-reported adherence was 50%. Participants also suggested the addition of audio. The study employed six nurses to support patients with app use and semi-structured interviews revealed that although they felt this was an innovative and helpful intervention, integration of CONEMO and daily responsibilities was challenging. Overall, CONEMO was found to be a feasible intervention.

CONEMO researchers also performed a composite study comprised of two pilot studies in Lima, Peru and one pilot study in São Paulo, Brazil with the dual goal of exploring the effectiveness of the CONEMO app and the feasibility to conduct a large randomized-control trial (39). The study enrolled 66 participants across the three sites. Data was collected actively: patients filled out a baseline PHQ-9, were then given access to CONEMO for 6 weeks, and filled a post-intervention PHQ-9, as well as passively: CONEMO system collected information such as sessions accessed and missed and interval between session access. Results of this study showed a general decrease in depressive symptoms based on decreasing PHQ-9 scores in 65–87% of participants, depending on the site, and a reduction in levels of functional disability. Challenges of this interventional study included difficulty with recruitment given that many patients were unable to read or write and a decline in session access over the course of the study.

Pratap et al. conducted a 3 month study to assess the feasibility of conducting a fully remote randomized controlled trial to screen for, assess, and treat depression in Latino individuals utilizing one of three apps to improve depressive symptoms: (1) EVO which uses therapeutic games, (2) iPST which employs psychotherapy principles, and (3) HTips which suggests mindfulness and behavioral exercises (40). The three apps were translated from English to Spanish by native Spanish speakers and professionals at Babble-on. The study enrolled 1,180 participants, but only 359 participated. Overall, feasibility of using mobile apps to remotely assess and treat depression was confirmed although the major challenges included a quick decline in app engagement and the higher cost and effort necessary to recruit Hispanic participants compared to non-Hispanic participants.

Intervention

Based on their feasibility findings, Pratap et al. then conducted a 3 month remote interventional study to compare recruitment and engagement of Hispanic and non-Hispanic participants and to compare treatment outcomes when participants utilized one of three apps: EVO, iPST, and HTips (42). The study remotely recruited and enrolled 1,020 participants, 389 of whom were Latino and 637 of whom were non-Latino. Participants were randomized to use one of the three apps for 3 months and PHQ-9 scores were collected at baseline and every week for the duration of the study. The study showed that PHQ-9 scores and self-reported disability scores decreased throughout the study without differences in recovery between Hispanics and non-Hispanics or by app used. The major challenges continued to be engagement, with Hispanic participants stopping the study nearly 2 weeks earlier than their counterparts and the high cost, high effort of enrolling Hispanic participants.

In this second interventional study (44), participants were recruited locally and nationally and randomized to one of three conditions for 8 weeks to address depressive symptoms: (1) ¡Aptívate! an app that uses behavioral activation, (2) iCouch which uses cognitive behavioral principles, and (3) treatment as usual, no app. The goals were to understand feasibility and efficacy of the ¡Aptívate! app. A total of 42 participants were enrolled (n = 22 ¡Aptívate!, n = 9 iCouch, n = 11 no app) and they self-reported app usage and completed the Spanish language Beck Depression Inventory-II (BDI-II) weekly. ¡Aptívate! is a self-help-based app and iCouch offers CBT. Results demonstrated lower depressive symptoms over time in those using the ¡Aptívate! app compared to treatment as usual, but no significant differences were found between ¡Aptívate! users and iCouch or between iCouch and treatment as usual. Challenges in this study included difficulties with local recruitment in South Carolina, although app engagement was higher in this group compared to nationally recruited participants.

In the third interventional study, researchers aimed to examine the mental health outcomes for Latinx Spanish-speaking patients with alcohol and other drug use (AOD) disorders with use of the CASA-CHESS mobile app for 8 weeks (43). This app was designed by translating and culturally adapting the theory-informed A-CHESS app into Spanish. The study enrolled 79 participants who were post-residential treatment for AOD, given a phone with the app, and followed for 6 months. Results show AOD symptoms for those that used the app for 4 months or longer were more stable with less use of illicit drugs, lower depressive and anxiety symptoms at 6 months compared to those who used the app for <4 months. Overall, this interventional study showed that an app can be an effective tool to provide continuity care for Spanish-speaking Latinos post-residential treatment. A major limitation of this work is the lack of app control or comparison group.

Discussion

This review of the commercial market shows that there are many mental health apps available, but only a limited amount (14.5%) are offered in Spanish and none have conducted effectiveness studies with Spanish-speaking individuals. This problem is not unique to mental health apps. A study on apps for diabetes shows promise as it found that 30% (28/92) on the Android and Apple stores were in Spanish. However, when researchers investigated the Spanish readability of these apps, they found it was well above recommended reading levels, essentially making them inaccessible to many end-users (45). Although apps in Spanish may be growing in number, app developers must ensure that future apps are customizable, usable, and effective for the end-user populations.

Of the 10 apps studied in this literature review, only 4 apps (EVO, iPST, HTips, ¡Aptívate!) were translated (33, 42). This is likely as the others were developed in collaboration with Spanish speakers from their inception. However, Spanish-speakers in the United States are of course themselves diverse and thus cultural adaptation of apps must also consider: (1) end-user characteristics including nationality, locality, dialect, literacy level, socioeconomic status, (2) end-users' understanding of targeted conditions and associated stigmas, and (3) environments in which apps will be used and customizable features responsive to those environments. Of course, in many countries Spanish speakers are the majority and unique considerations for each culture, region, and clinical need must be considered.

User-centered design can help address most aspects of translation and cultural adaptation during the development phase, but it also requires intensive efforts and heavy upfront investment in recruitment, data collection, and usability testing. Another technique to achieve these goals is the utilization of cultural brokers to culturally adapt apps, as in the case of “Visit Planner,” an app aimed at helping Spanish-speaking patients prepare for their primary care appointments (46). However, this methodology could still prove challenging in some communities where cultural brokers may are not easily identifiable. Community collaboration and trust may be necessary preconditions to access and work with these individuals. Overall, the literature shows that translation is necessary, but not sufficient to guarantee the usability and effectiveness of an app. Developers and researchers should aim to use user-centered development techniques when possible, but more work on viable alternatives that require less investment for app translation and cultural adaptation are needed.

Feasibility studies have demonstrated that it is possible to implement mental health apps to treat depressive symptoms in Spanish-speaking patients in different healthcare systems and that it is possible to conduct studies that enroll large numbers of Spanish-speaking patients, even when done remotely. However, we interpret these studies show there are three major barriers researchers have to contend with: (1) quick decline in app engagement, (2) app usability for some segments of the population, and (3) Increased expense and labor required to recruit Spanish-speaking participants.

User-centered design has been utilized as a method to empower potential end-users to contribute to app development and thus try to counteract usability and engagement concerns (25). The three feasibility studies discussed in this review did not describe the use of a user-centered design which may have contributed to a quick decline in app engagement. However, the studies also show that to counter engagement issues, app use should be paired with primary care, be accompanied by supportive nursing, or include features that help patients connect to mental health services. Nursing or other support staff, such a promotoras, can also help address usability issues in populations who may need extra technical help. Recruitment challenges may be multifaceted and may therefore require dynamic solutions. Work by Stuart Winter et al. describe best practices for including minorities in research including improving incentives, partnering with key community organizations, understanding the most used and effective methods of communication, addressing issues of distrust, and adding cultural brokers to the research team (26). Further work is needed to assess optimal incentives for Spanish-speaking populations and identify barriers to engagement in remote mental health studies.

Early data from interventional studies identified in this review demonstrate that mental health apps in Spanish may be effective tools to assess and treat depressive symptoms and help patients remain stable post-residential treatment for alcohol and AOD. Though the published literature on intervention studies is nascent, they show promising evidence that mobile health interventions can be successful for Spanish-speaking patients. This is particularly important given that these interventions will improve access to care for many patients in their native language, address the stigma associated with mental health conditions, and ultimately help close the gap on mental health disparities for Spanish-speaking patients. However, in the past 5 years few studies in the literature have appeared that more work needs to be done to increase research in this area. More apps need to be translated, culturally adapted and tested for feasibility and usability. Future efforts for the 32 apps available in Spanish identified in the market review should involve testing their effectiveness in this population. From these investigations, simple guidelines could be created to help patients, primary care providers, and/or mental health providers select the most appropriate app for each individual based on their characteristics, health literacy level, environment of use, personal preferences, and mental health condition.

Although the number of mental health apps available in Spanish is growing, translation of these apps alone may not be enough to improve access to these tools. A recent study of Medicare patients suggests that 38% of Spanish speakers do not have access to a smartphone with a data plan for wireless internet, precluding app use for many (27). A less visible but prevalent barrier to access is the lack of comfort with technology. The digital divide related to access to smartphones has also evolved into a lack of digital skills to navigate technology (28). Thus, Spanish-language apps are only as accessible as the users' digital literacy and comfort. While beyond the scope of this paper, we have seen this in our clinical work and as we have work to create a Spanish version of our teams' own open source mental health app (47, 48). To combat this, we have developed Digital Opportunities for Outcomes in Recovery Services (DOORS) to help people gain the core competencies and functional skills necessary to utilize digital health tools effectively (49). While DOORS is currently available in English, efforts to create partners and a team to build a Spanish version are underway. Future Spanish-language app development should consider the literacy levels of users and ponder what level of technical competency is necessary to utilize the tool.

As with any study, this review has several weaknesses that must be addressed. First, the market search was conducted on a database with 220 apps which is the largest database of mental health apps, however app stores host thousands of mental health-related apps. Thus, the percentage of apps available in Spanish may not be indicative of the entire landscape. Second, no single search term can discover all applicable articles on this topic. These results may also be influenced by publication bias due to apps or studies that may exist but have yet to be published. Third, while this was a U.S. based study in terms of the app marketplaces reviews and thus results may not generalizable to other countries and Spanish speakers, we note we did review research from around the world. Fourth, we looked at papers from the last 5 years which covers the majority of app research but will have missed earlier works.

Conclusion

Our study reveals that despite increasing interest in Spanish-language apps to address mental health, the commercial marketplace is not currently meeting the demand. None of the apps we reviewed on the marketplace were found in the academic literature, reflecting a gap between research and commercially available apps today. User-centered development design emerged as a leading strategy to increase the number of apps that are linguistically and culturally adapted to the Spanish-speaking population. Interventional studies show promising evidence that apps can help address mental health condition if challenges surrounding engagement and app usability are overcome. This study highlights that through thoughtful design and development, apps may hold the key to reducing mental health disparities in Spanish-speaking Hispanics in the United States.

Author Contributions

JT and AM conceptualized the study. EC and AM reviewed the literature. AM led the paper drafting. All authors discussed paper selection, edited multiple drafts, and reviewed the final paper.

Conflict of Interest

JT reports unrelated research support from Otsuka.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Abbreviations

mHealth, mobile health care; apps, mobile applications; AOD, alcohol and other drug use.

References

1. U.S. Census Bureau QuickFacts: United States. Available online at: https://www.census.gov/quickfacts/fact/table/US/RHI725218 (accessed April 27, 2020).

2. Lee S, and Held ML. Variation in mental health service use among U.S. Latinos by place of origin and service provider type. Psychiatr Serv. (2015) 66:56–64. doi: 10.1176/appi.ps.201300533

PubMed Abstract | CrossRef Full Text | Google Scholar

3. English Use on the Rise Among Latinos. Pew Research Center's Hispanic Trends Project. Available online at: https://www.pewresearch.org/hispanic/2015/05/12/english-proficiency-on-the-rise-among-latinos/ (accessed August 13, 2020).

4. Sudore RL, Landefeld CS, Pérez-Stable EJ, Bibbins-Domingo K, Williams BA, and Schillinger D. Unraveling the relationship between literacy, language proficiency, and patient–physician communication. Patient Educ Couns. (2009) 75:398–402. doi: 10.1016/j.pec.2009.02.019

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Schyve PM. Language differences as a barrier to quality and safety in health care: the joint commission perspective. J Gen Intern Med. (2007) 22(Suppl. 2):360–1. doi: 10.1007/s11606-007-0365-3

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Lokko HN, Chen JA, Parekh RI, and Stern TA. Racial and ethnic diversity in the US psychiatric workforce: a perspective and recommendations. Acad Psychiatry. (2016) 40:898–904. doi: 10.1007/s40596-016-0591-2

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Hispanics and Mobile Access to the Internet. Pew Research Center's Hispanic Trends Project. Available online at: https://www.pewresearch.org/hispanic/2016/07/20/3-hispanics-and-mobile-access-to-the-internet/ (accessed August 23, 2020).

8. Smartphones Help Blacks, Hispanics Bridge Some – but not all – Digital Gaps With Whites. Pew Research Center. Available online at: https://www.pewresearch.org/fact-tank/2019/08/20/smartphones-help-blacks-hispanics-bridge-some-but-not-all-digital-gaps-with-whites/ (accessed August 23, 2020).

9. Ramirez V, Johnson E, Gonzalez C, Ramirez V, Rubino B, and Rossetti G. Assessing the use of mobile health technology by patients: an observational study in primary care clinics. JMIR Mhealth Uhealth. (2016) 4:e41. doi: 10.2196/mhealth.4928

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Lagan S, Aquino P, Emerson MR, Fortuna K, Walker R, and Torous J. Actionable health app evaluation: translating expert frameworks into objective metrics. npj Dig Med. (2020) 3:1–8. doi: 10.1038/s41746-020-00312-4

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Roncero M, Belloch A, and Doron G. A novel approach to challenging OCD related beliefs using a mobile-app: an exploratory study. J Behav Ther Exp Psychiatry. (2018) 59:157–60. doi: 10.1016/j.jbtep.2018.01.008

CrossRef Full Text | Google Scholar

12. Bush NE, Dobscha SK, Crumpton R, Denneson LM, Hoffman JE, Crain A, et al. A virtual hope box smartphone app as an accessory to therapy: proof-of-concept in a clinical sample of veterans. Suicide Life Threat Behav. (2015) 45:1–9. doi: 10.1111/sltb.12103

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Flett JAM, Conner TS, Riordan BC, Patterson T, and Hayne H. App-based mindfulness meditation for psychological distress and adjustment to college in incoming university students: a pragmatic, randomised, waitlist-controlled trial. Psychol Health. (2020) 35:1049–74. doi: 10.1080/08870446.2019.1711089

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Roncero M, Belloch A, and Doron G. Can brief, daily training using a mobile app help change maladaptive beliefs? Crossover randomized controlled trial. JMIR Mhealth Uhealth. (2019) 7:e11443. doi: 10.2196/11443

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Mayseless N. Can intervention programs influence how the dyslexic brain processes low-level visual stimuli? Dev Neuropsychol. (2011) 36:949–54. doi: 10.1080/87565641.2011.606421

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Huberty J, Puzia M, Eckert R, and Larkey L. Cancer patients' and survivors' perceptions of the calm app: cross-sectional descriptive study. JMIR Cancer. (2020) 6:e16926. doi: 10.2196/16926

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Huberty J, Vranceanu A-M, Carney C, Breus M, Gordon M, and Puzia ME. Characteristics and usage patterns among 12,151 paid subscribers of the calm meditation app: cross-sectional survey. JMIR Mhealth Uhealth. (2019) 7:e15648. doi: 10.2196/15648

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Haimov I, Hanuka E, and Horowitz Y. Chronic insomnia and cognitive functioning among older adults. Behav Sleep Med. (2008) 6:32–54. doi: 10.1080/15402000701796080

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Bloom R, Schnaider-Beeri M, Ravona-Springer R, Heymann A, Dabush H, Bar L, et al. Computerized cognitive training for older diabetic adults at risk of dementia: study protocol for a randomized controlled trial. Alzheimers Dement. (2017) 3:636–50. doi: 10.1016/j.trci.2017.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Hunter JF, Olah MS, Williams AL, Parks AC, and Pressman SD. Effect of brief biofeedback via a smartphone app on stress recovery: randomized experimental study. JMIR Serious Games. (2019) 7:e15974. doi: 10.2196/15974

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Puzia M, Laird B, Green J, and Huberty J. Parents' perceptions of their children's engagement in a consumer-based meditation mobile app: cross-sectional survey study. JMIR Pediatr Parent. (2020) 3:e24536. doi: 10.2196/24536

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Huberty J, Green J, Glissmann C, Larkey L, Puzia M, and Lee C. Efficacy of the mindfulness meditation mobile app “calm” to reduce stress among college students: randomized controlled trial. JMIR Mhealth Uhealth. (2019) 7:e14273. doi: 10.2196/14273

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Huberty J, Eckert R, Larkey L, Joeman L, and Mesa R. Experiences of using a consumer-based mobile meditation app to improve fatigue in myeloproliferative patients: qualitative study. JMIR Cancer. (2019) 5:e14292. doi: 10.2196/14292

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Owen JE, Jaworski BK, Kuhn E, Makin-Byrd KN, Ramsey KM, and Hoffman JE. mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD coach. JMIR Mental Health. (2015) 2:e7. doi: 10.2196/mental.3935

PubMed Abstract | CrossRef Full Text

25. Mistler LA, Ben-Zeev D, Carpenter-Song E, Brunette MF, and Friedman MJ. Mobile mindfulness intervention on an acute psychiatric unit: feasibility and acceptability study. JMIR Ment Health. (2017) 4:e34. doi: 10.2196/mental.7717

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Kuhn E, Greene C, Hoffman J, Nguyen T, Wald L, Schmidt J, et al. Preliminary evaluation of PTSD coach, a smartphone app for post-traumatic stress symptoms. Milit Med. (2014) 179:12–8. doi: 10.7205/MILMED-D-13-00271

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Carpenter J, Crutchley P, Zilca RD, Schwartz HA, Smith LK, Cobb AM, et al. Seeing the “big” picture: big data methods for exploring relationships between usage, language, and outcome in internet intervention data. J Med Internet Res. (2016) 18:e241. doi: 10.2196/jmir.5725

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Huberty J, Eckert R, Larkey L, Kurka J, De Jesús SA, Yoo W, et al. Smartphone-based meditation for myeloproliferative neoplasm patients: feasibility study to inform future trials. JMIR Form Res. (2019) 3:e12662. doi: 10.2196/12662

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Parks AC, Williams AL, Kackloudis GM, Stafford JL, Boucher EM, and Honomichl RD. The effects of a digital well-being intervention on patients with chronic conditions: observational study. J Med Internet Res. (2020) 22:e16211. doi: 10.2196/16211

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Champion L, Economides M, and Chandler C. The efficacy of a brief app-based mindfulness intervention on psychosocial outcomes in healthy adults: a pilot randomised controlled trial. PLoS ONE. (2018) 13:e0209482. doi: 10.1371/journal.pone.0209482

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Denneson LM, Smolenski DJ, Bauer BW, Dobscha SK, and Bush NE. The mediating role of coping self-efficacy in hope box use and suicidal ideation severity. Arch Suicide Res. (2019) 23:234–46. doi: 10.1080/13811118.2018.1456383

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Possemato K, Kuhn E, Johnson E, Hoffman JE, Owen JE, Kanuri N, et al. Using PTSD coach in primary care with and without clinician support: a pilot randomized controlled trial. Gen Hosp Psychiatry. (2016) 38:94–8. doi: 10.1016/j.genhosppsych.2015.09.005

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Dahne J, Collado A, Lejuez CW, Risco C, Diaz VA, Kustanowitz J, et al. ¡Aptívate!: a Spanish-language behavioral activation mobile application for delivery via primary care. Psychol Serv. (2019) 16:271–5. doi: 10.1037/ser0000304

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Ospina-Pinillos L, Davenport TA, Navarro-Mancilla AA, Cheng VW, Alarcón AC, Rangel AM, et al. Involving end users in adapting a Spanish version of a web-based mental health clinic for young people in Colombia: exploratory study using participatory design methodologies. JMIR Ment Health. (2020) 7:e15914. doi: 10.2196/15914

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Ospina-Pinillos L, Davenport T, Mendoza Diaz A, Navarro-Mancilla A, Scott EM, and Hickie IB. Using participatory design methodologies to co-design and culturally adapt the Spanish version of the mental health eclinic: qualitative study. J Med Internet Res. (2019) 21:e14127. doi: 10.2196/14127

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Abujarad F, and Vaca FE. mHealth tool for alcohol use disorders among Latinos in emergency department. Proc Int Symp Hum Factors Ergon Healthc. (2015) 4:12–9. doi: 10.1177/2327857915041005

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Connelly K, Stein KF, Chaudry B, and Trabold N. Development of an ecological momentary assessment mobile app for a low-literacy, Mexican American population to collect disordered eating behaviors. JMIR Public Health Surveill. (2016) 2:e31. doi: 10.2196/publichealth.5511

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Muñoz RF, Bunge EL, Barrera AZ, Wickham RE, and Lee J. Using behavioral intervention technologies to help low-income and Latino smokers quit: protocol of a randomized controlled trial. JMIR Res Protoc. (2016) 5:e127. doi: 10.2196/resprot.5355

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Menezes P, Quayle J, Garcia Claro H, Da Silva S, Brandt LR, Diez-Canseco F, et al. Use of a mobile phone app to treat depression comorbid with hypertension or diabetes: a pilot study in Brazil and Peru. JMIR Ment Health. (2019) 6:e11698. doi: 10.2196/11698

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Pratap A, Anguera JA, Renn BN, Neto EC, Volponi J, Mooney SD, et al. The feasibility of using smartphones to assess and remediate depression in hispanic/latino individuals nationally. In: Lonini L, Shawen N, Ghaffari R, Rogers J, Jayarman A, editors. Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers. Maui (2017). p. 854–60.

Google Scholar

41. Brandt LR, Hidalgo L, Diez-Canseco F, Araya R, Mohr DC, Menezes PR, et al. Addressing depression comorbid with diabetes or hypertension in resource-poor settings: a qualitative study about user perception of a nurse-supported smartphone app in Peru. JMIR Ment Health. (2019) 6:e11701. doi: 10.2196/11701

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Pratap A, Renn BN, Volponi J, Mooney SD, Gazzaley A, Arean PA, et al. Using mobile apps to assess and treat depression in Hispanic and Latino populations: fully remote randomized clinical trial. J Med Internet Res. (2018) 20:e10130. doi: 10.2196/10130

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Muroff J, Robinson W, Chassler D, López LM, Lundgren L, Guauque C, et al. An outcome study of the CASA-CHESS smartphone relapse prevention tool for Latinx Spanish-speakers with substance use disorders. Subst Use Misuse. (2019) 54:1438–49. doi: 10.1080/10826084.2019.1585457

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Dahne J, Collado A, Lejuez CW, Risco CM, Diaz VA, Coles L, et al. Pilot randomized controlled trial of a Spanish-language behavioral activation mobile app (¡Aptívate!) for the treatment of depressive symptoms among United States Latinx adults with limited English proficiency. J Affect Disord. (2019) 250:210–7. doi: 10.1016/j.jad.2019.03.009

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Rodriguez JA, and Singh K. The Spanish availability and readability of diabetes apps. J Diabetes Sci Technol. (2018) 12:719–24. doi: 10.1177/1932296817749610

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Ruvalcaba D, Nagao Peck H, Lyles C, Uratsu CS, Escobar PR, and Grant RW. Translating/creating a culturally responsive spanish-language mobile app for visit preparation: case study of “trans-creation.” JMIR Mhealth Uhealth. (2019) 7:e12457. doi: 10.2196/12457

CrossRef Full Text | Google Scholar

47. Torous J, and Vaidyam A. Multiple uses of app instead of using multiple apps–a case for rethinking the digital health technology toolbox. Epidemiol Psychiatr Sci. (2020) 29:e100. doi: 10.1017/S2045796020000013

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Rodriguez-Villa E, Rauseo-Ricupero N, Camacho E, Wisniewski H, Keshavan M, et al. The digital clinic: implementing technology and augmenting care for mental health. Gen Hosp Psychiatry. (2020). doi: 10.1016/j.genhosppsych.2020.06.009. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Rodriguez-Villa E, Camacho E, and Torous J. Psychiatric rehabilitation through teaching smartphone skills to improve functional outcomes in serious mental illness. Internet Interv. (2021) 23:100366. doi: 10.1016/j.invent.2021.100366

PubMed Abstract | CrossRef Full Text | Google Scholar