Augmenting Mental Health in Primary Care: A 1-Year Study of Deploying Smartphone Apps in a Multi-site Primary Care/Behavioral Health Integration Program

Background: Integrating behavioral health (BH) services into primary care is an evidence-based intervention that can increase access to care, improve patient outcomes, and decrease costs. Digital technology, including smartphone apps, has the potential to augment and extend the reach of these integrated behavioral health services through self-management support impacting lifestyle behaviors. To date, the feasibility and acceptability of using mental health mobile apps within an integrated primary care setting has not yet been explored as part of routine clinical care.

Objectives: The objectives of this study were to (a) test the feasibility of using mental health applications to augment integrated primary care services; (b) solicit feedback from patients and providers to guide implementation, and (c) develop a mental health apps toolkit for system-wide dissemination.

Methods: Cambridge Health Alliance (CHA) is a safety-net healthcare system that includes three community hospitals and 12 Primary Care (PC) clinics serving nearly 150,000 ethnically and socioeconomically diverse patients around Boston. To select and disseminate mental health apps, a four-phase implementation was undertaken: (1) Evaluation of mental health mobile applications (2) Development of an apps toolkit with stakeholder input, (3) Conducting initial pilot at six primary care locations, and (4) Rolling out the app toolkit across 12 primary care sites and conducting 1-year follow-up survey.

Results: Among BH providers, 24 (75%) responded to the follow-up survey and 19 (83%) indicated they use apps as part of their clinical care. Anxiety was the most common condition for which app use was recommended by providers, and 10 (42%) expressed interest in further developing their knowledge of mental health apps. Among patients, 35 (65%) of participants provided feedback; 23 (66%) reported the tools to be helpful, especially for managing stress and anxiety.

Conclusions: Our findings indicate mental health apps are applicable and relevant to patients within integrated primary care settings in safety-net health systems. Behavioral health providers perceive the clinical value of using these tools as part of patient care, but require training to increase their comfort-level and confidence applying these tools with patients. To increase provider and patient engagement, mobile apps must be accessible, simple, intuitive and directly relevant to patients' treatment needs.

Introduction

Integrating behavioral health care services into primary care is an evidence-based intervention that can increase access to care, improve patient outcomes, and decrease costs (1). The need for integrated behavioral health (BH) is driven by the heavy burden of mental health conditions, including depression, which is now recognized by the World Health Organization as the leading global cause of disability (2), as well as insufficient numbers of specialized behavioral health clinicians like social workers, psychiatrists, psychologists, and case managers. Key elements of integrated care models include screening for high prevalence mental health conditions, integration of behavioral health clinicians into primary care settings, collaborative team building, and care management for patients (3). With ~40% of mental health care in the United States delivered in the primary care setting, the demand for integrated BH services has outpaced availability (4).

Like other areas of healthcare, digital technology has been proposed as a means to augment and extend the reach of BH integrated care services. These technologies can be classified into three categories: (1) patient facing, (2) primary care facing, and (3) virtual visits (5). Patient facing technologies to extend integrated care may include smartphone apps that offer self-management or guided interventions like Cognitive Behavioral Therapy. Primary care facing technologies include tools to support increased capacity and confidence to offer mental health treatment such as consultation platforms, tele-mentoring, and clinical decision support for integrated care teams. Virtual visits reflect the ability to offer telemedicine services to support integrated care. While there is a growing evidence base and clinical experience with using both virtual visits and primary care facing technologies (5) there is limited to no research or clinical evidence for using smartphone apps to support integrated care.

The lack of knowledge on incorporating mental health-related smartphone apps into integrated care systems is especially surprising considering the potential of mobile technology in mental health. Numerous survey studies have demonstrated those with mental health conditions own smartphones and are interested in using apps for their care (6–8). Meta-analysis of randomized clinical studies of smartphone apps for depressive symptoms (9) and anxiety disorders (10) have demonstrated early efficacy data. In many cases, consumers with mental health conditions are not waiting for the evidence to catch up; today over 10,000 mental health related apps are available from the iTunes and Android app stores (11). Case reports and clinical experience support that people are using these apps to learn about their mental health, track symptoms, and self-manage their condition (12, 13). However, exploring the utility of these apps outside of research studies remains largely uncharted.

Prior efforts transitioning promising mental health research apps into actual clinical care settings underscores the need for careful assessment. One substance use disorder app that showed promising evidence to reduce frequency of alcohol consumption (14) was later found to be difficult to support and maintain in actual clinical settings without the additional support and structure offered in the research setting (15). Reviews of smartphone app adoption by healthcare professionals, beyond mental health and integrated care settings, suggest that perceived barriers can include concerns about privacy, legal ramifications, cost, workload, and need for increased information technology (IT) support (16). Despite barriers, there is evidence that apps are being introduced to care. For instance, research from the United Kingdom suggests that already 30% of mental health providers recommend apps to patients (17). On the patient side, there is also concern that many commercially available mental health apps may not be designed for those with more severe symptoms (18).

In this study, we sought to provide early clinical data from both the patient and clinician perspective on the usage and integration of mental health apps in a busy integrated BH care service at Cambridge Health Alliance (CHA), a safety net healthcare system that includes three community hospitals and 12 Primary Care (PC) clinics which serve nearly 150,000 ethnically and socioeconomically diverse patients in the greater Boston area (19). The healthcare system had implemented an integrated model of care, based on the collaborative care model (20) and Screening, Brief Intervention and Referral to Treatment (SBIRT) (21), to support screening, assessment, and treatment of behavioral health conditions in primary care clinics. The integrated BH staff provide evidence-based brief clinical interventions focusing on patients primarily experiencing low to moderate anxiety, depression, and substance use disorders. Consult psychiatrists provide diagnostic clarification and management strategies for the primary care team. Integrated psychotherapists provide brief treatment (6–8 sessions) grounded in evidence-based treatment modalities including Cognitive Behavioral Therapy (CBT), Problem-Solving Therapy (PST), Behavioral Activation (BA), and Mindfulness Based Stress Reduction (MBSR). Care managers work primarily with patients experiencing low to moderate anxiety and depression; they use interventions including Motivational Interviewing (MI) and BA to support patients in reaching health and lifestyle-related goals and to bridge patients to BH providers. To answer the question of whether mental health apps can be adopted within routine BH care and serve as a helpful tool for patients, we present the results of one institution's experience evaluating, piloting, and disseminating mental health apps across an integrated primary care system.

Methods

Step 1: Selecting Apps

The first step in introducing apps into care was to identify a series of existing apps that could be recommended to patients receiving integrated care services. Given that the vast majority of these apps fall outside of the Food and Drug Administration (FDA) regulation and there is no formal evaluation or gold standard for judging these apps, we assembled a seven-person multi-disciplinary team of integrated therapists, care managers, administrative and Informational Technology (IT) leadership, and patients to identify and evaluate mental health self-management apps over the course of 10 months. We sought apps that offered patient facing interventions, avoided direct use with a clinician, were free, and claimed to provide privacy protections for patient generated data—understanding there is no set of gold standard apps or criteria to pick apps. Thus, we focused on apps that were: (1) aligned with clinical interventions utilized in our program; (2) developed by a “trusted” source such as medical, academic, and/or research institutions, and/or companies with clinical consultants; (3) free to download, and (4) contained privacy statements at the time of our evaluation. Drawing from our workgroup's clinical knowledge and experience caring for CHA's safety-net patient population, we also considered apps' accessibility in terms of our patients' language needs, literacy levels, and alignment with treatment goals (22). In seeking such apps, we read select literature to help orient us to broader findings and challenges in the mHealth space (23), consulted with app researchers and designers, and sought feedback from our clinical teams and the patients they serve. Without validated app evaluation tools to guide our selection process, we relied on expert consensus throughout the four-stage study process to select a set of apps that best supported our clinical interventions and the needs of our patient population.

Step 2: Creating a Toolkit Around Apps

We identified an initial list of 13 applications and sought guidance from our institution's Patient Partner Lead who represents CHA's patient partners, a group of patient volunteers who serve on quality improvement teams to advise on key projects. Drawing on feedback from our Patient Partner Lead, staff member expertise, papers on app evaluation (24), and elements of the user version of the Mobile Apps Rating System (U-MARS) (25), the workgroup next narrowed the list to nine apps and drafted a process for introducing the apps through the Primary Care Behavioral Health Integration program (PCBHI) in Primary Care. In drafting the process for introducing these apps to patients, we aimed to incorporate several elements: (a) suggestions to staff as to when to introduce the apps; (b) appropriate language for staff to use in discussing the apps with patients, and (c) brief descriptions of the apps in patient-facing materials. The evaluation team and Patient Partner Lead co-designed a set of tools for introducing the BH apps, including a patient-facing summary sheet describing the risks/benefits of using smartphone applications and a mobile apps “toolkit” that contained descriptions of the purpose and main features of the nine selected apps. We then trained six care managers to identify patients who might benefit from using mobile apps and introduce the tools to patients (including clinical rationale, product demonstration, goal setting, and plan for follow-up). Care managers were given additional training around discussing app privacy/security with patients and reviewed all points detailed within the patient-facing summary sheet. As a quality improvement project to improve access to high quality self-management tools within primary care, this implementation study was not subject to formal approval by our Institutional Review Board as per Cambridge Health Alliance's ethics guidelines and national regulations. Before introducing the selected mobile apps to patients, care managers obtained informed verbal and written consent from all pilot participants and no personally identifiable information was collected during the course of the pilot.

Step 3: Initial Pilot Testing

Before disseminating the apps toolkit across our integrated care system, the team conducted pilot-testing over 2 months at the six primary care clinics where the trained care managers were located. During the pilot, patients were referred to care managers by consult psychiatrists, integrated therapists, and primary care providers through “warm handoffs” or the real-time transfer of patients from one provider to another. When demonstrating mobile applications, care managers used iPads available in the clinic or their own smartphones (with appropriate security measures in place). After ~2–4 weeks, care managers followed up with patients either in person or by phone to conduct structured interviews in which patients were asked the following questions: Did you use the app? Did you find it to be helpful? Do you have any additional comments?

Following the formal conclusion of the pilot, care managers continued to gather feedback related to patients' experience over the course of 1 year. Preliminary themes were identified through review of all available data, several group discussions leading to consensus in our multidisciplinary team, and updated with data gathered in the larger roll out as described below.

Following the pilot test, we evaluated patient and care manager feedback and adapted procedures to roll out the program to all 12 of our primary-care practices. Our final toolkit contained seven apps as we removed two apps from the list of nine used during the pilot. As smartphone apps are dynamic and always changing, during this phase of the study new information about a smoking app became available that led our team to substitute for one from the pilot—reflecting the real-world challenge of using apps in health care (see Figure 1).

FIGURE 1

Figure 1. App selection across stages of the study. *Quit guide replaced quitStart with post-pilot input from CHA's tobacco treatment team.

Step 4: Staff Training and App Dissemination

To facilitate use of apps by staff, we wrote two electronic medical record (EMR) standardized “smart phrases” which could be incorporated into patient-facing after-visit summary documents. These “smart phrases” contained the list of recommended self-management apps and the patient-facing summary sheet explaining risks/benefits of using mobile apps. Through a series of staff meetings, we trained all integrated BH staff (therapists, consult psychiatrists, care managers) on best practices for utilizing mental health apps within clinical care. Because of the pilot testing and early staff engagement in the project, many were already familiar with the basic premise of the project and supportive of its goals.

One year after staff training and dissemination of CHA's mobile apps toolkit across all 12 primary care clinics, we followed up by sending a survey to integrated BH staff. In this survey, we asked questions exploring (a) staff utilization of apps; (b) perceived clinical impact, and (c) experience introducing apps to patients.

Results

Quantitative Patient Experience

Over the 2-month duration of the pilot intervention, care managers introduced mental health apps to 56 adult patients. Fifty-four (96.4%) patients agreed to use the tools while 2 (3.5%) patients declined (1 due to security concerns and 1 due to time constraints). Patients in the pilot were 18–70 years old with a mean age of 36.5 years, and 37 (66.1%) were female. English was not the language of medical care for 6 (10.7%) patients; 4 (7.1%) primarily spoke Portuguese and 2 (3.5%) primarily spoke Spanish. The tools were introduced to 35 (62.5%) patients for anxiety and stress-related issues and to 20 (35.7%) patients for depression. Eight (14.2%) patients received apps to target sleep issues and 3 (5.3%) patients were introduced to apps for other issues. Thirteen (23.2%) of the 56 patients were introduced to the tools for multiple conditions/issues.

The care managers were unable to follow-up with 19 patients during the 2-month pilot period. Of the 35 (64.8%) patients who provided feedback on usage of the tools, 7 (20%) patients were unable to use the tools (3 due to equipment limitations, 4 for other reasons). Twenty-three (65.7%) of patients who provided feedback found the apps to be helpful. Twenty-four (68.5%) of patients reported using Breathe2Relax and 24 (68.5%) reported using Self-Help for Anxiety Management (SAM). Patients used Breathe2Relax to help them sleep, to manage anxiety, and to prepare for stressful or mentally taxing situations. When using SAM, patients tended to access just a few of the features that focused on symptom relief, education about anxiety, strategies for managing strong emotions, and mood tracking. (Figure 2)

FIGURE 2

Figure 2. Screenshots of Breathe2Relax (Left two) and Self-Help for Anxiety Management (Right two).

Qualitative Patient Experience

After the 2-month pilot and over the course of the following year, care managers collected general feedback from patients that is presented in Table 1 as a combination of paraphrasing and direct quotes. Review of this feedback by our multidisciplinary team and discussion yielded consensus around 7 broad themes: ease of use, educational value, role in self-monitoring and enhancing self-awareness, relaxation/mindfulness, ability to incorporate into daily life, data and security concerns, and value of staff role in encouraging engagement with mental health apps.

TABLE 1

Table 1. Key themes and illustrative quotes from the 1 year study.

Quantitative Staff Experience

After 1 year of the intervention (which involved 32 integrated BH staff), 24 (75%) completed the follow-up survey. Within the integrated BH team roles, nine (100%) care managers, 12 (75%) psychotherapists, and 3 (43%) consult psychiatrists responded. The number of responses to each survey item varied.

Twenty-three (95.8%) BH staff members answered the survey question regarding their utilization of mobile tools with patients; nineteen people (82.6%) indicated that they incorporate BH apps into their clinical work. Five (25%) BH staff members out of 20 indicated that they introduce apps to patients 25–50% of the time, and 9 (45%) introduce mobile tools <25% of the time. When using apps with patients, the condition addressed most often by BH staff was anxiety 20 (83.3%), followed by stress 18 (75%), depression 14 (58.3%), alcohol use 6 (25%), and tobacco use 6 (25%). Of the 23 (95.8%) staff members who reported on their experience using the CHA's mobile app toolkit, the apps most commonly used were Breathe2Relax, Mindfulness Coach, and Self-Help for Anxiety Management (see Table 2 below). The survey also assessed techniques and strategies staff employed in introducing apps to patients: 19 (79.1%) reported discussing access/interest/ability, 14 (58.3%) discussed clinical basis for recommending that app and how it connects with the patient's goals, 14 (58.3%) demonstrated how to use the app, 13 (54.1%) helped the patient download the app, 13 (54.1%) followed up and discussed app use with patients at later visits, 12 (50%) set a follow up plan for app use, 9 (37.5%) discussed security and privacy risk associated with app use, and 7 (29.1%) created an action plan around app use.

TABLE 2

Table 2. Apps most frequently recommended by BH staff.

BH staff found a variety of benefits and challenges to incorporating apps into clinical care (see Table 3) and 10 out of 24 BH staff (42%) expressed a need for more practice and training on using each tool within the CHA's mobile app toolkit.

TABLE 3

Table 3. Staff views on the benefits and challenges of utilizing BH apps.

Qualitative Staff Experience

Consult psychiatrists, therapists, and care managers were also asked to provide qualitative/free-text feedback on apps in the 1 year follow-up survey. Representative responses for the three most utilized apps are presented below in Table 4 alongside patient feedback. Feedback concerning clinical use, accessibility, patient engagement/motivation, and other areas is presented below in Table 5.

TABLE 4

Table 4. Top three apps utilized by staff during the 1 year study (as reported through the staff survey).

TABLE 5

Table 5. Organizing themes and illustrative quotes from staff survey.

Discussion

Our results of piloting the use of smartphone apps in a primary care setting with integrated BH staff demonstrate the feasibility, opportunity, and challenges of introducing mobile technologies into formal care settings. Through our program of mobile app evaluation, pilot-testing, and 1 year long local dissemination into routine practice we demonstrated that mental health apps can be introduced to patients of varying ages, literacy levels, and English-language proficiency. On the patient side, our results suggest that patients found the mobile tools helpful in learning about their mental health conditions, practicing relaxation skills, and monitoring mood symptoms. Barriers to use included technical limitations (e.g., lack of smartphone, limited data), difficulty incorporating into daily life, low motivation, and limited mental health app options for non-English speaking patients. On the clinical side, members of the integrated care teams were receptive to apps but used them with patients <50% of the time—with chief barriers including lack of time to discuss apps and being unfamiliar or uncomfortable with apps.

While our study results reflect the experience at a single healthcare system in an urban environment, several lessons learned in our experience are generalizable. Reflecting on both the successes as well as areas for further improvement in our development and rollout of apps for integrated care teams offers insights for others seeking to build their own app libraries, develop app toolkits, onboard staff, and engage patients.

Selecting Apps and the Need to Update

Selecting appropriate apps to recommend for use in integrated care is a core element in a successful rollout, but also a moving target as these apps are constantly updating and evolving. Relying on websites or services that claim to curate apps and score them based on certain metrics is not helpful, as such scores do not take into account the unique needs of intended users (e.g., lower literacy levels), and do not reflect the simple reality that apps frequently update and change. We found that gathering a multidisciplinary panel to search, evaluate, and test apps was useful and practical. Such a panel should remain an ongoing and integral part of any app rollout and ideally meet at least quarterly to continue to update and evaluate new apps and new evidence. In terms of searching for new apps to support integrated care, we noted that apps selected in our search focused on helping patients learn more about their mental health conditions, enhance self-awareness, develop skills related to distress tolerance, and improve anxiety management. While we realize that today's mental health apps offer functionalities that can allow for real time monitoring and facilitate communication with the treatment team, such functionalities raise a number of complex questions related to risk/privacy/liability that could not be addressed within the scope of this quality improvement project.

Building a Toolkit and the Need for More Hands-On Training or a New Digital Navigator

Although the app toolkit that we developed to help the clinical teams introduce, integrate, and support app use was critical to success, it was not sufficient in and of itself. Offering the clinical team a list of apps, core information and talking points on each app, and electronic medical record smart phrases to document app usage was key to the feasibility of the intervention. Clinical staff recommended apps that had a clear purpose, simple user-interface, and were easy to navigate and demonstrate within a brief period of time. When introducing Breathe2Relax, for example, staff were often able to review the technique of diaphragmatic breathing, help patients to personalize settings, and practice the app's guided breathing exercise in under 10 min. Mental health apps that have a clear purpose and that are easy to review during staff trainings may enhance BH staff confidence and efficiency in introducing these tools to patients and facilitate higher adoption across the primary care system.

However, we found that the clinical teams sought additional support in understanding how each app worked, in demonstrating apps to patients, and having enough time in clinical visits to discuss an app. This suggests two possible responses: either (a) further hands-on training for clinical staff or (b) the introduction of a new member of the care team in the form of a digital navigator. While there is little evidence to support optimal teaching strategies for app use, we propose that hands on training involving role playing introducing and using apps with patients may offer an effective means to help the clinical team feel more confident in using apps in care settings. Another option would be to refer patients to a new member of the care team called a digital navigator—much as we did in the initial pilot study with the care manager—who is an expert at setting up, teaching, and supporting use of apps. This is akin to referring a patient to a pharmacist on the team to learn more about medication or dietician to craft a customized healthy eating plan.

Supporting Patients With Apps and Ensuring Apps Are Accessible

Patients in our study found apps useful and preferred those with clear purpose that had an engaging interface, were intuitive to use, and easy to navigate. The preference for simplicity aligns with findings within the field of user experience and design (26) as well as emerging research (27, 28) suggesting that apps containing a limited number of concepts and activities may be more appealing than those with a number of features. Apps containing limited text and simple/concrete language demand less from a cognitive standpoint (29) and may prove most accessible if patients are turning to apps during times of emotional distress when their cognitive abilities are compromised. Of note, the most popular feature within our mobile apps “toolkit” was “Picture Peace” a grounding exercise within Self-Help for Anxiety Management (SAM) that involves swiping across a blank screen to reveal a nature photo beneath. Patients described this feature as being “fun” and chose to use it over other exercises that were less visually pleasing and that demanded more from a cognitive standpoint.

However, the lack of apps available in multiple languages was also a limiting factor that made it impossible to use apps with some patients who may have benefited from this intervention. Several patients were also unable to run apps on their phone because of issues installing the apps or lacking sufficient memory on their smartphone to run the apps.

Connecting to Care Through Apps

Our results also suggest new clinical applications for mental health apps beyond those studied and implemented in our study. Within our integrated primary care model, care managers often serve as a “bridge” to longer-term BH care, providing education about BH conditions, teaching basic distress-tolerance skills, and focusing on brief interventions related to lifestyle behavior change. In using mental health apps, patients can begin to develop greater insight into their thoughts, feelings, and behaviors, an essential component of any therapeutic treatment intervention. Providing patients with self-management apps while they are waiting for in-person BH services may also help patients to stay connected to their care team and cultivate a sense of “emotional holding” during this bridging period (29). These apps may also spread awareness and interest in BH services, as evidenced by multiple patients who shared a recommended app with friends and family because they thought it may be useful to them as well.

These same apps may help introduce patients to BH treatment who would have otherwise not been interested or declined to engage because of stigma, discomfort or misunderstanding about the field and treatments offered. Thus, the potential of population health enabled through apps and digital technology will remain a focus of our continued research. While our study did not explore potential cost-savings associated with our intervention, we plan to investigate the financial implications of digitally augmented integrated BH care in future studies.

Limitations

Like all studies, ours has several limitations. As a real-world implementation study that sought to understand feasibility of app selection, uptake, and introduction into integrated care clinics—we were unable to control for many variables including patient severity, disease state, and clinician participation among others. In considering our patients' feedback that was not incorporated into a formal qualitative framework, we recognize that having care managers solicit feedback directly from patients may have influenced patient self-report and the subsequent interpretation of this data. Also given how often apps update and change—we realize that our results are difficult for others to replicate especially as we tailored our research to the local needs of the patients we serve. While we believe our study included a representative sample, it is difficult to quantify the number of types of patients, or clinicians, who are willing to engage and use apps as part of care. Further, we realize there are currently no gold standard apps, gold standard methods to select apps, or gold standard tools to quantify the impact of apps on care. Thus, we hope that the process and results of this present research can help guide others in taking the first steps to study the best ways to introduce digital technologies like apps into their clinical settings.

Conclusions

Although there are barriers to implementing digital technologies like smartphone apps into integrated BH settings, with careful planning and stakeholder engagement they can be useful tools in real-world clinical settings. While apps alone are not a panacea to addressing limited access to mental health care and lifestyle interventions, when deployed thoughtfully, they can help extend care to support patients with mild and moderate behavioral health conditions on a personal level and communities on a population level.

Author Contributions

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

Funding

JT is supported by a Research Fellowship from the American Psychiatric Association Foundation and a Career Development Award from the NIMH: 1K23MH116130-01.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. World Health Organization World Organization of National Colleges Academies Academic Associations of General Practitioners/Family Physicians. Integrating Mental Health into Primary Care: A Global Perspective. New York, NY: World Health Organization (2008).

2. http://www.who.int/en/news-room/fact-sheets/detail/depression

3. Joseph R, Kester R, O'Brien C, Huang H. The evolving practice of psychiatry in the era of integrated care. Psychosomatics (2017) 58:466–73. doi: 10.1016/j.psym.2017.04.003

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Wang PS, Demler O, Olfson M, Pincus HA, Wells KB, Kessler RC. Changing profiles of service sectors used for mental health care in the United States American. J Psychiatry (2006) 163:1187–98. doi: 10.1176/ajp.2006.163.7.1187

CrossRef Full Text | Google Scholar

5. Raney L, Bergman D, Torous J, Hasselberg M. Digitally driven integrated primary care and behavioral health: how technology can expand access to effective treatment. Curr Psychiatry Rep. (2017) 19:86. doi: 10.1007/s11920-017-0838-y

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Torous J, Chan SR, Tan SY, Behrens J, Mathew I, Conrad EJ, et al. Patient smartphone ownership and interest in mobile apps to monitor symptoms of mental health conditions: a survey in four geographically distinct psychiatric clinics. JMIR Ment Health (2014) 1:e5. doi: 10.2196/mental.4004

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Chan S, Godwin H, Gonzalez A, Yellowlees PM, Hilty DM. Review of use and integration of mobile apps into psychiatric treatments. Curr Psychiatry Rep. (2017) 19:96. doi: 10.1007/s11920-017-0848-9

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Anthes E. Pocket psychiatry: mobile mental-health apps have exploded onto the market, but few have been thoroughly tested. Nature (2016) 532:20–4. doi: 10.1038/532020a

CrossRef Full Text | Google Scholar

9. Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. (2017). The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 16:287–98. doi: 10.1002/wps.20472

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? a meta-analysis of randomized controlled trials. J Affect Disord. (2017) 218:15–22. doi: 10.1016/j.jad.2017.04.046

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Torous J, Roberts LW. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry (2017) 74:437–8. doi: 10.1001/jamapsychiatry.2017.0262

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Sandoval LR, Torous J, Keshavan MS. Smartphones for smarter care? self-management in schizophrenia. Am J Psychiatry (2017) 174:725–8. doi: 10.1176/appi.ajp.2017.16090990

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Torous J, Roux S. Patient-driven innovation for mobile mental health technology: case report of symptom tracking in schizophrenia. JMIR Ment Health (2017) 4:e27. doi: 10.2196/mental.7911

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Gustafson DH, McTavish FM, Chih MY, Atwood AK, Johnson RA, Boyle MG, et al. A smartphone application to support recovery from alcoholism: a randomized clinical trial. JAMA Psychiatry (2014) 71:566–72. doi: 10.1001/jamapsychiatry.2013.4642

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Ford II JH, Alagoz E, Dinauer S, Johnson KA, Pe-Romashko K, Gustafson DH. Successful organizational strategies to sustain use of A-CHESS: a mobile intervention for individuals with alcohol use disorders. J Med Internet Res. (2015) 17:e201. doi: 10.2196/jmir.3965

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Gagnon MP, Ngangue P, Payne-Gagnon J, Desmartis M. m-Health adoption by healthcare professionals: a systematic review. J Am Med Informat Assoc. (2015) 23:212–20. doi: 10.1093/jamia/ocv052

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Bennion MR, Hardy G, Moore RK, Millings A. E-therapies in England for stress, anxiety or depression: what is being used in the NHS? A survey of mental health services. BMJ Open (2017) 7:e014844. doi: 10.1136/bmjopen-2016-014844

CrossRef Full Text | Google Scholar

18. Sarkar U, Gourley GI, Lyles CR, Tieu L, Clarity C, Newmark L, et al. Usability of commercially available mobile applications for diverse patients. J Gen Int Med. (2016) 31:1417–26. doi: 10.1007/s11606-016-3771-6

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Zallman L, Joseph R, O'Brien C, Benedetto E, Grossman E, Arsenault L, et al. (2017). Does behavioral health integration improve primary care providers' perceptions of health-care system functioning and their own knowledge? Gen Hosp Psychiatry 46:88–93. doi: 10.1016/j.genhosppsych.2017.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Archer J, Bower P, Gilbody S, Lovell K, Richards D, Gask L, et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev. (2012):10:CD006525. doi: 10.1002/14651858.CD006525.pub2

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Babor TF, McRee BG, Kassebaum PA, Grimaldi PL, Ahmed K, Bray J. Screening, brief intervention, and referral to treatment (SBIRT) toward a public health approach to the management of substance abuse. Subst Abuse (2007) 28:7–30. doi: 10.1300/J465v28n03_03

PubMed Abstract | CrossRef Full Text | Google Scholar

22. East ML, Havard BC. Mental health mobile apps: from infusion to diffusion in the mental health social system. JMIR Ment Health (2015) 2:e10. doi: 10.2196/mental.3954

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Torous J, Firth J, Huckvale K, Larsen ME, Cosco TD, Carney R, et al. The emerging imperative for a consensus approach toward the rating and clinical recommendation of mental health apps. J Nerv Ment Dis. (2018) 206:662–6 doi: 10.1097/NMD.0000000000000864

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Chan S, Torous J, Hinton L, Yellowlees P. (2015). Brief Communication: toward a framework for evaluating mobile mental health apps. Telemed J E Health 21:1038–41 doi: 10.1089/tmj.2015.0002

CrossRef Full Text | Google Scholar

25. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the user version of the mobile application rating scale (uMARS). JMIR Mhealth Uhealth (2016) 4:e72. doi: 10.2196/mhealth.5849

PubMed Abstract | CrossRef Full Text | Google Scholar

26. https://www.nngroup.com/articles/simplicity-vs-choice/?lm=outcomes-vs-features&pt=article

27. Mohr DC, Tomasino KN, Lattie EG, Palac HL, Kwasny MJ, Weingardt K, et al. (2017). Intellicare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J Med Internet Res. 19:1–14. doi: 10.2196/jmir.6645

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Bakker D, Kazantzis N, Rickwood D, Rickard N. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Ment Health (2016) 3:e7 doi: 10.2196/mental.4984

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Topham P, Matthews P. Project SAM: Developing an App to Provide Self-Help for Anxiety PART 1: REPORT (2015). Available online at: http://eprints.uwe.ac.uk/27231