Sylvie Bernaerts1*
Tom Van Daele1,2
Christian Korthé Carlsen3
Søren Lange Nielsen3Jolanda Schaap4
Yvette Roke4
- 1Psychology and Technology, Centre of Expertise Care and Well-Being, Thomas More University of Applied Sciences, Antwerp, Belgium
- 2Centre for Technological Innovation, Mental Health and Education, Queen's University Belfast, Belfast, United Kingdom
- 3Centre for Digital Psychiatry, Region of Southern Denmark, Odense, Denmark
- 4Expertise Center for Autism Spectrum Disorder, GGZ Centraal, Almere, Netherlands
Introduction
Over the past decades, the importance of mental health is increasingly being acknowledged, with more people reaching out for help. However, mental healthcare struggles to help all in need. Those finding their way to formal services face long waiting lists, while for others, the associated stigma is still too large to reach out. Both cases result in unmet needs, which remain a pressing issue. One attempt to overcome these challenges is to rely on digital mental health, the use of technology for mental health interventions, ranging from promotion, prevention, and treatment to maintenance. Technologies can for example include computers and smartphones, extended reality, wearables, social media, chatbots (1) and may or may not make use of artificial intelligence. A wealth of evidence already supports the efficacy and effectiveness of online interventions for common mental health conditions, such as depression and anxiety in older adults (2), and in adolescents and young people (3) as well as – to a lesser extent - their cost-effectiveness (4). Despite this potential, successful implementation of these interventions and other forms of digital mental health has proven to be challenging, particularly concerning adaptation, uptake and adherence (5). As this is a multifaceted challenge a single solution is non-existent. Addressing this challenge requires taking into account many aspects and perspectives, with implementation sciences gaining increased attention as a result. In this opinion paper, we highlight user involvement as one important aspect in the development, implementation and international adaptation of digital mental health interventions which, to date, still often seems to be overlooked. In the following paragraphs we will define the concept, highlight the potential of involving users to facilitate uptake of digital mental health, and argue for the need for clear guidelines on how to do so, not only for initial development, but also for subsequent international adaptation.
Approaches
In principle, user involvement means that different stakeholders or users are included in one or more steps of the design process. In a context of digital mental health interventions (DMHIs), users might entail patients or clients and their friends and family, clinicians, mental health organizations, and developers. There are, however, multiple approaches to involving these users, of which we highlight three methods: user- centred design, participatory design (or co-design), and user innovation. Within these approaches, the extent to which users have control over design decisions varies: low in user-centred design, higher in participatory design (or co-design), and the highest in user innovation (6). According to Mao et al. (7), user-centred design is a multidisciplinary design approach that actively involves users to improve the understanding of both user requirements and task requirements, as well as the iteration of design and evaluation. In co-design, (potential) users are invited to cooperate with designers, researchers and developers in an innovation process starting from idea generation to decision making (8). Although both approaches are very similar, the difference lies in the starting point and in the extent of the user involvement. In user-centred design, users act as consultants for the designers (after a design idea has already been formulated) and provide feedback throughout the design process. In co-design, users are considered as partners throughout the process from need exploration and idea generation onwards, which ensures that user's' needs and preferences are met and that technologies are acceptable and helpful (9, 10). Finally, in user innovation, the design and development of new products or services is started by end-users, either individual end- users or intermediate users (e.g., organisations) (11). According to a systematic review by Moore et al. (12) user-centred design has been the most reported approach to involve end- users in digital health innovations. However, co-design can be put forward as the more sensible approach to involve vulnerable populations, such as children (9) and older adults (13), but also individuals with mental health conditions (14). Specific methods range from brief user consultation through a review process all the way up to true collaboration. There is not one clear method or process of user involvement, yet common methods include focus groups, surveys, interviews, prototype/storyboards, think-aloud exercises and literature search (12). In particular, Sanz et al. (15) have shown that studies using co-design methods, mostly rely on interviews and workshops, followed by meetings and surveys. In sum, although there is not a single method to involve users, the most promising and sensible approach seems to be co- design: involving users as partners rather than mere consultants from the onset of the design process (9, 13, 16).
Potential
Throughout the years, multiple reviews highlighting the relevance of user involvement during the development of DMHIs have been conducted and the results have shown both similarities and differences.
Torous et al. (14) theorized that low DMHI engagement could be due to poor usability, lack of user-centric design and/or a lack of trust (among other reasons), suggesting co-design with users as a potential solution. Indeed, involving different users in the development and implementation of DMHIs can limit known barriers to uptake and engagement. For example, Liverpool et al. (17) have shown that child and youth engagement with DMHIs is influenced by intervention- specific factors, such as suitability, usability and acceptability on the one hand, and person- specific factors, such as motivation, opportunity and capability.
Similarly, a review by Borghouts et al. (18) has also identified barriers and facilitators to user engagement with DMHIs, user-related (e.g., beliefs, experience and skills), as well as intervention-related (e.g., content, perceived fit and usefulness) barriers and facilitators to user engagement with DMHIs. All of these can be enhanced or tackled, respectively, by involving users in a co-design process.
Orlowski et al. (19) found that user involvement (named consumer consultation in their study) helped to shape specific DMHIs for youth, but they also stated that the effects of user involvement in intervention design are unclear due to limited evidence on specific outcomes and insufficient implementation after piloting in research. In addition, Fischer et al. (13) revealed that involving older adults in technology design (not limited to DMH) leads to better learning of the user's needs, designs adjusted to these needs and better quality of the design, but also showed that effects on acceptance and uptake are unclear. In line with these reviews, the findings of Bevan Jones et al. (9) corroborate the notion that there is little evidence on the impact of user involvement on uptake, adherence and intervention effectiveness.
In contrast, a more recent review has shown positive effects of user involvement, in particular, to enhance cultural sensitivity, enrich ideas, increase acceptance of the DMHIs, better engagement and a sense of community (16). Taken together, while research is unclear on specific outcomes concerning uptake and efficacy, findings do support the importance of involving users in a co-design process to tackle barriers and enhance facilitators to uptake and engagement.
International adaptation of digital mental health interventions
One promising application for user involvement is in the context of international adaptation. Considering the vast number of available DMHIs, in particular mental health apps (14), it is more sensible to use resources to adapt existing, evidence-based DMHIs for use in other countries, rather than to reinvent the wheel. In this respect, adapting interventions to the proposed target population has been a longstanding recommendation (20). Developing or designing technologies to be used beyond a country's borders, however, requires particular considerations further than mere translation of the particular intervention's content. Involving users in the adaptation process can, for example, help to inform about potential user characteristics that may be associated with lower adherence and/or higher drop-out rates (21).
One's approach should therefore take into consideration the target population's cultural, clinical and regulatory aspects, to name only a few. For example, the US has been dominating the app market for smartphone-based mental health apps (22). This means that most apps, evidence-based or not, are primarily in English and developed within the US context, adhering to local regulations and referencing local services. Given that user engagement with a DMHI is enhanced by perceived fit – how well users feel the intervention has culturally appropriate content and understandable language (18) – potential users from Europe, Africa, or Asia might be less interested due to language barriers and lack of cultural sensitivity. To the best of our knowledge, however, evidence on best practices for international adaptation of digital mental health interventions is limited, more so since clear methods for developing and implementing apps in broader international contexts are scarce. In one example, Storm et al. (23) conducted usability tests of an American prototype app called PeerTECH, a peer support app for individuals with a serious mental health condition, with Norwegian users, including clients, clinicians and peer support workers. By doing so, researchers learned that app's adaptation to the Norwegian context would be viable and useful. However, no information on concrete development steps was provided. In another example, Bartlett et al. (24) assessed how well an Australian company involved Arabic-speaking refugees, refugee advocates and healthcare workers during a design thinking process. Their goal was to develop a web-based application to deliver local, evidence-based and culturally relevant health information to its non-English speaking users. Based on their results, relevant recommendations were suggested concerning key communication principles to take into account. Nevertheless, a structured approach for practitionersor researchers to involveusers from different cultures was not discussed. We therefore argue that more research assessing international user involvement is necessary to inspire concrete guidelines for development and international adaptation of digital mental health interventions.
Need for guidelines
Although researchers have provided frameworks, recommendations and specific methods to involve users, this information seems to be insufficiently specific, nor easily retrievable for entrepreneurs and mental health organizations to use. There is, therefore, a clear need for practice- oriented guidelines aimed at stakeholders on different levels, such as policy makers, entrepreneurs and mental health organizations, on how to involve these different users and mental health professionals in the development, implementation and adaptation of mental health technology.
The formulation of these guidelines entails the consideration of multiple critical factors, of which we will highlight four. As a first point, guidelines require more consistency in terminology. Literature on user involvement mentions the concepts of co-design, co-production, co-creation, participatory design, user involvement, etc, seemingly interchangeably. Although these concepts each have their own definitions, and their operationalisation sometimes also differs, the underlying notion is the same, namely (the importance of) involving different stakeholders, specifically users. A second point involves the need for a comprehensive framework. Similar to the terminology, there are multiple frameworks or theories describing how to involve users, for example, the British Design's Double Diamond model (25), or the Generative Co-Design Framework for Healthcare Innovation (26). No study to date, however, has appeared to have described the use of the Double Diamond model for involving users to design a DMHI. In addition, citation analysis shows that the latter has mainly been referenced for its description of co-design principles rather than for following the steps of the framework itself. In one example, the StigmaBeat project has adopted the framework to involve marginalized youngsters to develop short films for reducing mental health stigma (27). In another example, parents of children with cancer were involved in the co-design process of a paediatric cancer pain management app (28). However, in both cases, no information is provided on resulting adoption or user engagement. A third point of attention constitutes the gaps in evidence concerning development of digital mental health interventions. A recent review by Brotherdale et al. (16) on co-production for digital mental health interventions revealed that there is considerable variability concerning which users to involve, the stage and role of their involvement, which methods are used, which frameworks are implemented and how to deal with power dynamics between designers or producers and users, making it difficult to provide evidence-based guidelines. Notwithstanding these gaps, Brotherdale et al. (16) have also identified several commonalities among studies. Successful involvement of users is often hindered by resource constraints, recruitment challenges, conflicting views within the stakeholders and power imbalances between users and designers. It is, therefore, important to suggest potential (evidence-based) solutions and clearly defined steps on how to tackle these barriers. As a fourth and final point, there are, to the best of our knowledge, no evidence-based recommendations for international adaptation of available digital mental health interventions. It is, however, essential to involve local stakeholders as cultural and regulatory variations between nations are plausible. In light of these evolutions, one initiative that aims to contribute to the aforementioned challenges is the “Successful User Participation Examples and Recommendations”-project (SUPER). Funded by Interreg North Sea Region, it aims to develop guidelines for entrepreneurs and mental health organizations on how to involve different stakeholders, in particular users such as patients and mental health professionals, in the (transnational) development, implementation and adaptation of mental health technology.
Conclusion
Successful implementation of digital mental health interventions has proven challenging, and in this opinion paper we wanted to argue that user involvement has the potential to provide at least part of the solution. Although evidence on the impact of user involvement on intervention effectiveness is lacking, its added value for increasing cultural sensitivity, enriching ideas, and increasing acceptance of the digital mental health interventions, and improve engagement is clear (16). Nevertheless, translation to practice is hampered by the fact that clear user involvement steps are rarely properly documented and reported in research. Moreover, concrete evidence-based (or even evidence-inspired) guidelines and steps are lacking, making it difficult for practitioners, developers, and healthcare organizations to adequately involve relevant stakeholders in the design and development process, as well in the increasingly common international adaptation of digital mental health applications. Initiatives, such as the SUPER project, are currently underway to help offer a concrete framework and guidelines. Nevertheless, this will still require uptake in research, as well as practice, to lead to improved user involvement and, ideally, also better digital mental health.
Author contributions
SB: Conceptualization, Writing – original draft, Writing – review & editing. TV: Conceptualization, Writing – review & editing. CC: Writing – review & editing. SN: Writing – review & editing. JS: Writing – review & editing. YR: Writing – review & editing.
Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article.
The Successful User Participation Examples and Recommendations (SUPER) project received funding from the European Union’s Interreg North Sea programme.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. De Witte NAJ, Joris S, Van Assche E, Van Daele T. Technological and digital interventions for mental health and wellbeing: an overview of systematic reviews. Front Digit Health. (2021) 3:754337. doi: 10.3389/FDGTH.2021.754337/BIBTEX
2. Dworschak C, Heim E, Maercker A. Efficacy of internet-based interventions for common mental disorder symptoms and psychosocial problems in older adults: a systematic review and meta-analysis. Internet Interv. (2022) 27. doi: 10.1016/J.INVENT.2022.100498
3. Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment Health. (2021) 8(4):e25847. doi: 10.2196/25847
4. Rohrbach PJ, Dingemans AE, Evers C, Van Furth EF, Spinhoven P, Aardoom JJ, et al. Cost-effectiveness of internet interventions compared with treatment as usual for people with mental disorders: systematic review and meta-analysis of randomized controlled trials. J Med Internet Res. (2023) 25:e38204. doi: 10.2196/38204
5. Berardi C, Antonini M, Jordan Z, Wechtler H, Paolucci F, Hinwood M. Barriers and facilitators to the implementation of digital technologies in mental health systems: a qualitative systematic review to inform a policy framework. BMC Health Serv Res. (2024) 24(1):1–19. doi: 10.1186/S12913-023-10536-1
6. Jarke J. Co-creating digital public services. Public Admin Inf Technol. (2021) 6:15–52. doi: 10.1007/978-3-030-52873-7_3/TABLES/4
7. Mao JY, Vredenburg K, Smith PW, Carey T. The state of user-centered design practice. Commun ACM. (2005) 48(3):105–9. doi: 10.1145/1047671.1047677
8. Sanders EBN, Stappers PJ. Co-creation and the new landscapes of design. Co- Design. (2008) 4(1):5–18. doi: 10.1080/15710880701875068
9. Bevan Jones R, Stallard P, Agha SS, Rice S, Werner-Seidler A, Stasiak K, et al. Practitioner review: co-design of digital mental health technologies with children and young people. J Child Psychol Psychiatry Allied Discip. (2020) 61(8):928–40. doi: 10.1111/JCPP.13258
10. Hodson E, Dadashi N, Delgado R, Chisholm C, Sgrignoli R, Swaine R. Co- design in mental health; mellow: a self-help holistic crisis planning mobile application by youth, for youth. Design J. (2019) 22(sup1):1529–42. doi: 10.1080/14606925.2019.1594975
11. Bogers M, Afuah A, Bastian B. Users as innovators: a review, critique, and future research directions. J Manage. (2010) 36(4):857–75. doi: 10.1177/0149206309353944
12. Moore G, Wilding H, Gray K, Castle D. Participatory methods to engage health service users in the development of electronic health resources: systematic review. J Particip Med. (2019) 11(1):e11474. doi: 10.2196/11474
13. Fischer B, Peine A, Östlund B. The importance of user involvement: a systematic review of involving older users in technology design. Gerontologist. (2020) 60(7):513–23. doi: 10.1093/geront/gnz163
14. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. BMJ Ment Health. (2018) 21(3):116–9. doi: 10.1136/eb-2018-102891
15. Sanz MF, Acha BV, García MF. Co-design for people-centred care digital solutions: a literature review. Int J Integr Care. (2021) 21(2):16. doi: 10.5334/IJIC.5573
16. Brotherdale R, Berry K, Branitsky A, Bucci S. Co-producing digital mental health interventions: a systematic review. Digital Health. (2024) 10. doi: 10.1177/20552076241239172
17. Liverpool S, Mota CP, Sales CMD, Čuš A, Carletto S, Hancheva C, et al. Engaging children and young people in digital mental health interventions: systematic review of modes of delivery, facilitators, and barriers. J Med Internet Res. (2020) 22(6):e16317. doi: 10.2196/16317
18. Borghouts J, Eikey E, Mark G, De Leon C, Schueller SM, Schneider M, et al. Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. J Med Internet Res. (2021) 23(3):e24387. doi: 10.2196/24387
19. Orlowski SK, Lawn S, Venning A, Winsall M, Jones GM, Wyld K, et al. Participatory research as one piece of the puzzle: a systematic review of consumer involvement in design of technology-based youth mental health and well-being interventions. JMIR Human Factors. (2015) 2(2):e12. doi: 10.2196/HUMANFACTORS.4361
20. Van Daele T, Karekla M, Kassianos AP, Compare A, Haddouk L, Salgado J, et al. Recommendations for policy and practice of telepsychotherapy and e-mental health in Europe and beyond. J Psychother Integr. (2020) 30(2):160–73. doi: 10.1037/int0000218
21. Karekla M, Kasinopoulos O, Neto DD, Ebert DD, Van Daele T, Nordgreen T, et al. Best practices and recommendations for digital interventions to improve engagement and adherence in chronic illness sufferers. Eur Psychol. (2019) 24(1):49–67. doi: 10.1027/1016-9040/A000349
22. Grand View Research. Mental Health Apps Market Size & Trends (2022). Available online at: https://www.grandviewresearch.com/industry-analysis/mental-health-apps-market-report (accessed July 5, 2024).
23. Storm M, Fjellså HMH, Skjærpe JN, Myers AL, Bartels SJ, Fortuna KL. Usability testing of a Mobile health application for self-management of serious mental illness in a Norwegian community mental health setting. Int J Environ Res Public Health. (2021) 18(16):8667. doi: 10.3390/IJERPH18168667
24. Bartlett R, Boyle JA, Simons Smith J, Khan N, Robinson T, Ramaswamy R. Evaluating human-centred design for public health: a case study on developing a healthcare app with refugee communities. Res Involv Engagem. (2021) 7(1):1–13. doi: 10.1186/S40900-021-00273-2/TABLES/4
25. The Double Diamond - Design Council (n.d.). Available online at: https://www.designcouncil.org.uk/our-resources/the-double-diamond/ (accessed July 15, 2024).
26. Bird M, McGillion M, Chambers EM, Dix J, Fajardo CJ, Gilmour M, et al. A generative co-design framework for healthcare innovation: development and application of an end-user engagement framework. Res Involv Engagem. (2021) 7(1):1–12. doi: 10.1186/S40900-021-00252-7/FIGURES/1
27. Hine R, Gladstone B, Reupert A, O'Dea L, Cuff R, Yates S, et al. Stigmabeat: collaborating with rural young people to co-design films aimed at reducing mental health stigma. Qual Health Res. (2024) 34(6):491–506. doi: 10.1177/10497323231211454
Keywords: digital mental health, user participation, end-user involvement, guidelines, adaptation
Citation: Bernaerts S, Van Daele T, Carlsen CK, Nielsen SL, Schaap J and Roke Y (2024) User involvement in digital mental health: approaches, potential and the need for guidelines. Front. Digit. Health 6:1440660. doi: 10.3389/fdgth.2024.1440660
Received: 29 May 2024; Accepted: 24 July 2024;
Published: 22 August 2024.
Edited by:
Uffe Kock Wiil, University of Southern Denmark, DenmarkReviewed by:
Daniel Z. Q. Gan, The University of Melbourne, AustraliaCopyright: © 2024 Bernaerts, Van Daele, Carlsen, Nielsen, Schaap and Roke. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Sylvie Bernaerts, sylvie.bernaerts@thomasmore.be