Maximizing the Impact of e-Therapy and Serious Gaming: Time for a Paradigm Shift

Internet interventions for mental health, including serious games, online programs, and apps, hold promise for increasing access to evidence-based treatments and prevention. Many such interventions have been shown to be effective and acceptable in trials; however, uptake and adherence outside of trials is seldom reported, and where it is, adherence at least, generally appears to be underwhelming. In response, an international Collaboration On Maximizing the impact of E-Therapy and Serious Gaming (COMETS) was formed. In this perspectives’ paper, we call for a paradigm shift to increase the impact of internet interventions toward the ultimate goal of improved population mental health. We propose four pillars for change: (1) increased focus on user-centered approaches, including both user-centered design of programs and greater individualization within programs, with the latter perhaps utilizing increased modularization; (2) Increased emphasis on engagement utilizing processes such as gaming, gamification, telepresence, and persuasive technology; (3) Increased collaboration in program development, testing, and data sharing, across both sectors and regions, in order to achieve higher quality, more sustainable outcomes with greater reach; and (4) Rapid testing and implementation, including the measurement of reach, engagement, and effectiveness, and timely implementation. We suggest it is time for researchers, clinicians, developers, and end-users to collaborate on these aspects in order to maximize the impact of e-therapies and serious gaming.

Introduction

The rationale for internet interventions for mental health is commonly centered on the following premises:

– Mental disorders, such as anxiety and depression, are common, disabling, and costly (1).

– Evidence-based interventions have been developed; however, the majority of people who would benefit do not receive any treatment (2).

– Largely, this “treatment gap” is due to structural or health system-related barriers (such as costs and lack of trained therapists), and social barriers, such as stigma (3).

– Internet therapies can offer scalable approaches whereby large numbers of people can receive treatment and/or prevention, potentially bypassing barriers related to cost, location, lack of trained professionals, and stigma (4).

Systematic reviews and meta-analyses of randomized controlled trials of internet interventions and computerized therapies delivered off-line (e.g., via CDRom) for anxiety and/or depression have reported good evidence of effectiveness (5–7), with adherence rates from 26 to 76% (8).

Despite the robust rationale and promising evidence, relatively few evidence-based interventions have been implemented or made publicly available. Among those that have been implemented in naturalistic or “real-world” settings (i.e., outside of traditional trials), limited data regarding implementation, uptake, and impact have been published. Available data suggest that significant numbers of people may be interested in online mental health support. For example, the publicly available self-help program, MoodGYM, attracted approximately 38,000 registrants over a 14-month period (9), and the mental health app, Happify, had been downloaded between 100,000 and 500,000 times on Google Play as at 16 December 2015. Moreover, a high proportion of people who complete the self-assessments in such interventions have substantial symptoms (10, 11). Despite this, naturalistic use of internet interventions for mental health appears to be associated with high attrition (non-adherence or drop out from the intervention); with notably higher rates of attrition in implemented programs than in randomized controlled trials, even when the same program is used. For example, in field studies of MoodGYM, only 3.9% of public registrants completed at least three of the five modules, in contrast to 53.8% of participants in a controlled trial of the same intervention (12). Less than 7% of public registrants continued past two modules in a newer version of the program (9), and similar results were found with adolescents (13). In another example, only 23% of users of a PTSD coach app used it during the first month after download, and the median time spent per session was less than one minute (14).

Increasing human support has been a core strategy for enhancing adherence to online interventions (8, 15, 16). This is promising with many trials finding higher rates of adherence to supported interventions than pure self-help interventions (17), although this is not always the case (18). Regardless of the comparison to pure self-help, attrition is still a challenge for supported internet interventions. For example, in a recent independent study by Gilbody and others (19), even with weekly telephone support, non-adherence was such that there was no treatment gain for patients accessing computerized therapies (MoodGYM and Beating the Blues) over those allocated to primary health care alone. Moreover, given that part of the rationale for computerized therapies is their scalability and low barriers to helpseeking, alternative approaches to increasing engagement should also be considered.

Together, findings from naturalistic or implementation studies suggest that potential users, including those with significant symptoms, are interested in internet interventions for mental health (20–23); however, implementation and engagement require improvement. We contend that the current paradigm or approach for the development of evidence-based internet interventions for mental health is typically researcher led; that the programs are often designed to replicate tested face-to-face therapies online; and that these are usually tested using classic validation designs (randomized controlled trials), with a focus on the efficacy of stand-alone interventions. Many of these components are critical to demonstrate that internet interventions can be effective under trial conditions as shown in Figure 1. However, to increase the impact of serious games and e-therapies, further developments are required.

FIGURE 1

Figure 1. Toward maximizing the impact of e-therapy and serious gaming for mental health.

An international group initiated by the last author (Heleen Riper) met in Amsterdam and Valencia with students and game designers. The group included authors of the present paper and others involved in e-health topics, including the fields of serious gaming (SPARX, Michael’s Game) and virtual reality (EMMA, BUTLER), apps for depression (MOODBUSTER) and substance use (Stop-cannabis), and online interventions for suicide prevention (PITSTOP suicide). An international Collaboration on Maximizing the impact of E-Therapy and Serious Gaming (COMETS) was established. We identified four pillars that need stronger emphasis for advancing research and increasing the impact of e-therapies and serious gaming. As shown in Figure 1, these are:

1. Increased focus on user-centered approaches (including user-centered design of interventions, and user responsiveness or individualization within interventions).

2. Greater emphasis on engagement (utilizing processes, such as gaming, telepresence and persuasive technology, and incorporating measures of engagement).

3. Increased collaboration across geographical regions, sectors, and interest groups.

4. Rapid testing and implementation.

The present paper outlines these potential approaches for increasing the real-world impact of internet interventions. New emphases or approaches should enhance, not replace, rigorous research-based approaches.

Toward a New Paradigm

Increased Focus on User-Centered Approaches

We propose that one of the key ways of increasing the impact of internet interventions is through increasing the focus on user-centered approaches. This would include user-centered design processes and greater individualization within programs, with the latter perhaps utilizing increased modularization.

User-Centered Design

While some computerized therapies and serious games have been designed with significant user input, we contend that uptake and adherence to internet interventions can be enhanced with greater involvement and understanding of users. This is not merely about consulting users on drafts but also about a deep understanding of user needs and preferences, and actively involving users in design processes from the outset. For example, a traditional research-centered process might begin with reviewing published evidence and subsequently planning an internet intervention containing six to ten modules to be completed at a rate of about one per week, hereby approximating evidence-based face-to-face therapies. By contrast, a user-centered design would begin with users, to understand issues such as how and when they would be willing to use the internet for mental wellbeing, and to explore their current behavior, needs, and preferences. Such an approach might suggest alternative processes or content. For example, alternative frequencies and durations of use might be proposed in order to reflect how people actually utilize the internet for psychological needs. Alternative processes for engagement or therapeutic change could also be identified for investigation (such as the use of sharable content or the opportunity to help others). The evidence for each of these components should, of course, be investigated. The point being not to replace research with user-centered design, but to utilize user-centered processes alongside scientific research.

Increasing Individualization

Alongside an increased focus on user-centered design of interventions, increased individualization, responsiveness, and choice within interventions may be important for engagement. To date, most evidence-based internet interventions for mental health are not very individualized. That is, all people using the same program generally all receive the same content, albeit sometimes with some optional modules. This is out of keeping with contemporary personalized experiences of the internet, which is very choice based. Engaging clients in defining their goals may result in better compliance than a more clinical and generalized aim of treating their diagnostic condition (24, 25). It should also be simple for users to select, from effective alternatives, their own preferred options or approaches for meeting goals. Modularization of interventions is promising in this regard.

Exploration of Modular Approaches

Limitations of current disorder-focused approaches to psychological therapies have been increasingly recognized (26). Briefly, most evidence-based psychological therapies have been developed for single disorders, such as depression, anxiety, and so on. However, in clinical practice, co-morbidity is the rule. In addition, there is an overlap in the techniques used to treat different disorders. In order to make treatments more efficient and to deal with clinical realities, a modular approach for face-to-face therapy has been developed, and early clinical trials show promise with better clinical outcomes delivered in less time (26–29). Should modularization prove effective, this approach would be feasible online, and could facilitate increased user choice and increased collaboration between groups.

Increasing Engagement

A second key area for increasing the impact of internet interventions will involve the use of approaches that motivate continued usage (adherence) and improve user engagement (30, 31). The use of serious gaming and gamification, enhanced telepresence, and increased use of persuasive technology are promising in this regard. Moreover, the routine assessment of engagement may help to further develop the field and monitor progress toward this goal.

Serious Gaming and Gamification

“Serious games” are interventions that are games, or that utilize elements of gaming, as an integral and primary method for achieving a serious purpose, such as a health or educational goal (32). Gamification refers to the addition of gaming elements (such as challenges, reward, and experiences of exploration) to a non-game environment. The inclusion of gaming elements within computerized psychotherapies, and games or game-like environments with embedded therapeutic content, has been tested in several trials (32, 33). This approach is at an early stage, with few (if any) trials performed independent of developers (32, 34). However, there is promising evidence for serious games in other areas of health and behavior change (35–38). The potential for mental health has been identified (10, 39–44), and is supported by relatively low attrition rates in initial trials of mental health interventions utilizing these strategies (10, 45, 46).

Enhanced Telepresence

A therapeutic relationship is arguably a critical “active ingredient” of therapy (47). Increasing human support for users of internet interventions, via telephone, text, email, or face-to-face contact, appears to be helpful (8, 15, 16). Increasing social “telepresence,” or the feeling of connections with others within the computer program itself (48), may also hold promise. This can be achieved by using thoughtful design processes; for example, in the SPARX computerized CBT program, the “Guide” was designed as a virtual therapist, with warm welcoming wording, a carefully selected voice actor, and active rapport building. In interviews, young people reported feeling that the guide and other virtual characters in the computer program cared about them, and that this enhanced their experience of the intervention (49).

Increased Use of Persuasive Technology

The science of persuasive technology refers to the use of technology to influence human behavior, motivation, and attitudes through human–computer interaction or computer-mediated communication (50). Examples of persuasive technology include the use of automated support to increase primary task completion, such as automated SMS or mobile phone messages, email prompts, continued feedback by the program, and built-in explanations of why the program might help (51). Ethical issues related to the use of persuasive technologies must be carefully taken into account. Persuasion should be based on consent (52), and should help people to change the behavior they would like to change. Furthermore, not all participants require or prefer the same amount (or the same kind) of support, and this assistance may only be needed at critical times during the treatment program (53). Nevertheless, persuasive technology has been significant in promoting engagement and behavior change in other areas, and has arguably been underused in the development of internet interventions for mental health.

Measuring Engagement

A critical variable for improving the impact of computerized therapy interventions is patient or user engagement. According to Patient Health Engagement (PHE) models (54), making patients highly engaged in their care sustains them in attributing full meaning to the therapy, and in enacting self-management behaviors effectively, even when life contexts change. When effectively engaged, patients also develop a sustainable perspective about their actual and possible conditions, which can be better integrated into action (55). In this way, patient engagement can be considered as a compass to help developers customize their interventions. Patient engagement also has another advantage for developers and researchers: it can be easily measured using validated instruments (see Table 1).

TABLE 1

Table 1. Available validated tools for assessing Patient Engagement.

Rapid Testing and Implementation

Rapid Prototyping and Testing

Many health care interventions are developed after first consulting the literature, and a small number of experts and consumers. They are often then piloted within small groups, after which minor adjustments may be made before the intervention is tested in a larger trial. It can, therefore, be a number of years from initial development until the results of a controlled trial are published. Replication may be required, and only then does implementation become a priority. This drawn-out process is problematic when testing e-health interventions due to the speed of technology change. Alternative models of rapid development and iterative testing should be considered (58); for example, using agile software design principles, such as the lean start-up method (59) or scrum (60).

In agile development processes, the product is tested with users from the outset using rapid development and testing feedback loops. An important component involves the development of a minimal viable product (MVP). An MVP is a barely finished product that contains an essential element, but is missing details, and is provided to end-users to gage their reactions and inform the next steps in development. Reponses to the product are measured and used to inform next steps that are rapidly developed and tested in the same way. This iterative process involves close collaboration between designers, software developers, and end-users. Larger scale testing gradually replaces small opportunistic samples as progressively more complex features are tested. When a near-finished version is ready, more traditional testing can be carried out, for example, via a randomized controlled trial. As described by Mohr et al. (61), internet interventions can utilize approaches that focus on evaluating the working mechanisms, rather than a locked-down version of the intervention. Such a framework allows for improvements in functionality to be made during a trial, subsequently resulting in a more generalizable and durable intervention.

A Planned Focus on Implementation

Many internet interventions for mental health have been developed and shown positive results in trials, but are not publicly available, or, are available, but have limited uptake or adherence. This highlights two points. First, that efficacy alone is not sufficient to indicate an intervention will have a significant impact on health. Reach (or exposure to and uptake of the intervention) and adherence must also be evaluated, and findings used to improve programs and their implementation (62). Second, the necessary conditions for the sustainable implementation of interventions, including consideration of the project’s future ownership and the identification of possible revenue streams for ongoing hosting and updates, must be considered from the outset. Implementation may, therefore, necessitate non-traditional collaborations.

Increased Collaboration

Intersectorial

A key opportunity for improving the impact of computerized therapies and serious gaming is increased use of diverse knowledge and skills. The design of internet interventions for mental health should involve input from different fields, including – but not limited to – users, therapists, computer engineers, game designers, behavior change experts, and human factors specialists. Such collaboration requires researchers to move beyond their discipline and consider new knowledge, methods, and techniques. Current approaches for evidence-based internet interventions for mental health are often initiated and led by researchers. Alternative approaches, for example, where researchers join projects that are led by users, game developers, or internet and software experts, should also be considered.

International Collaborations

International collaboration is a further key opportunity for improving the long-term sustainability and growth of e-therapy. Many internet interventions are developed within a specific country or jurisdiction, and fail to make use of the international nature of the internet, with subsequent limitations in terms of funding and the impact of the proposed intervention. These stand-alone programs are often supported by modest financial resources, resulting in small-scale clinical trials or case studies. A promising strategy is to develop stronger multinational teams and projects.

An Ambitious Future Vision

Should increased collaboration and modular user-centered approaches be pursued, greater gains in population mental health may be realized. This could be achieved, for example, by creating a user-centered online platform or ecosystem that allows users to select the components that most appeal to them or that are recommended based on their self-assessments. Should such a model become a reality, components could be continuously developed and additional ones added over time. A well-designed system could allow uptake, adherence and effectiveness of components to be routinely measured and compared. We propose that, in the future, such systems could safely invite user-generated content and input from other researchers and developers within agreed guidelines. Should this vision be achieved, the input and energy of diverse groups could be harnessed to facilitate development in the field.

Conclusion

Evidence-based mental health interventions are promising; however, uptake and adherence outside of trial settings have not yet met hopes and projections. We have proposed that it is time for a paradigm shift in order to maximize the impact of evidence-based e-therapies and serious gaming. Promising directions include a greater focus on user-centered approaches (including user-centered design, individualization within interventions, and exploration of modularized programs), increased emphasis on engagement (utilizing processes such as gaming, telepresence and persuasive technology, and measuring engagement), increased international and intersectorial collaboration, and rapid testing and implementation. We propose that, in the future, such systems could safely invite both user-generated content and input from other researchers and developers. In each case, input should be within agreed guidelines.

Author Contributions

HR initiated the COMETS group. CB, RB, GR, AG, YK, TF, DB, AK, HL, and SM developed or contributed to the concepts of the COMETS group, which are represented in this paper. TF, DB, YK, AG, GR, CB, RB, FA, and LB drafted the paper. AK, SM, HL, and HR contributed substantial content to the paper. All co-authors approved the paper.

Conflict of Interest Statement

TF and SM are co-developers of SPARX computerized therapy for depression and can benefit from any commercialization of it outside of New Zealand. The remaining authors have no conflict of interest to declare.

Funding

No funding was received for the development of this paper. SM, TF, and LB are funded by the University of Auckland. GR and AG are funded by Ministero della Salute.

References

1. Kessler RC, Aguilar-Gaxiola S, Alonso J, Chatterji S, Lee S, Ormel J, et al. The global burden of mental disorders: an update from the WHO world mental health (WMH) surveys. Epidemiol Psichiatr Soc (2009) 18(1):23–33. doi:10.1017/S1121189X00001421

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Kazdin AE, Blase SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspect Psychol Sci (2011) 6(1):21–37. doi:10.1177/1745691610393527

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Andrade LH, Alonso J, Mneimneh Z, Wells JE, Al-Hamzawi A, Borges G, et al. Barriers to mental health treatment: results from the WHO world mental health surveys. Psychol Med (2014) 44(6):1303–17. doi:10.1017/S0033291713001943

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Christensen H, Hickie IB. E-mental health: a new era in delivery of mental health services. Med J Aust (2010) 192(Suppl 11):S2–3.

Google Scholar

5. Adelman CB, Panza KE, Bartley CA, Bontempo A, Bloch MH. A meta-analysis of computerized cognitive-behavioral therapy for the treatment of DSM-5 anxiety disorders. J Clin Psychiatry (2014) 75(7):e695–704. doi:10.4088/JCP.13r08894

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U. Internet-delivered psychological treatments for mood and anxiety disorders: a systematic review of their efficacy, safety, and cost-effectiveness. PLoS One (2014) 9(5):e98118. doi:10.1371/journal.pone.0098118

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Ebert DD, Zarski AC, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One (2015) 10(3):e0119895. doi:10.1371/journal.pone.0119895

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev (2012) 32(4):329–42. doi:10.1016/j.cpr.2012.02.004

CrossRef Full Text | Google Scholar

9. Christensen H, Griffiths K, Groves C, Korten A. Free range users and one hit wonders: community users of an internet-based cognitive behaviour therapy program. Aust N Z J Psychiatry (2006) 40(1):59–62. doi:10.1080/j.1440-1614.2006.01743.x

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Khazaal Y, Chatton A, Monney G, Nallet A, Khan R, Zullino D, et al. Internal consistency and measurement equivalence of the cannabis screening questions on the paper-and-pencil face-to-face ASSIST versus the online instrument. Subst Abuse Treat Prev Policy (2015) 10:8. doi:10.1186/s13011-015-0002-9

PubMed Abstract | CrossRef Full Text | Google Scholar

11. BinDhim NF, Shaman AM, Trevena L, Basyouni MH, Pont LG, Alhawassi TM. Depression screening via a smartphone app: cross-country user characteristics and feasibility. J Am Med Inform Assoc (2015) 22(1):29–34. doi:10.1136/amiajnl-2014-002840

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Christensen H, Griffiths KM, Korten AE, Brittliffe K, Groves C. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J Med Internet Res (2004) 6(4):e46. doi:10.2196/jmir.6.4.e46

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Neil AL, Batterham P, Christensen H, Bennett K, Griffiths KM. Predictors of adherence by adolescents to a cognitive behavior therapy website in school and community-based settings. J Med Internet Res (2009) 11(1):e6. doi:10.2196/jmir.1050

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Owen JE, Jaworski BK, Kuhn E, Makin-Byrd KN, Ramsey KM, Hoffman JE. mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD coach. JMIR Ment Health (2015) 2(1):e7. doi:10.2196/mental.3935

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Christensen H, Reynolds J, Griffiths KM. The use of e-health applications for anxiety and depression in young people: challenges and solutions. Early Interv Psychiatry (2011) 5(Suppl 1):58–62. doi:10.1111/j.1751-7893.2010.00242.x

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Newman MG, Szkodny LE, Llera SJ, Przeworski A. A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: is human contact necessary for therapeutic efficacy? Clin Psychol Rev (2011) 31(1):89–103. doi:10.1016/j.cpr.2010.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med (2007) 37(03):319–28. doi:10.1017/S0033291706008944

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Berger T, Hämmerli K, Gubser N, Andersson G, Caspar F. Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help. Cogn Behav Ther (2011) 40(4):251–66. doi:10.1080/16506073.2011.616531

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ (2015) 351:h5627. doi:10.1136/bmj.h5627

CrossRef Full Text | Google Scholar

20. Van De Belt TH, Engelen LJ, Berben SA, Teerenstra S, Samsom M, Schoonhoven L. Internet and social media for health-related information and communication in health care: preferences of the Dutch general population. J Med Internet Res (2013) 15(10):2607. doi:10.2196/jmir.2607

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Krebs P, Duncan DT. Health app use among US mobile phone owners: a national survey. JMIR Mhealth Uhealth (2015) 3(4):e101. doi:10.2196/mhealth.4924

CrossRef Full Text | Google Scholar

22. Zia JK, Le T, Munson S, Heitkemper MM, Demiris G. Download alert: understanding gastroenterology patients’ perspectives on health-related smartphone apps. Clin Transl Gastroenterol (2015) 6:e96. doi:10.1038/ctg.2015.25

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Horgan Á, Sweeney J. Young students’ use of the internet for mental health information and support. J Psychiatr Ment Health Nurs (2010) 17(2):117–23. doi:10.1111/j.1365-2850.2009.01497.x

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Ackerman SJ, Hilsenroth MJ, Baity MR, Blagys MD. Interaction of therapeutic process and alliance during psychological assessment. J Pers Assess (2000) 75(1):82–109. doi:10.1207/S15327752JPA7501_7

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Finn SE. In Our Clients’ Shoes: Theory and Techniques of Therapeutic Assessment. Mahwah, NJ: Erlbaum (2007).

Google Scholar

26. Chorpita BF, Daleiden EL, Weisz JR. Modularity in the design and application of therapeutic interventions. Appl Prev Psychol (2005) 11(3):141–56. doi:10.1016/j.appsy.2005.05.002

CrossRef Full Text | Google Scholar

27. Weisz JR, Chorpita BF, Palinkas LA, Schoenwald SK, Miranda J, Bearman SK, et al. Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: a randomized effectiveness trial. Arch Gen Psychiatry (2012) 69(3):274–82. doi:10.1001/archgenpsychiatry.2011.147

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Barlow DH, Allen LB, Choate ML. Toward a unified treatment for emotional disorders. Behav Ther (2004) 35(2):205–30. doi:10.1016/S0005-7894(04)80036-4

CrossRef Full Text | Google Scholar

29. Farchione TJ, Fairholme CP, Ellard KK, Boisseau CL, Thompson-Hollands J, Carl JR, et al. Unified protocol for transdiagnostic treatment of emotional disorders: a randomized controlled trial. Behav Ther (2012) 43(3):666–78. doi:10.1016/j.beth.2012.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res (2012) 14(6):e152. doi:10.2196/jmir.2104

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Ludden GD, Van Rompay TJ, Kelders SM, Van Gemert-Pijnen JE. How to increase reach and adherence of web-based interventions: a design research viewpoint. J Med Internet Res (2015) 17(7):e172. doi:10.2196/jmir.4201

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Fleming TM, Cheek C, Merry SN, Thabrew H, Bridgman H, Stasiak K, et al. Serious games for the treatment or prevention of depression: a systematic review. Revista de Psicopatologia y Psicologia Clinica (2014) 19(3):227–42. doi:10.5944/rppc.vol.19.num.3.2014.13904

CrossRef Full Text | Google Scholar

33. Li J, Theng YL, Foo S. Game-based digital interventions for depression therapy: a systematic review and meta-analysis. Cyberpsychol Behav Soc Netw (2014) 17(8):519–27. doi:10.1089/cyber.2013.0481

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Cheek C, Fleming T, Lucassen MF, Bridgman H, Stasiak K, Shepherd M, et al. Integrating health behavior theory and design elements in serious games. JMIR Ment Health (2015) 2(2):e11. doi:10.2196/mental.4133

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Govender M, Bowen RC, German ML, Bulaj G, Bruggers CS. Clinical and neurobiological perspectives of empowering pediatric cancer patients using videogames. Games Health J (2015) 4(5):362. doi:10.1089/g4h.2015.0014

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Ustinova KI, Perkins J, Leonard WA, Hausbeck CJ. Virtual reality game-based therapy for treatment of postural and co-ordination abnormalities secondary to TBI: a pilot study. Brain Inj (2014) 28(4):486–95. doi:10.3109/02699052.2014.888593

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Prosperini L, Fanelli F, Petsas N, Sbardella E, Tona F, Raz E, et al. Multiple sclerosis: changes in microarchitecture of white matter tracts after training with a video game balance board. Radiology (2014) 273(2):529–38. doi:10.1148/radiol.14140168

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Staiano AE, Abraham AA, Calvert SL. Adolescent exergame play for weight loss and psychosocial improvement: a controlled physical activity intervention. Obesity (2013) 21(3):598–601. doi:10.1002/oby.20282

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Girard B, Turcotte V, Bouchard S, Girard B. Crushing virtual cigarettes reduces tobacco addiction and treatment discontinuation. Cyberpsychol Behav (2009) 12(5):477–83. doi:10.1089/cpb.2009.0118

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Khazaal Y, Chatton A, Prezzemolo R, Zebouni F, Edel Y, Jacquet J, et al. Impact of a board-game approach on current smokers: a randomized controlled trial. Subst Abuse Treat Prev Policy (2013) 8(1):3. doi:10.1186/1747-597X-8-3

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Anguera JA, Boccanfuso J, Rintoul JL, Al-Hashimi O, Faraji F, Janowich J, et al. Video game training enhances cognitive control in older adults. Nature (2013) 501(7465):97–101. doi:10.1038/nature12486

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Fish MT, Russoniello CV, O’Brien K. The efficacy of prescribed casual videogame play in reducing symptoms of anxiety: a randomized controlled study. Games Health J (2014) 3(5):291. doi:10.1089/g4h.2013.0092

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Russoniello CV, Fish M, O’Brien K. The efficacy of casual videogame play in reducing clinical depression: a randomized controlled study. Games Health J (2013) 2(6):341. doi:10.1089/g4h.2013.0010

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Botella C, Breton-López J, Quero S, Baños RM, Garcia-Palacios A, Zaragoza I, et al. Treating cockroach phobia using a serious game on a mobile phone and augmented reality exposure: a single case study. Comput Human Behav (2011) 27(1):217–27. doi:10.1016/j.chb.2010.07.043

CrossRef Full Text | Google Scholar

45. Fleming T, Dixon R, Frampton C, Merry S. A pragmatic randomized controlled trial of computerized CBT (SPARX) for symptoms of depression among adolescents excluded from mainstream education. Behav Cogn Psychother (2012) 40(5):529–41. doi:10.1017/S1352465811000695

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MF. The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial. BMJ (2012) 344:e2598. doi:10.1136/bmj.e2598

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Weisz JR, McCarty CA, Valeri SM. Effects of psychotherapy for depression in children and adolescents: a meta-analysis. Psychol Bull (2006) 132(1):132–49. doi:10.1037/0033-2909.132.1.132

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Westerman D, Skalski PD. Presence and computers: a ghost in the machine? In: Bracken C, Skalski PD, editors. Immersed in Media: Telepresence in Everyday Life. New York: Routledge (2010). p. 63–86.

Google Scholar

49. Fleming T, Lucassen M, Stasiak K, Shepherd M, Merry S. The impact and utility of computerised therapy for educationally alienated teenagers: the views of adolescents who participated in an alternative education-based trial. Clin Psychol (2016). doi:10.1111/cp.12052

CrossRef Full Text | Google Scholar

50. Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun Assoc Inform Syst (2009) 24(1):485–500.

Google Scholar

51. Donkin L, Glozier N. Motivators and motivations to persist with online psychological interventions: a qualitative study of treatment completers. J Med Internet Res (2012) 14(3):e91. doi:10.2196/jmir.2100

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Spahn A. And lead us (not) into persuasion…? Persuasive technology and the ethics of communication. Sci Eng Ethics (2012) 18(4):633–50. doi:10.1007/s11948-011-9278-y

CrossRef Full Text | Google Scholar

53. Wootton BM, Dear BF, Johnston L, Terides MD, Titov N. Remote treatment of obsessive-compulsive disorder: a randomized controlled trial. J Obsessive Compuls Relat Disord (2013) 2(4):375–84. doi:10.1016/j.jocrd.2013.07.002

CrossRef Full Text | Google Scholar

54. Graffigna G, Barello S, Bonanomi A, Lozza E. Measuring patient engagement: development and psychometric properties of the patient health engagement (PHE) scale. Front Psychol (2015) 6:274. doi:10.3389/fpsyg.2015.00274

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Forbat L, Cayless S, Knighting K, Cornwell J, Kearney N. Engaging patients in health care: an empirical study of the role of engagement on attitudes and action. Patient Educ Couns (2009) 74(1):84–90. doi:10.1016/j.pec.2008.07.055

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res (2004) 39:1005–26. doi:10.1111/j.1475-6773.2004.00269.x

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Wasson J, Coleman EA. Health confidence: an essential measure for patient engagement and better practice. Fam Pract Manag (2014) 21(5):8–12.

Google Scholar

58. van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res (2011) 13(4):e111. doi:10.2196/jmir.1672

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Ries E. The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. New York: Crown Business (2011).

Google Scholar

60. Schwaber K, Sutherland J. The Scrum Guide: The Definitive Guide to Scrum: The Rules of the Game. (2011). Available from: http://scrumguides.org

Google Scholar

61. Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res (2015) 17(7):e166. doi:10.2196/jmir.4391

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health (1999) 89(9):1322–7. doi:10.2105/AJPH.89.9.1322

PubMed Abstract | CrossRef Full Text | Google Scholar