SIMON: A Digital Protocol to Monitor and Predict Suicidal Ideation

Each year, more than 800,000 persons die by suicide, making it a leading cause of death worldwide. Recent innovations in information and communication technology may offer new opportunities in suicide prevention in individuals, hereby potentially reducing this number. In our project, we design digital indices based on both self-reports and passive mobile sensing and test their ability to predict suicidal ideation, a major predictor for suicide, and psychiatric hospital readmission in high-risk individuals: psychiatric patients after discharge who were admitted in the context of suicidal ideation or a suicidal attempt, or expressed suicidal ideations during their intake. Specifically, two smartphone applications -one for self-reports (SIMON-SELF) and one for passive mobile sensing (SIMON-SENSE)- are installed on participants' smartphones. SIMON-SELF uses a text-based chatbot, called Simon, to guide participants along the study protocol and to ask participants questions about suicidal ideation and relevant other psychological variables five times a day. These self-report data are collected for four consecutive weeks after study participants are discharged from the hospital. SIMON-SENSE collects behavioral variables -such as physical activity, location, and social connectedness- parallel to the first application. We aim to include 100 patients over 12 months to test whether (1) implementation of the digital protocol in such a high-risk population is feasible, and (2) if suicidal ideation and psychiatric hospital readmission can be predicted using a combination of psychological indices and passive sensor information. To this end, a predictive algorithm for suicidal ideation and psychiatric hospital readmission using various learning algorithms (e.g., random forest and support vector machines) and multilevel models will be constructed. Data collected on the basis of psychological theory and digital phenotyping may, in the future and based on our results, help reach vulnerable individuals early and provide links to just-in-time and cost-effective interventions or establish prompt mental health service contact. The current effort may thus lead to saving lives and significantly reduce economic impact by decreasing inpatient treatment and days lost to inability.

Introduction

Digitalization has captured much of human society and is omnipresent in individuals' everyday lives. People carry their smartphone with them most of the time, even in times of crisis (1). This innovation provides new opportunities to help reach vulnerable individuals in critical moments [e.g., (2, 3)]. One group that could particularly benefit from this are individuals at risk for suicide. Suicide is one of the leading causes of deaths, and the numbers continue to rise. As a consequence, a better understanding, prediction, and prevention has been made one of the top priorities on international research agendas including the World Health Organization (4).

One of the greatest challenges to understand, predict, and prevent suicide has long been that it has to be intervened upon as it occurs and evolves in real life. Recent studies show that suicidal thoughts vary considerably throughout daily life, and can escalate quickly [for overviews, see (5, 6)]. Mobile technology can help address this challenge. For example, smartphones can be leveraged to perform real time collection of relevant self-report data and behavior, which can lead to just-in-time interventions (7). For instance, iHealth or intelligent Health has been proposed, in which the incorporation of new technologies into clinical practice helps shifting mental health care from a reactive to a proactive, participatory, and personalized domain, by for instance enhancing real-time self-monitoring and supporting medical decision making (8). With regards to suicide specifically, there has been a rapid increase in the use of mobile technology to help prevent suicide, but a major problem is that existing suicide prevention smartphone applications are not evidence-based or clinically validated (9, 10).

Before just-in-time interventions are possible, proximal risk factors of suicidal behavior have to be identified. Proximal risk factors are factors that predict the short-term occurrence of suicidal behaviors (11). Recently, there has been an increase in research that investigated proximal risk factors of suicidal ideation in daily life, of which most are based on Joiner's interpersonal theory of suicide (12–18). A key concept of Joiner's interpersonal theory of suicide, and a development beyond earlier suicide theories, is its ideation-to action framework, which explains why many individuals that think about suicide do not actually commit an attempt.

Joiner's interpersonal theory of suicide is one of the most rigourously researched and empirically supported theories of suicide (19, 20). The theory assumes a range of proximal suicide risk factors, and provides testable predictions of who will most likely develop suicidal ideations and who will most likely attempt suicide. It thus holds much promise to further our understanding of how certain suicide risk factors interact, and provides concrete targets for prevention and intervention efforts. In essence, it proposes that an individual will not die by suicide unless he or she has both the desire to die by suicide and the ability to do so. According to the theory, suicidal desire is caused by the simultaneous presence of two causal risk factors: (1) thwarted belongingness and (2) perceived burdensomeness, and hopelessness about these states (21, 22). Thwarted belongingness describes the experience of alienation from friends, family, or other subjectively important social circles. These comprise loneliness (i.e., feeling disconnected from others) and the absence of reciprocal care (i.e., having no one to turn to). Perceived burdensomeness refers to the view that one's existence is a burden on friends, family members, and/or society. It comprises two facets: self-hate (i.e., hating oneself) and feelings of liability (i.e., viewing one's death as more valuable than personal worth to others). Importantly, these cognitive-affective states are seen as dynamic and influenced by inter- and intra- personal factors such as experiencing family conflict, living alone, lacking social support, and readiness to interpret others behavior as rejection (22).

Specifically relevant for clinical practice are new clinical concepts that, building further on the research above, explicitly focus on imminent, acute risk factors, such as the Suicide Crisis Syndrome [SCS; (23)] and Acute Suicidal Affective Disturbance [ASAD; (24)]. For instance, ASAD is theorized to be characterized by: (1) a geometric increase in suicidal intent over the course of hours or days; (2) one or both of the following: marked social alienation (i.e., perceptions of being a liability on others) and/or marked self-alienation (i.e., perceptions of one's self being a burden); and (3) perceptions that these are hopelessly intractable; and (4) two or more manifestations of overarousal (i.e., insomnia).

Advances in real-time monitoring technology, also called ecological momentary assessment (EMA) or experience sampling (25), in which people's current behaviors and experiences are repeatedly sampled in real time in their natural environments (26), have thus recently made it possible to investigate such proximal and imminent factors as they occur and arise in daily life. Also here, the need and potential for individualized medicine is advocated, in which smartphone-based ecological momentary assessment and passive collection of information from sensors can provide a digital phenotype to develop tailored therapeutic and preventive approaches for suicide (10, 27). The big advantage of including the use of passive mobile sensing, is that it leverages the data people generate every day through their normal phone use without placing any additional burden to them. Emerging studies in this regard indeed suggest the potential utility of passive mobile sensing in predicting mental health [for a review, see (28)], mental health crises [e.g., see the EARS-project; (29)], and suicide risk (30).

Although existing research has now shown the potential short-term predictive value of some of these factors for suicidal ideation, the available evidence is inconclusive and cannot provide clear recommendations for clinical routine care yet (5). For instance, in past studies increases in hopelessness and loneliness went together with momentary suicidal ideation but were limited in predicting short-term change in suicidal ideation (16). To move the field forward, there has been a call for (1) larger, longer studies, (2) studies conducted during critical high-risk periods, and (3) the use of passive mobile sensing information (e.g., via smartphones or wearables that can deliver behavioral data without placing additional burden on participants) to improve predictability of suicidal ideation (5). Indeed, in this regard, projects are rapidly arising that exactly tailor to these needs, such as MAPS (Mobile Assessment for the Prediction of Suicide; https://grantome.com/grant/NIH/U01-MH116923-01), the Emma app [Ecological Momentary Mental Assessment; (31)], or the Smartcrisis Study [Smartphone Survey of Suicidal Risk; (32, 33)]. Preliminary results from this research indeed suggests the feasibility (33, 34) and potential utility of combining EMA with passive mobile sensing in predicting and intervening in suicidal crises (34).

In our study, we aim to build further on this rapidly increasing research by designing and implementing a digital mental health protocol based on psychological theory – the interpersonal theory of suicide – and passive mobile sensing information. We focus on a high-risk population: psychiatric patients after discharge from an inpatient stay who were admitted in the context of suicidal ideation or a suicidal attempt, or expressed suicidal ideations at their intake interview after admission. Especially the month after discharge is a critical period associated with high rates of suicidality and mood deterioration and readmission (35). The objective of this study is to test in a sample of 100 participants whether (1) implementation of a digital mental health protocol or smartphone applications, based on self-reports and behavioral measures, is feasible and accepted and whether (2) suicidal ideation and psychiatric hospital readmission can be predicted from variables derived from these applications.

Methods and Analyses

Selection of Participants

One-hundred participants will be recruited from the Psychiatric University Hospital, Zurich, Switzerland. This number was determined based on a power analysis for multilevel data of a longitudinal study design (36). We considered a three-level nested structure of the longitudinal data with repeated EMAs (Level 1), collected across subjects (Level 2), and nested within different days (Level 3) and the simplest model, an unconditional three-level model (37) with

$\begin{array}{ll}{Y}_{tij}={\gamma }_{000}+u_{00j}+r_{oij}+{ϵ}_{tij}& (1)\end{array}$

where Y is the suicidal ideation at hour t for participant i at day j as modeled by a linear combination of a grand mean suicidality score (γ₀₀₀) averaged across all repeated measures for all participants during all days. In addition, we added three random effect estimates at Level 3 (u_00j), Level 2 (r_oij), and Level 1 (ϵ_tij).

Consequently, we computed the intra class correlation (ICC), the design effect, and finally the power. First, the ICC is defined as the proportion of outcome variation on Level 2 and the expected correlation on Level 1t, and calculated with

$\begin{array}{ll}ICC=\frac{{\tau }_{000}}{({\tau }_{000}+{\sigma }^2)}& (2)\end{array}$

where τ₀₀₀ is the random intercept and σ² the unexplained variability in outcomes. We chose an approximation using the ICC of a previous, similar study (18) with ICC = 0.52. Next, we computed the design effect, a parameter that quantifies the violation of independence on the estimates of the standard error (38), with

$\begin{array}{ll}DesignEffect=1+(m-1)\times ICC& (3)\end{array}$

where m is the number of assessments per subject (m = 5 × 28). This results in a design effect of 73.8 which indicates the need for multilevel modeling (38). Finally, the power can be calculated with

$\begin{array}{ll}Power=\frac{n\times m}{1+(m-1)\times ICC}& (4)\end{array}$

where n is the number of participants, m the number of assessments per participant, adjusted for the ICC. This can be rewritten as

$\begin{array}{ll}n=\frac{(Power\times (1+m-1)\times ICC)}{m}& (5)\end{array}$

Assuming no missing data with Power = 80%, m = 140, and ICC = 0.52, we would need 42 subjects. Yet, missing data especially when dealing with EMA should be taken into account. Thus, we calculated the sample size for different percentages of missing data points (50, 60, 70, and 80%). Results do not suggest a sample size larger than n = 42. Last, due to the imputed ICC, we also computed the sample size with different values for the ICC (Figure 1). Based on this, a sample size of 80 would be sufficient even in the case of an ICC of 1. Considering also the likely dropout rate, we aim at recruiting 100 participants which will allow us to detect the true effect with 80% probability at an alpha level of 0.05.

FIGURE 1

Figure 1. Relationship between the intra class correlation coefficient and the sample size. With increasing correlation a larger sample size is needed to still detect the true effect with a probability of 80% at an alpha level of 0.05.

Besides multilevel modeling, we aim to apply machine leaning models to predict suicidality. The goal of the ML models will be to model the relationships between predictors and outcome (suicidal ideation, suicide attempts during follow up, hospital readmission), which requires equivalent power to detect any given univariate relationship between a dependent and independent variable. Model fit is estimated by permitting high dimensionality while penalizing model fit for increased complexity through the use of a loss function. While power is less of a concern in ML models, reproducibility and over-fitting is a significant risk, requiring strategies such as cross-validation to guard against this risk. Given the relatively small set of theory driven features included in the model, we anticipate n = 100 will allow for model estimation (n = 60) and hold-out cross-validation (n = 40) will be sufficient to train and test an ML model using the proposed predictors to forecast primary outcomes [see also (39, 40)].

Patients are included if they meet the following criteria: (a) admission to the hospital after a suicide attempt or in the context of suicidal ideation, and/or suicidal ideation were identified in the first diagnostic intake interview, (b) sufficient knowledge of the German language, (c) having a smartphone, (d) discharge in accord with a clinician, with established outpatient care contact to the physician or psychologist. Patients are excluded if they meet the following criteria: (a) having plans to leave the greater Zurich area within the study period, (b) sharing a smartphone with another person, (c) being active military personnel (as passive sensing and EMA assessments would be challenging in active duty). There are no age restrictions. Researchers will keep track of all incoming patients in the hospital and contact the treating psychologist or physician in case of eligibility. When a patient meets the inclusion criteria and the treating psychologist or physician approves, the patient will be approached by the researcher and informed about the study.

Based on the Psychiatric University Hospital's report from 2019, patients have an average inpatient stay of 24.6 days. The average patient is 40.2 years old, with females (47.2%) and males (52.8%) almost equally distributed, and admitted mainly because of substance use disorders (27%), schizophrenia spectrum disorders (24%), affective disorders (26%), anxiety disorders (11%), and personality disorders (7.5%).

Procedure and Materials

The study will consist of different parts: a baseline assessment, a 4-week period of ecological momentary assessment in which the smartphone applications run, and a follow-up. Participants will be reimbursed with up to 120 CHF if they answer the smartphone applications' questions in more than 60 % of the time.

Baseline Assessment

The baseline assessment entails (1) detailed information about the study and informed consent, (2) assessment of the current mental disorders with the Mini International Neuropsychiatric Interview [MINI version (14)], (3) a short video-taped semi-structured qualitative interview, (4) electronic questionnaires that evaluate relevant psychological variables, and (5) the installation of the smartphone applications on participants' phones. During the baseline assessment, participants will also get a booklet that contains additional information on the aims of the study, crisis information in case of emergency, and the smartphone applications. The baseline assessment will thus occur within the hospital stay, after patients are able to and have provided informed consent to participate in the study. The exact timing of this assessment is expected to vary, as it depends on patients' acute symptom severity and their capacity to perform an interview, practical constraints and the schedule of the patient.

Table 1 lists the questionnaires and other assessments that will be used at baseline and/or at follow-up. These measures are thus a combination of self- and clinician-reports (MINI), and a video-taped qualitative interview for which participants provide separate consent. During the qualitative video interview, participants answer questions about experiences with different valences (i.e., positive, negative, neutral) and temporal dimensions (i.e., past, present, future). The videos will be used to derive markers for psychopathology using physiology, facial activity, language use, and vocal characteristics.

TABLE 1

Table 1. Questionnaires and assessments conducted at baseline and/or follow-up.

Ecological Momentary Assessment

Two smartphone applications will be installed on participants' smartphones. The first application (SIMON-SELF) is used for collecting self-report data according to a pre-defined ecological momentary assessment protocol. The second application collects smartphone sensor data (SIMON-SENSE). The two applications are made available for Android and iOS and described in more detail in the following paragraphs.

SIMON-SELF

MobileCoach (www.mobile-coach.eu) (53, 54), an open source software platform for delivering ecological momentary assessments and digital health interventions, was used to develop the SIMON-SELF application. The configuration of the ecological momentary assessments, i.e., timing and the self-report items, is defined via a graphical user interface by the co-authors of this paper on the MobileCoach server. The server then sends this content to SIMON-SELF, a mobile application that uses a conversational agent (named Simon in this study) to administer the self-reports to the study participants. A conversational agent is a computer program that imitates a human being, and which has the potential to establish a working alliance with participants (55) and thus, to increase involvement with the application (56, 57). Exemplary screenshots of SIMON-SELF are depicted in Figure 2.

FIGURE 2

Figure 2. The SIMON-SELF application: welcome screen (left), the chat screen (middle) and the menu (right).

After the application is installed on the participants' smartphone via Google's Play Store or Apple's App Store, conversational agent Simon provides details about the mobile application. This includes a demonstration on how to fill out the self-reports. Then, Simon asks participants to indicate whether they are about to leave the clinic, so that the experience sampling protocol covering the period post-discharge will be promptly initiated for the day following their discharge.

From then on, participants will be asked to answer the experience sampling surveys 5 times per day during a defined period of 12 h, according to a stratified random interval scheme with the time frame being divided into five equal intervals. This means that individualization of the time frame is possible, but only with a fixed range of 12 h (e.g., from 9 AM to 9 PM, from 10 AM to 10 PM, et cetera). Participants will be asked upon installation of the application what timeframe they prefer. Every day, Simon will greet participants in the morning, wish them good night in the evening, and will prompt them to answer the questions. The specific questions that will be asked, can be found in Table 2. One block of questions is only asked in the morning (e.g., about sleep), one block only in the evening (e.g., about mood), and there is one block that shows up with every survey.

TABLE 2

Table 2. Self-report items of the ecological momentary assessment administered through SIMON-SELF.

Compliance to the protocol is promoted through multiple strategies. Every second day, Simon gives participants an update on their compliance, and the feedback depends on more/less than 60% compliance. Simultaneously, participants will be sent automatically generated text messages with the same feedback via SMS, i.e., an additional communication channel compared to in-app chat messages. Finally, Simon tells participants every week that the researchers are very grateful for their participation in the study, and that they are helping to improve future suicide prevention methods. To further increase compliance, researchers will contact participants in case of non-compliance. Finally, participants will receive a personalized summary of their collected self-reports after successful study completion. Specifically, they receive visual feedback, containing a series of charts that summarize their changes in key variables, e.g., sleep, suicidality, other psychological characteristics and feelings, over the time of the study.

In addition, an emergency button is made available in the side menu of SIMON-SELF (see screenshot three, Figure 2). It provides three different helpline numbers, according to urgency and specific need. Participants receive information about these helplines and their services upon installation of the application. There is also a number available on the application that participants can reach in case of technical issues.

SIMON-SENSE

To assess relevant context variables such as physical activity, sleep, and social connectedness, the mobile sensing application SIMON-SENSE records sensor data commonly available via smartphones. We use the open source framework AWARE (67) for this purpose. SIMON-SENSE records and sends the data in a secure way to a server located at the university of the corresponding author (University of Zurich). The specific data sources, data types and collection frequencies are listed in Table 3. The application runs in the background, and thus requires no interaction with study participants. Because this application might drain the battery of participants' phones, Simon reminds participants every evening to charge their smartphone.

TABLE 3

Table 3. Data sources, data types and collection frequency of the SIMON-SENSE application.

Follow-Up

After 4 weeks of ecological momentary assessment, a follow-up assessment takes place at the Psychiatric University Hospital, Zurich. Participants will fill out questionnaires (see Table 1) of which most are validated and have been assessed at baseline already. To gain insight into user experience of the apps, participants will fill in the Research Experience Questionnaire and the App Questionnaire. In addition to these quantitative measures, research assistants will be instructed to encourage participants to give also qualitative feedback on the app usage. Both sources of feedback will be valuable for the further development, particularly of the in-house developed SIMON-SELF app and the design of subsequent studies. Finally, participants receive payment for their participation in the study.

Data Management

The experience sampling and the passive mobile sensing data will be transmitted via a Secure Sockets Layer (SSL) connection to a study server. This server can only be accessed by a password. Data from the baseline and follow-up questionnaires will also be saved on this study server. The study server is provided by the University of Zurich, Switzerland. To match different datasources, a unique user number is generated for each participant. The only file containing participant's full personal information and respective unique user number, is kept in a separate document and stored in a locked file cabinet.

Data Analyses

The main research aim of this study is to investigate short-term predictors of suicidal ideation and psyciatric hospital readmission in a high risk-population. To this end, two kinds of analyses are planned.

First, prediction models using various learning algorithms will be developed. The development of such models involves several steps. In a first step, the raw sensor data has to be preprocessed involving feature extraction, scaling, selection, and dimensionality reduction. Smartphone data “features” derived from sensor data and the experience sampling indices as well as data from the baseline questionnaires will then be fed into machine learning models to identify the variables, and combinations thereof, that predict suicidal ideation and psychiatric hospital readmission. In a second step, the data will be split into a training and test data set to assess how the derived algorithms generalize to new data (68). The training dataset will also be split into subsets and k-fold cross-validations will be applied. The performance of the resulting model will then be evaluated using the test data set. This procedure will be repeated for various learning algorithms (e.g., random forest, support vector machines). The learning algorithms will also discard irrelevant information that does not help to improve the predictive value of the model using partitioning for categorical states (is suicidal ideation high/low, were participants readmitted to the hospital). After comparing the performance across algorithms, the best overall model will be selected.

We expect to construct a model that efficiently predicts suicidal ideation and psychiatric hospital readmission using a combination of sensory data and psychological data.

Second, longitudinal multilevel models will be applied to predict suicidal ideation and psychiatric hospital readmission from a combination of predictors based on theory. This will allow to compare between patient differences (between-person level) and to make predictions on an individual level (within-person level) by fitting individual symptom trajectories. Considering the dynamic nature of suicidal ideation, it is cruicial to identify what predicts within-person changes.

Ethics

This study follows ethical and safety guidelines, such as those put forward by Nock and others (69). In accordance with these guidelines, participants will not be excluded on the basis of elevated risk of suicide, participants will be elaborately informed before participation on all suggested elements (e.g., whether responses will trigger intervention actions; providing participants with information about who will have access to their data), and recommended technical and safety procedures are in place (e.g., figuring out what to do when technology fails, and providing participants with standardized informations on items of data-collection). Regarding safety specifically, all participants will receive detailed information about local help lines in case of crisis, and emergency. This information will be presented with the mobile application multiple times throughout the 4-weeks assessment. A standard operation procedure is established in cases of emergencies according to which the researchers are going to act.

We decided not to monitor and pro-actively respond to various levels of risk in real-time with interventions (although suggested by Nock and others), due to several reasons. First, there is currently no agreement on how to determine a participant's current level of risk and criteria for acute level of risk. Further, data cannot be monitored continuously due to several practical reasons (specifically, data are only uploaded when a Wi-Fi connection is available and our study does not provide 24-7 tracking of the data overnight). This study is not an intervention study, but rather a naturalistic study that monitors potentially powerful predictors of suicidal ideation and hospital readmission, as well as suicidal ideation itself. It is stressed to participants that this is not an intervention study, but that the information collected as part of this study will inform and help develop such efforts. Participants can thus only be enrolled in the study if they have a physician and/or psychotherapist attending to them following discharge. Informed consents are obtained after patients received elaborate information about the study procedures and the fact that they can exit the study at any time.

The setup of the study has been discussed with clinicians, psychologists, and patients and piloted to minimize any potential risks or problems. Treating physicians and psychotherapists are involved when patients are approached and enrolled into the study.

The study was reviewed and approved by the Ethics Committee of the Faculty of Arts and Social Sciences of the University of Zurich, Switzerland. All collected data will be anonymised. Results will be published in medical and technical peer-reviewed journals.

Discussion

This study builds further on an emerging line of research by testing in a large sample of high-risk individuals whether (1) a digital mental health protocol with self-reports and behavioral measures can be implemented and whether (2) suicidal ideation and psychiatric hospital readmission can be predicted from variables derived from this protocol. The results from this study will build on and extend the growing body of research on prediction markers of suicidal ideation by mobile health technology [for an overview see (5)]. Identifying reliable prediction markers of suicidal ideation is crucial to help develop just-in-time and cost-effective interventions. For instance, information about these predictors could then be fed back to clinicians and mental health services in real time to provide the support and interventions needed by each individual patient.

A better treatment of suicidal ideation is of vital importance as suicide is a major public health concern. As a consequence, it has been placed high on many national and international research agendas. In addition to being one of the most dramatic intrapersonal consequences of mental health problems, its interpersonal and economic costs are also enormous [e.g., (70)]. Digital technologies provide exciting opportunities to help reduce the number of suicide by accounting for particular challenges associated with its prevention.

Limitations

To optimize continued participation in this population, in which drop-out and low compliance are common problems, and because there is no intervention aspect to the study for participants, we decided to reimburse them. However, this decision may limit the ecological validity of the study in the sense of being comparable with real-world usage of smartphone applications for high-risk suicidal individuals (who are not reimbursed). Further, we decide to conduct a follow-up after four weeks, immediately after the EMA-part of the study, because of multiple reasons. First, we aim to diminish participant drop-out. Second, the first weeks after psychiatric discharge contain a much higher risk for suicide than any period thereafter or other treatment events (71–73). However, this choice has as a disadvantage that given the rarity of suicide, a low incidence of suicidal crises is expected to occur in such a short period. Finally, we determined sample size on power considerations for multilevel data of a longitudinal study design, and acknowledge that this is on the lower side for machine learning models.

Data Availability Statement

Data will be available on OSF.

Ethics Statement

The studies involving human participants were reviewed and approved by PhF UZH Ethics comittee (IRB). The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individuals for the publication of any potentially identifiable images or data included in this article.

Author Contributions

BK and US conceptualized and initiated the project. HS, MC, SV, and ES provided input on psychiatric background, and recruitment and feasibility. PS and TK provided input on the technical background and wrote the paragraphs about conversational agents, sensor data collection, and MobileCoach for ecological momentary assessments. LS wrote the first draft of the manuscript under supervision of BK. AR and SH contributed to further versions of the manuscript. All authors contributed to the article and approved the submitted version.

Funding

The project is co-funded by the Swiss National Foundation's Digital Lives Funding Scheme, Award number 10DL12_183251 to BK and US.

Conflict of Interest

The authors PS and TK are affiliated with the Center for Digital Health Interventions, a joint initiative of the Department of Management, Technology, and Economics at ETH Zurich and the Institute of Technology Management at the University of St. Gallen, which is funded in part by the Swiss health insurer CSS. TK is also co-founder of Pathmate Technologies, a university spin-off company that creates and delivers digital clinical pathways and has used the open source MobileCoach platform for that purpose, too. However, Pathmate Technologies is not involved in the intervention described in this paper.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank Nina Klee, Carlo Berther, and Sarina Blaser for their invaluable help with the study setup, patient recruitment, and data collection. We would also like to thank all physicians and psychologists who helped recruiting patients, as well as the patients themselves for their efforts invested in this study.

References

1. Larsen ME, Nicholas J, Christensen H. A systematic assessment of smartphone tools for suicide prevention. PLoS ONE. (2016) 11:e0152285. doi: 10.1371/journal.pone.0152285

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Müller AM, Blandford A, Yardley L. The conceptualization of a just-in-time adaptive intervention (JITAI) for the reduction of sedentary behavior in older adults. Mhealth. (2017) 37. doi: 10.21037/mhealth.2017.08.05

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Wahle F, Kowatsch T, Fleisch E, Rufer M, Weidt S. Mobiel sensing and support for people with depression: a pilot trial in the wild. JMIR Mhealth Uhealth. (2016) 4:e111. doi: 10.2196/mhealth.5960

PubMed Abstract | CrossRef Full Text | Google Scholar

4. World Health Organization. Preventing Suicide: A Global Imperative. World Health Organization (2014). Available online at: https://www.who.int/health-topics/suicide

Google Scholar

5. Kleiman EM, Nock MK. Real-time assessment of suicidal thoughts and behaviors. Curr Opin Psychol. (2018) 22:33–7. doi: 10.1016/j.copsyc.2017.07.026

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Spangenberg L, Forkmann T, Glaesmer H. Investigating dynamics and predictors of suicidal behaviors using ambulatory assessment. Neuropsychiatrie. (2015) 29:139–43. doi: 10.1007/s40211-015-0142-1

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Ballard ED, Gilbert JR, Wusinich C, Zarate CA Jr. New methods for assessing rapid changes in suicide risk. Front Psychiatry. (2021) 12. doi: 10.3389/fpsyt.2021.598434

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Berrouiguet S, Perez-Rodriguez MM, Larsen M, Baca-Garcia E, Courtet P, Oquendo M. From eHealth to iHealth: transition to participatory and personalized medicine in mental health. J Med Int Res. (2018) 20:e2. doi: 10.2196/jmir.7412

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Torous J, Larsen ME, Depp C, Cosco TD, Barnett I, Nock MK, et al. Smartphones, sensors, and machine learning to advance real-time prediction and interventions for suicide prevention: a review of current progress and next steps. Curr Psychiatry Rep. (2018) 20:51. doi: 10.1007/s11920-018-0914-y

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Melia R, Hickey FK, Bogue J, Duggan J, O' Sullivan M, Young K. Mobile health technology interventions for suicide prevention: systematic review. JMIR Mhealth Uhealth. (2020) 8:e12516. doi: 10.2196/12516

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychol Bull. (2017) 143:187–232. doi: 10.1037/bul0000084

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Armey MF, Brick L, Schatten HT, Nugent NR, Miller IW. Ecologically Assessed Affect and Suicidal Ideation Following Psychiatric Inpatient Hospitalization. Gen Hosp Psychiatry. (2018) 63:89–96. doi: 10.1016/j.genhosppsych.2018.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Ben-Zeev D, Young MA, Depp CA. Real-time predictors of suicidal ideation: mobile assessment of hospitalized depressed patients. Psychiatry Res. (2012) 197:55–9. doi: 10.1016/j.psychres.2011.11.025

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Czyz EK, Horwitz AG, Yeguez CE, Ewell F, King CA. Parental self-efficacy to support teens during a suicidal crisis and future adolescent emergency department visits and suicide attempts. J Clin Child Adol Psychol. (2018) 47:S384–S396. doi: 10.1080/15374416.2017.1342546

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hallensleben N, Glaesmer H, Forkmann T, Rath D, Strauss M, Kersting A, et al. Predicting suicidal ideation by interpersonal variables, hopelessness and depression in real-time. An ecological momentary assessment study in psychiatric inpatients with depression. Eur Psychiatry. (2019) 65:43–50. doi: 10.1016/j.eurpsy.2018.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Kleiman EM, Turner BJ, Fedor S, Beale EE, Huffman JC, Nock MK. Examination of real-time fluctuations in suicidal ideation and its risk factors: results from two ecological momentary assessment studies. J Abnor Psychol. (2017) 126:726–38. doi: 10.1037/abn0000273

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Kyron MJ, Hooke GR, Page AC. Daily assessment of interpersonal factors to predict suicidal ideation and non-suicidal self-injury in psychiatric inpatients. J Cons Clin Psychol. (2018) 86:556. doi: 10.1037/ccp0000305

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Littlewood DL, Kyle SD, Carter LA, Peters S, Pratt D, Gooding P. Short sleep duration and poor sleep quality predict next-day suicidal ideation: an ecological momentary assessment study. Psychol Med. (2019) 49:403–11. doi: 10.1017/S0033291718001009

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Joiner TE Jr, Van Orden KA, Witte TK, Selby EA, Ribeiro JD, Lewis R, et al. Main predictions of the interpersonal-psychologica theory of suicidal behavior: empirical tests in two samples of young adults. J Abno Psychol. (2009) 118:634–46. doi: 10.1037/a0016500

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Van Orden K, Conwell Y. Suicides in late life. Curr Psychiatry Rep. (2011) 13:234–41. doi: 10.1007/s11920-011-0193-3

CrossRef Full Text | Google Scholar

21. Joiner TE Jr. Why People die by Suicide. Cambridge, MA: Harvard University Press (2005).

Google Scholar

22. Van Orden KA, Witte TK, Cukrowicz KC, Braithwaite SR, Selby EA, Joiner TE Jr. The interpersonal theory of suicide. Psychol. Rev. (2010) 117:575–600. doi: 10.1037/a0018697

CrossRef Full Text | Google Scholar

23. Schuck A, Calati R, Barzilay S, Bloch-Elkouby S, Galynker I. Suicide crisis syndrome: a review of supporting evidence for a new suicide-specific diagnosis. Behav Sci Law. (2019) 37:223–39. doi: 10.1002/bsl.2397

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Tucker RP, Michaels MS, Rogers ML, Wingate LR, Joiner TE. Construct validity of a proposed new diagnostic entity: acute suicidal affective disturbance (ASAD). J Affect Dis. (2016) 189:365–78. doi: 10.1016/j.jad.2015.07.049

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Larson R, Csikszentmihalyi M. The experience sampling method. N Direc Methodol Soc Behav Sci. (1983) 15:41–56.

Google Scholar

26. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Ann Rev Clin Psychol. (2008) 4:1–32. doi: 10.1146/annurev.clinpsy.3.022806.091415

CrossRef Full Text | Google Scholar

27. Barrigon ML, Courtet P, Oquendo M, Baca-Garcia E. Precision medicine and suicide: an opportunity for digital health. Curr Psychiatry Rep. (2019) 12:1–8. doi: 10.1007/s11920-019-1119-8

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Cornet VP, Holden RJ. Systematic review of smartphone-based passive sensing for health and well-being. J Biomed Infor. (2018) 77:120–32. doi: 10.1016/j.jbi.2017.12.008

CrossRef Full Text | Google Scholar

29. Lind MN, Byrne ML, Wicks G, Smidt AM, Allen NB. The effortless assessment of risk states (EARS). JMIR Mental Health. (2018) 5:e10334. doi: 10.2196/10334

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Haines-Delmont A, Chahal G, Bruen AJ, Wall AK, Khan CT, Sadashiv R, et al. Testing suicide risk prediction algorithms using phone measurements with patients in acute mental health settings: feasibility study. JMIR Mhealth Uhealth. (2020) 8:e15901. doi: 10.2196/15901

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Morgieve M, Genty C, Azé J, Dubois J, Leboyer M, Vaiva G, et al. A digital companion: the EMMA app, for ecological momentary assessment and prevention of suicide: quantitative case series study. JMIR Mhealth Uhealth. (2020) 8:e15741. doi: 10.2196/15741

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Berrouiget S, Barrigon ML, Castroman JL, Courtet P, Artés-Rodriguez A, Baca-Garcia E. Combining mobile-health (mHealth) and articial intelligence (AI) methods to avoid suicide attempts: the smartcrises study protocol. BMC Psychiatry. (2019) 19:1–9. doi: 10.1186/s12888-019-2260-y

CrossRef Full Text | Google Scholar

33. Moreno-Munoz P, Romero-Medrano L, Moreno A, Herrera-Lopez J, Baca-Garcia E, Artés-Rodriguez A. Passive detection of behavioral shifts for suicide attempt prevention. Arxiv (2020). Retrieved from: https://arxiv.org/abs/2011.09848

Google Scholar

34. Porras-Segovia A, Molina-Madueno RM, Berrouiguet S, Lopez-Castroman J, Barrigon ML, Pérez-Rodriguez MS, et al. Smartphone-based ecological momentary assessment (EMA) in psychiatric patients and student controls: a real-world feasibility study. J Affect Dis. (2020) 274:733–41. doi: 10.1016/j.jad.2020.05.067

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Chung DT, James Ryan C, Hadzi-Pavlovic D, Preet Singh S, Stanton C, Large MM. Suicide rates after discharge from psychiatric facilities: a systematic review and meta-analysis. JAMA Psychiatry. (2017) 74:694–702. doi: 10.1001/jamapsychiatry.2017.1044

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Diggle P, Diggle PJ, Heagerty P, Liang KY, Heagerty PJ, Zeger S. Analysis of Longitudinal Data. Oxford: Oxford University Press (2002).

Google Scholar

37. Peugh JL, Heck RH. Conducting three-level longitudinal analyses. J Early Adol. (2017) 37:7–58. doi: 10.1177/0272431616642329

CrossRef Full Text | Google Scholar

38. Peugh JL. A practical guide to multilevel modeling. J School Psychol. (2010) 48:85–112. doi: 10.1016/j.jsp.2009.09.002

CrossRef Full Text | Google Scholar

39. Hua J, Xiong Z, Lowey J, Suh E, Dougherty ER. Optimal number of features as a function of sample size for various classification rules. Bioinformatics. (2005) 21:1509–15. doi: 10.1093/bioinformatics/bti171

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Vabalas A, Gowen E, Poliakoff E, Casson AJ. Machine learning algorithm validation with a limited sample size. PLoS ONE. (2019) 14:e0224365. doi: 10.1371/journal.pone.0224365

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The mini-international neuropsychiatric interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. (1998) 59:22–33. doi: 10.1037/t18597-000

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Beck AT, Steer RA, Brown GK. Beck depression. Inventory-II. (1996) 78:490–8. doi: 10.1037/t00742-000

CrossRef Full Text | Google Scholar

43. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Person Soc Psychol. (1988) 54:1063–70. doi: 10.1037/0022-3514.54.6.1063

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Spitzer RL, Kroenke K, Williams JB, Patient Health Questionnaire Primary Care Study Gr. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. JAMA. (1999) 282:1737–44. doi: 10.1001/jama.282.18.1737

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Chu C, Hom MA, Stanley IH, Gai AR, Nock MK, Gutierrez PM, et al. Non-suicidal self-injury and suicidal thoughts and behaviors: a study of the explanatory roles of the interpersonal theory variables among military service members and veterans. J Consu Clin Psychol. (2018) 56–68. doi: 10.1037/ccp0000262

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Bernstein DP, Fink L, Handelsman L, Lovejoy M, Wenzel K, Sapareto E, et al. Initial reliability and validity of a new retrospective measure of child abuse and neglect. Am J Psychiatry. (1994) 1132–6. doi: 10.1176/ajp.151.8.1132

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Weathers FZ, Blake DD, Schnurr PP, Kaloupek DG, Marx BP, Keane TM. The Life Events Checklist For DSM-5 (LEC-5). (2013). Available online at: www.ptsd.va.gov. Retrieved from: (March, 13, 2019).

PubMed Abstract | Google Scholar

48. Van Orden KA, Cukrowicz KC, Witte TK, Joiner JT. Thwarted belongingness and perceived burdensomness: construct validity and psychometric properties of the interpersonal needs questionnaire. Psychol Assess. (2012) 24:197–215. doi: 10.1037/a0025358

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Kliem S, Lohmann A, Möble T, Brähler E. Psychometric properties and measurement invariance of the Beck hopelessness scale (BHS): results from a German representative population sample. BMC Psychiatry. (2018) 1:110. doi: 10.1186/s12888-018-1646-6

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Spangenberg L, Glaesmer H, Scherer A, Gecht J, Barke A, Mainz V, et al. Fearlessness about death and suicidal behavior: psychometric properties of the german version of the revised acquired capability for suicide scale (ACSS-FAD). Psychiatr Praxis. (2016) 43:95–100. doi: 10.1055/s-0034-1387375

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Schwarzer R, Jerusalem M. Generalized self-efficacy scale. In: Weinmann J, Wright S, Johnston M, editors. Measures in Health Psychology: A User's Portfolio. Causal and Control Beliefs. Windsor: Nfer-Nelson (1995). p. 35–37.

52. Snyder CR, Harris C, Anderson JR, Holleran SA, Irving LM, Sigmon ST. The will and the ways: development and validation of an individual-differences measure of hope. J Person Soc Psychol. (1991) 60:570–85. doi: 10.1037/0022-3514.60.4.570

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Filler A, Kowatsch T, Haug S, Wahle F, Staake T, Fleisch E. MobileCoach: A Novel Open-Source Platform For the Design of Evidence-Based, Scalable, and Low-Cost Behavioral Health Interventions -Overview and Preliminary Evaluation of the Public Health Context. 14th ed. St. New York, NY: Annual Wireless Telecommunications Symposium (2015).

Google Scholar

54. Kowatsch T, Volland D, Shih I, Rüegger D, Künzler F, Barata F, et al. Design and Evaluation of a Mobile Chat App For the Open Source Behavioral Health Intervention Platform MobileCoach. Nternational Conference on Design Science Research in Information System and Technology. Cham: Springer (2017). p. 485–489. Available online at: https://link.springer.com/chapter/10.1007/978-3-319-59144-5_36#citeas

Google Scholar

55. Bickmore T, Gruber A, Picard R. Establishing the computer-patient working lliance in automated health behavior change interventions. Pat Educ Couns. (2005) 59:21–30. doi: 10.1016/j.pec.2004.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Tinschert P, Barata F, Kramer J, Rassouli F, Steurer-Stey C, Puhan M, et al. Don't Lose Heart: Preliminary Engagement Results in an Ecological Momentary Assessment (EMA) Study Evaluating Digital Biomarkers For Asthma. Auckland: Abstract presented at the International Society for Research on Internet Interventions (ISRII) (2019).

Google Scholar

57. Hauser-Ulrich S, Künzli H, Meier-Peterhans D, Kowatsch T. A smartphone-based health care chatbot to promote self-management of chronic pain (SELMA): pilot randomized controlled trial. JMIR mHealth and uHealth. (2020) 8:e15806. doi: 10.2196/15806

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Espie CA, Kyle SD, Hames P, Gardani M, Fleming L, Cape J. The sleep condition indicator: a clinical screening tool to evaluate insomnia disorder. BMJ Open. (2014) 4:e004183. doi: 10.1136/bmjopen-2013-004183

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Kroenke K, Spitzer RL, Williams JB. The Patient Health Questionnaire-2: validity of a two-item depression screener. Med Care. (2003) 11:1248–92. doi: 10.1097/01.MLR.0000093487.78664.3C

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Forkmann T, Spangenberg L, Rath D, Hallensleben N, Hegerl U, Kersting A, et al. Assessing suicidality in real time: a psychometric evaluation of self-report items for the assessment of suicidal ideation and its proximal risk factors using ecological momentary assessments. J Abno Psychol. (2018) 127:758–69. doi: 10.1037/abn0000381

CrossRef Full Text | Google Scholar

61. Fraser C, Keating M. The effect of a creative art program on self-esteem, hope, perceived social support, and self-efficacy in individuals with multiple sclerosis: a pilot study. J Neurosci Nurs. (2014) 46:330–6. doi: 10.1097/JNN.0000000000000094

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Ryon HS, Gleason ME. The locus of control in daily life. Person Soc Psychol Bull. (2014) 40:121–31. doi: 10.1177/0146167213507087

CrossRef Full Text | Google Scholar

63. Betella A, Verschure PF. The affective slider: a digital self-assessment scale for the measurement of human emotions. PLoS ONE. (2016) 11:e0148037. doi: 10.1371/journal.pone.0148037

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Wilhelm P, Schoebi D. Assessing mood in daily life. Eur J Pyschol Assess. (2007) 23:258–67. doi: 10.1027/1015-5759.23.4.258

CrossRef Full Text | Google Scholar

65. Laurenceau J-P, Barrett LF, Pietromonaco PR. Intimacy as an interpersonal process: the importance of self-disclosure, partner disclosure, and perceived partner responsiveness in interpersonal exchanges. J Person Soc Psychol. (1998) 74:1238. doi: 10.1037/0022-3514.74.5.1238

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Gadassi R, Bar-Nahum LE, Newhouse S, Anderson R, Heiman JR, Rafaeli E, et al. Perceived partner responsiveness mediates the association between sexual and marital: a daily diary study in newlywed couples. Arch Sex Behav. (2016) 45:109–20. doi: 10.1007/s10508-014-0448-2

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Ferreira D, Vassilis K, Dey AK. AWARE: mobile context instrumentation framework. Front ICT. (2015) 6. doi: 10.3389/fict.2015.00006

CrossRef Full Text | Google Scholar

68. Dobbin KK, Simon RM. Optimally splitting cases for training and testing high dimensional classifiers. BMC Med Genom. (2011) 4. doi: 10.1186/1755-8794-4-31

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Nock MK, Kleiman EM, Abraham M, Bentley KH, Brent DA, Buonopane RJ, et al. Consensus statement on ethical & safety practices for conducting digital monitoring studies with people at risk of suicide and related behaviors. Psychiatr Res Clin Prac. (2020) 3:57–66. doi: 10.1176/appi.prcp.20200029

CrossRef Full Text | Google Scholar

70. Yang B, Lester D. Recalculating the economic cost of suicide. Death Stud. (2007) 31:351–61. doi: 10.1080/07481180601187209

CrossRef Full Text | Google Scholar

71. Goldacre M, Seagroatt V, Hawton K. Suicide after discharge from psychiatric inpatient care. Lancet. (1993) 342:283–6. doi: 10.1016/0140-6736(93)91822-4

CrossRef Full Text | Google Scholar

72. Qin P, Nordentoft M. Suicide risk in relation to psychiatric hospitalization: evidence based on longitudinal registers. Arch Gen Psychiatry. (2005) 62:427–32. doi: 10.1001/archpsyc.62.4.427

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Valenstein M, Kim HM, Ganoczy D, McCarthy JF, Zivin K, Austin KL, et al. Higher-risk periods for suicide among VA patients receiving depression treatment: prioritizing suicide prevention efforts. J Affect Dis. (2009) 112:50–58. doi: 10.1016/j.jad.2008.08.020

PubMed Abstract | CrossRef Full Text | Google Scholar