Systematic review of machine learning utilization within outpatient psychodynamic psychotherapy research

Introduction: Although outpatient psychodynamic psychotherapy is effective, there has been no improvement in treatment success in recent years. One way to improve psychodynamic treatment could be the use of machine learning to design treatments tailored to the individual patient's needs. In the context of psychotherapy, machine learning refers mainly to various statistical methods, which aim to predict outcomes (e.g., drop-out) of future patients as accurately as possible. We therefore searched various literature for all studies using machine learning in outpatient psychodynamic psychotherapy research to identify current trends and objectives.

Methods: For this systematic review, we applied the Preferred Reporting Items for systematic Reviews and Meta-Analyses Guidelines.

Results: In total, we found four studies that used machine learning in outpatient psychodynamic psychotherapy research. Three of these studies were published between 2019 and 2021.

Discussion: We conclude that machine learning has only recently made its way into outpatient psychodynamic psychotherapy research and researchers might not yet be aware of its possible uses. Therefore, we have listed a variety of perspectives on how machine learning could be used to increase treatment success of psychodynamic psychotherapies. In doing so, we hope to give new impetus to outpatient psychodynamic psychotherapy research on how to use machine learning to address previously unsolved problems.

Introduction

Outpatient psychodynamic psychotherapy is effective in treating various psychological disorders (1–3). Further positive effects include a reduced number of sick leaves, a reduction of health care utilization, less psychiatric hospitalizations after therapy, and a reduced relapse rates for depression (4–6). A number of factors that predict successful therapy are also known, such as improving the working alliance (7, 8), therapeutic agency (7, 9, 10) or the patient's ability to perceive emotions (11, 12) which lead to a reduction in symptom burden. However, as Leichsenring et al. (13) point out, recent substantial improvements in treatment success have been scarce. The authors (13) recommend that future studies focus primarily on non-responders and drop-outs to improve available treatments. Identifying characteristics and features of non-responders and drop-outs would allow treatment to be tailored more specifically to the patients (13). However, Leichsenring et al. (13) neglect the fact that theoretical or statistical models are needed to accurately predict whether the patient's treatment will be successful or unsuccessful. We argue that such models can be developed with machine learning.

Machine learning is a field of computer science, in which the “computer” is supposed to “learn” models from data (14). “Learning” in this context means that statistical models are adapted to the data until they optimally perform a previously defined task, for example predicting drop-out rates of psychotherapy (14, 15). These statistical models can be the same methods that are used in classical statistical approaches, such as regression analyses. There is therefore no clear boundary between machine learning and classical statistical approaches (14). However, machine learning differs from classical statistical approaches in the primary way the developed models are evaluated. In classical statistical approaches, the developed models are assessed primarily with the help of statistical significance, explained variance, and many other characteristic values (16). In contrast, models in the machine learning approach are assessed primarily by how well they can perform the task they have “learned” on new data that is unknown to the often iterative model fitting process (14, 15, 17). In practice, this means that a model is “taught” by means of a first data set and then evaluated on a second data set. Machine learning approaches can further be divided into unsupervised and supervised learning (18). The primary goal of unsupervised learning is to discover relationships and structures in the data (14, 15). Commonly used statistical models for unsupervised machine learning include explanatory factor analysis, k-means clustering, and hierarchical clustering (19). While supervised learning also discovers correlations and structures in the data, the goal is to determine the value of a dependent variable as accurately as possible (14, 15). A prerequisite for this is that the dependent variable is known, both in the data set in which the model is being “taught”, and in which it is being evaluated. Commonly used statistical models for supervised machine learning include regression analysis, support vector machines, random forest, and latent discriminant analysis (19). For a more detailed description of machine learning and its own terminology, the interested reader is referred here to Dwyer et al. and Bi et al. (14, 15).

Because models developed using the machine learning approach are primarily evaluated for their ability to perform a previously defined task for new unknown data, they often perform better (assuming access to an appropriate dataset) in tasks such as predicting whether a patient's treatment will be successful, as opposed to models developed using the classical statistical approach. Machine learning thus has the potential to develop models that could lead to psychodynamic psychotherapeutic treatments being more successful. However, it is unclear whether machine learning is currently used in psychodynamic psychotherapy research. In 2019, Aafjes-van Doorn et al. (19) found 51 studies which utilized machine learning to analyse psychotherapy. Most of those studies were initial proof-of-concept studies, which either predicted the outcome of therapy, or automatically rated patient behavior for further analyses. Most of these 51 studies used transcripts of psychotherapy sessions as data and utilized supervised machine learning to answer their research questions. However, Aafjes-van Doorn et al. (19) did not differentiate between specific treatment approaches. Machine learning may have other applications in psychodynamic psychotherapy research because of the focus on the patient's unconscious. Yet, to the best of our knowledge, there seem to be no current systemic reviews about machine learning within outpatient psychodynamic psychotherapy research. Therefore, we focused our literature review on the use of machine learning in outpatient psychodynamic psychotherapy research.

Methods

For this review, we applied the Preferred Reporting Items for systematic Reviews and Meta-Analyses (PRISMA) Guidelines.

Search strategy and eligibility criteria

We searched the two most comprehensive databases regarding psychotherapy research, “PsycInfo” and “PubMed”. In preparation for this mini-review, we searched several databases (PsycInfo, PubMed, Heidi, Google Scholar and IEEE Xplore) for relevant literature and did not obtain any additional results beyond those from PubMed and PsycInfo. Therefore, we estimated that there would be little loss of knowledge if we omitted further databases. All 36 combinations of the terms (psychothera^* OR thera^* OR clinical assessment) AND (machine learning OR artificial intelligence OR neural network OR deep learning) AND (patient^* OR client^* OR mental health) with no limitation on publication year were searched. As there is a corpus of theoretical work comparing neural processes to artificial intelligence that considers how to use the conclusions for psychotherapy, we added the term (patient^* OR client^* OR mental health) to the search. We sought to omit such work. The two searches were conducted on 17th September 2021 and 2nd January 2023, respectively.

Eligibility criteria

To be eligible, studies had to be original works, treat their patients with outpatient psychotherapy and use machine learning as a statistical method. Results were limited to publications in English. There were no further eligibility criteria.

Selection process

The first and second author read all abstracts of the articles and selected the studies which appeared to meet the eligibility criteria. In a second step, the manuscripts were read and discussed among the first and second author. During this stage, studies whose psychotherapeutic treatment was not psychodynamic, or whose treatment consisted only of diagnostics, were not considered for this review. Furthermore, all studies that did not use machine learning as a statistical method were removed. Lastly, studies which treated patients with several treatment approaches, yet did not differentiate between them, were excluded, as it was impossible to attribute the results to one specific treatment approach.

Data items

The first and second author independently retrieved the research question and the respective use of machine learning from the included studies. They also retrieved the sample size and data used for machine learning.

Bias assessment

To assess outcome bias, we checked which characteristics the studies reported. According to Lantz (17), a study should at least report accuracy, specificity and sensitivity for an unbiased report of a machine learning model. Therefore, these aspects were taken into consideration during the selection process. Since it could be assumed that machine learning is a new field in outpatient psychodynamic psychotherapy research (19), our primary goal was to gain an overview of research conducted in this area. Therefore, a full assessment of report quality using the TRIPOD (20) criteria would have been beyond the scope of this work.

Results

Study selection

The initial search of both databases yielded 1,358 results, the second 6,206. In total, 3,216 were duplicate records, which were removed before screening. Of the 4,348 records screened, 4,289 were excluded, as they did not meet the eligibility criteria. Most excluded records were related to brain research and prediction of recovery processes after surgery. The second largest group of excluded records were associated with treating patients with other approaches than outpatient psychodynamic psychotherapy. In total, 59 records appeared to have met our eligibility criteria. Two records were inaccessible via any platform. Another 27 reports were excluded because outpatient psychodynamic psychotherapy was not a form of treatment. Nine of the 27 studies were conducted by a research group under Professor Atkins, who successfully created an automatic transcription and evaluation tool for short term psychotherapy, namely motivational interviewing (21–29). Ten of the 27 excluded studies either tried to predict the outcome of cognitive behavior therapy by using natural language processing, or tried to predict optimal therapeutic interventions with sociodemographic data (30–40). Another 16 studies excluded were reviews about machine learning and its utilization within psychotherapy, psychopharmacotherapy, and diagnostics (19, 27, 38, 41–51). Another 8 Studies were excluded because they treated patients with psychotherapy, yet did not specify which kind of treatment approach they used (22, 29, 52–57). Lastly, two studies retrieved were theoretical studies (58, 59). To summarize, only 4 out of 4,348 records screened were eligible for our review.

Study characteristics and results

An overview of the four studies that were deemed eligible, as well as the utilized machine learning methods and their bias assessment, can be seen in Tables 1, 2. Two of the four studies were single-dyad studies. Villmann et al. (63) examined the possible use of artificial neural networks to investigate psycho-physiological parameters derived during the therapy sessions. The authors measured five physiological parameters across 37 therapy sessions for both the patient and therapist and transcribed all sessions. The machine learning model applied was a growing-self-organizing map to combine the psychophysiological data into emotional entropy. Emotional entropy can be understood as emotional variability or emotional energy. The session transcripts were processed with the Mergenthaler Cycle Model (64), which groups the words in the transcripts into four topics: relaxing, experiencing, reflecting and connecting. The authors then compared the emotional entropy with the Mergenthaler Cycle topics. In doing so, they found a cyclic process (64). The patient experiences an interpersonal conflict which increases emotional entropy. The conflict is then reflected upon, and the patient connects the interpersonal conflict with an inner conflict. This connection unleashes emotional energy, which enables a structural change within the patient. Afterwards, a period of relaxation and stabilization follows. Villmann et al. (63) described their proof-of-concept study as an initial first step, which should be verified in future studies.

TABLE 1

Table 1. Study characteristics.

TABLE 2

Table 2. Utilized machine learning models.

The second single-dyad study was done by Laskoski et al. (62). They used a random-forest model to predict patient distress based on coded interventions from a videotaped psychoanalysis, consisting of 120 sessions. Trained judges rated the psychotherapist's interventions with the Psychotherapy Process Q-Set (65). The patient answered the Outcome Questionnaire after each session (66). The random-forest model had an AUC of 0.725, sensitivity of 79%, specificity of 79%, and accuracy of 70.5 % in predicting patient stress after therapy sessions. Additionally, the authors calculated the variable importance of the predictors and found standard techniques of psychodynamic therapy, e.g., drawing the patient's attention to unconscious content or linking the patient's feelings to past situations, to be the most important factors in reducing patient distress.

The third study was conducted by Atzil-Slonim et al. (60). They used Latent Dirichlet allocation to extract various topics from session transcripts. Then, a sparse multinomial logistic regression was used to predict the social functioning and symptom distress after each therapy session based on the topics discussed. Social functioning and symptom distress of the patient were measured with the Outcome Rating Scale and Symptom-Checklist (67, 68). In total, they analyzed 873 therapy sessions deriving from 58 patients and 52 therapists. Results showed that an increase in positive topics was positively correlated with high social functioning and associated with a decrease in symptoms distress. Conversely, an increase of negatively connotated topics correlated with an increase of symptom distress. Accuracy of the final model was at 75.6 % with regard to predicting social functioning.

Halfon et al. (61) tried to predict four basic emotions (joy, anger, sadness and anxiety) of children within a psychodynamic play therapy. Their sample consisted of at least two randomly drawn videotaped sessions per therapy. In total, 148 videotaped sessions of 53 children and 24 psychotherapists were selected. Emotional expressions of children were coded by trained judges using the affect expression items of the Children's Play Therapy Instrument (69). The videos were transcribed separately. Afterwards, the authors trained several supervised machine learning models to predict the affect expressions of children based on the transcript or the video. Overall, a fusion strategy, which combined text analysis and facial recognition to predict affect expressions, achieved the best results. Still, affect expression predictions of the final model correlated on average r = 0.30 with the ratings of trained judges. Halfon et al. (61) concluded that the “automatic affect analysis is promising, however, needs further development.”

Synthesis of results

Our review identified four studies that utilized machine learning within outpatient psychodynamic psychotherapy research. All four studies are proof-of-concept studies. Furthermore, none of the four studies reported their results without bias. All four studies differed in the type of data they used for their machine learning models and their study aims. All studies trained their machine learning models with data from completed outpatient psychodynamic psychotherapies only. Lastly, three of the four studies were done within the last 5 years.

Discussion

Some authors call for psychodynamic psychotherapies to be tailored to patients as much as possible in order to be more successful (13). However, this requires models that allow an accurate prediction of whether the therapy will be successful or unsuccessful. We argue that models developed using the machine learning approach are particularly well suited for this purpose. Within machine learning, models are evaluated primarily for their ability to perform a predetermined task on new data (14, 15). However, since it was unknown how widely machine learning is represented in outpatient psychodynamic psychotherapy research, we conducted a review. Our systematic review identified four proof-of-concept studies that utilized machine learning. Three studies were published between 2019 and 2021 and two studies had a single-dyad sample. All four studies utilized machine learning to evaluate completed outpatient psychodynamic psychotherapies. It seems that machine learning has only recently entered outpatient psychodynamic psychotherapy research. However, this could also mean that researchers are not yet aware of what machine learning can be used for in psychodynamic psychotherapy research. We therefore want to present perspectives and ideas that can be used for future psychodynamic psychotherapy studies. We would then like to highlight a possible risk that might occur when using machine learning in psychotherapy.

To make psychotherapies more successful, Leichsenring et al. (13) suggest that therapy should be tailored to the needs of patients with high non-response and drop-out probability. A possible implementation of this idea would be to predict therapy success or drop-out at the beginning of the therapy and to include these predictions in the therapy planning. Some studies within cognitive behavioral therapy research attempted to implement this (42, 53, 70, 71). A commonality of these studies is that socio-demographic data was collected before the start of psychotherapy and used to predict psychotherapy success with the help of machine learning models. Psychotherapy success was operationalised differently, either as symptom improvement, drop-out or improvement in quality of life.

Another implementation of the idea of tailoring therapy to patients' needs could include feedback to the therapists. De Jong et al. (72) were able to show that feedback to the therapists reduces the drop-out probability of patients by 20% and leads to stronger symptom improvement. Machine learning can be used in this context to develop models that automatically evaluate audio transcripts of psychotherapy sessions and provide feedback to the therapist. The therapist would then be able to get timely feedback about possible pathological developments and could intervene accordingly. A research group led by Professor Atkins is currently attempting to implement this idea (21, 22, 25, 26, 28). Currently, they are only successful in doing this for the very standardized Motivational Interviewing (21). However, it seems possible to build on the work of this group and provide therapists with feedback on variables that are relevant to psychodynamic therapy, such as the agency, working alliance, and the patient's structural integration of personality (7, 8, 11).

A third way to tailor therapy to patients' needs is to predict the fit between therapist and patient. Delgadillo et al. (33) found that there are differences between therapists in the effectiveness with which they treat individual patient groups. Their final machine learning model identified 17 classes of patient-to-therapist matches, which vary greatly in their effectiveness. Building on this idea of Delgadillo et al. (33), it would be conceivable to develop a machine learning models that can predict which therapist has the highest probability of achieving a successful therapy with a patient.

On the other hand, the use of machine learning in psychotherapy research should be carefully considered. As the previous ideas illustrate, models developed with machine learning have the ability to automate many processes, such as feedback to and allocation of patients to therapists. This poses the risk that such models could become an unreflective and potentially discriminatory standard (26, 73). In other words, minorities and vulnerable groups are disadvantaged, for instance, by being denied psychotherapeutic treatment because the model predicts that treatment will be unsuccessful. In this context, Hirsch et al. (26) examined how well their model, which gave feedback on Motivational Interviewing, was accepted by therapists. They found that novices in particular tended to accept the feedback without reflection. Therefore, Besse et al. (73) warned that this can also systematically create discrimination, especially if the model was not developed on the basis of theoretical considerations and representative data (73, 74). The use of machine learning in psychotherapy research should therefore be embedded in existing theories.

Limitations

Several limitations of the presented work must be mentioned. Some studies which utilized machine learning as a method within outpatient psychodynamic psychotherapy research may not have been considered in our review, as we had rather strict criteria for inclusion. Although Zilcha-Mano et al. (56) treated some patients with psychodynamic psychotherapy, it was excluded in our review, as they did not differentiate their results between treatment approaches. Furthermore, we only included studies which explicitly mentioned, in their abstracts, that they used machine learning, deep learning or a form of artificial intelligence. It is conceivable that articles referring to their methodology with the name of statistical model, instead of machine learning, were not included. Furthermore, articles that did not mention their methodology within the abstract, although they used machine learning, may also have been disregarded. As we only found four studies within outpatient psychodynamic psychotherapy research that used machine learning, our review summarizes the first attempts at adopting machine learning into this field of research. Therefore, various limitations mentioned by Aafjes-van Doorn et al. (19) also apply to this review. As these are among the first studies in this area, they should be interpreted cautiously, namely as proof-of-concepts studies, so that the significance of their results is not overestimated. Therefore, we also refrained from assessing the quality of the studies using TRIPOD criteria (20). Thus, we recommend that future reviews in this field use the TRIPOD criteria (20) to assess the quality of studies.

Conclusion

Although much research has been done on psychodynamic psychotherapy, the treatment success of this therapy method has not improved. We argue that machine learning is a way to develop models that detect non-responders and patients with high drop-out probability early and enable intervention. However, since it was unknown how widespread machine learning is in outpatient psychodynamic psychotherapy research, we felt it necessary to conduct a review of current research. We found four studies, three of which were carried out between 2019 and 2021. Thus, machine learning seems to have entered this field of research only recently and researchers might not yet be aware of its possible uses. We have therefore outlined some possibilities, ideas, and perspectives on how machine learning can be used to improve the success of psychodynamic psychotherapies. Thus, we hope to give new impetus to outpatient psychodynamic psychotherapy research on how to use machine learning to address previously unsolved problems.

Author contributions

IR planned the study, conducted the literature search, and wrote the manuscript. NG was responsible with IR for selecting the studies to include into the review. SS-T used her expertise to correct the parts about machine learning in the manuscript. JS was an advisor about machine learning. MS was substantially aided in improving the accuracy of the results and discussion. H-CF and CN were the supervisors for this study. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Steinert C, Munder T, Rabung S, Hoyer J, Leichsenring F. Psychodynamic therapy: as efficacious as other empirically supported treatments? A meta-analysis testing equivalence of outcomes. Am J Psychiatry. (2017) 174:943–53. doi: 10.1176/appi.ajp.2017.17010057

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Ehrenthal JC, Dinger U, Nikendei C. Aktuelle entwicklungen der psychodynamischen psychotherapieforschung. Psychotherapeut. (2014) 59:212–8. doi: 10.1007/s00278-014-1045-5

CrossRef Full Text | Google Scholar

3. Leichsenring F, Leweke F, Klein S, Steinert C. The empirical status of psychodynamic psychotherapy—an update: Bambi's alive and kicking. Psychother Psychosom. (2015) 84:129–48. doi: 10.1159/000376584

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Maljanen T, Knekt P, Lindfors O, Virtala E, Tillman P, Harkanen T, et al. The cost-effectiveness of short-term and long-term psychotherapy in the treatment of depressive and anxiety disorders during a 5-year follow-up. J Affect Disord. (2016) 190:254–63. doi: 10.1016/j.jad.2015.09.065

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Yonatan-Leus R, Strauss AY, Cooper-Kazaz R. Psychodynamic psychotherapy is associated with sustained reduction in health care utilization and cost. Clin Psychol Psychother. (2021) 28:642–55. doi: 10.1002/cpp.2527

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Rosso G, Aragno E, Cuomo A, Fagiolini A, Di Salvo G, Maina G. Five-year follow-up of first-episode depression treated with psychodynamic psychotherapy or antidepressants. Psychiatry Res. (2019) 275:27–30. doi: 10.1016/j.psychres.2019.02.073

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Huber J, Jennissen S, Nikendei C, Schauenburg H, Dinger U. Agency and alliance as change factors in psychotherapy. J Consult Clin Psychol. (2021) 89:214–26. doi: 10.1037/ccp0000628

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Volz M, Jennissen S, Schauenburg H, Nikendei C, Ehrenthal JC, Dinger U. Intraindividual dynamics between alliance and symptom severity in long-term psychotherapy: why time matters. J Couns Psychol. (2021) 68:446. doi: 10.1037/cou0000545

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Jennissen S, Connolly Gibbons MB, Crits-Christoph P, Schauenburg H, Dinger U. Insight as a mechanism of change in dynamic therapy for major depressive disorder. J Couns Psychol. (2021) 68:435. doi: 10.1037/cou0000554

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Jennissen S, Huber J, Ehrenthal JC, Schauenburg H, Dinger U. Association between insight and outcome of psychotherapy: systematic review and meta-analysis. Am J Psychiatry. (2018) 175:961–9. doi: 10.1176/appi.ajp.2018.17080847

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Cierpka M. Operationalisierte Psychodynamische Diagnostik OPD-2: das Manual für Diagnostik und Therapieplanung. Berlin: Huber (2006).

Google Scholar

12. Halstensen K, Gjestad R, Luyten P, Wampold B, Granqvist P, Stalsett G, et al. Depression and mentalizing: a psychodynamic therapy process study. J Couns Psychol. (2021) 68:705–18. doi: 10.1037/cou0000544

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Leichsenring F, Steinert C, Ioannidis JPA. Toward a paradigm shift in treatment and research of mental disorders. Psychol Med. (2019) 49:2111–7. doi: 10.1017/S0033291719002265

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Bi Q, Goodman KE, Kaminsky J, Lessler J. What is machine learning? A primer for the epidemiologist. Am J Epidemiol. (2019) 188:2222–39. doi: 10.1093/aje/kwz189

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Dwyer DB, Falkai P, Koutsouleris N. Machine learning approaches for clinical psychology and psychiatry. Annu Rev Clin Psychol. (2018) 14:91–118. doi: 10.1146/annurev-clinpsy-032816-045037

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Döring N, Bortz J. Forschungsmethoden und Evaluation in den Sozial- und Humanwissenschaften. Berlin, Heidelberg: Springer (2016). doi: 10.1007/978-3-642-41089-5

CrossRef Full Text | Google Scholar

17. Lantz B. Machine Learning with R: Expert Techniques for Predictive Modeling. Birmingham: Packt publishing ltd. (2019).

Google Scholar

18. Barber D. Bayesian Reasoning and Machine Learning. Cambridge: Cambridge Univ Press. (2015).

Google Scholar

19. Aafjes-van Doorn K, Kamsteeg C, Bate J, Aafjes M. A scoping review of machine learning in psychotherapy research. Psychother Res. (2021) 31:92–116. doi: 10.1080/10503307.2020.1808729

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Bmj. (2015) 350:g7594. doi: 10.1136/bmj.g7594

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Flemotomos N, Martinez VR, Chen Z, Singla K, Ardulov V, Peri R, et al. Automated evaluation of psychotherapy skills using speech and language technologies. Behav Res Methods. (2021) 54:690–711. doi: 10.3758/s13428-021-01623-4

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Gaut G, Steyvers M, Imel ZE, Atkins DC, Smyth P. Content coding of psychotherapy transcripts using labeled topic models. IEEE J Biomed Health Inform. (2017) 21:476–87. doi: 10.1109/JBHI.2015.2503985

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Goldberg SB, Flemotomos N, Martinez VR, Tanana MJ, Kuo PB, Pace BT, et al. Machine learning and natural language processing in psychotherapy research: alliance as example use case. J Couns Psychol. (2020) 67:438–48. doi: 10.1037/cou0000382

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Goldberg SB, Tanana M, Imel ZE, Atkins DC, Hill CE, Anderson T. Can a computer detect interpersonal skills? Using machine learning to scale up the facilitative interpersonal skills task. Psychotherapy Res. (2020) 31:281–88. doi: 10.1080/10503307.2020.1741047

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Hirsch T, Merced K, Narayanan S, Imel ZE, Atkins DC. Designing contestability: interaction design, machine learning, and mental health. DIS. (2017) 2017:95–9. doi: 10.1145/3064663.3064703

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Hirsch T, Soma C, Merced K, Kuo P, Dembe A, Caperton DD, et al. “It's hard to argue with a computer:” investigating psychotherapists' attitudes towards automated evaluation. DIS. (2018) 2018:559–71. doi: 10.1145/3196709.3196776

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Imel ZE, Caperton DD, Tanana M, Atkins DC. Technology-enhanced human interaction in psychotherapy. J Couns Psychol. (2017) 64:385–93. doi: 10.1037/cou0000213

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Imel ZE, Pace BT, Soma CS, Tanana M, Hirsch T, Gibson J, et al. Design feasibility of an automated, machine-learning based feedback system for motivational interviewing. Psychotherapy. (2019) 56:318–28. doi: 10.1037/pst0000221

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Tanana MJ, Soma CS, Kuo PB, Bertagnolli NM, Dembe A, Pace BT, et al. How do you feel? Using natural language processing to automatically rate emotion in psychotherapy. Behav Res Methods. (2021) 1–14. doi: 10.3758/s13428-020-01531-z

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Aalbers G, Engels T, Haslbeck JMB, Borsboom D, Arntz A. The network structure of schema modes. Clin Psychol Psychother. (2021) 28:1065–78. doi: 10.1002/cpp.2577

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Bohannon J. The synthetic therapist: Some people prefer to bare their souls to computers rather than to fellow humans. Science. (2015) 349:250–1. doi: 10.1126/science.349.6245.250

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Delgadillo J, Gonzalez Salas Duhne P. Targeted prescription of cognitive–behavioral therapy versus person-centered counseling for depression using a machine learning approach. J Consult Clin Psychol. (2020) 88:14–24. doi: 10.1037/ccp0000476

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Delgadillo J, Rubel J, Barkham M. Towards personalized allocation of patients to therapists. J Consult Clin Psychol. (2020) 88:799–808. doi: 10.1037/ccp0000507

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Ewbank MP, Cummins R, Tablan V, Bateup S, Catarino A, Martin AJ, et al. Quantifying the Association Between Psychotherapy Content and Clinical Outcomes Using Deep Learning. JAMA Psychiatry. (2020) 77:35–43. doi: 10.1001/jamapsychiatry.2019.2664

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Ewbank MP, Cummins R, Tablan V, Catarino A, Buchholz S, Blackwell AD, et al. Understanding the relationship between patient language and outcomes in internet-enabled cognitive behavioural therapy: A deep learning approach to automatic coding of session transcripts. Psychother Res. (2020) 31:300–12. doi: 10.1080/10503307.2020.1788740

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Gómez Penedo JM, Schwartz B, Giesemann J, Rubel JA, Deisenhofer A-K, Lutz W, et al. For whom should psychotherapy focus on problem coping? A machine learning algorithm for treatment personalization. Psychotherapy Res. (2021) 32:151–64. doi: 10.1080/10503307.2021.1930242

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Hilbert K, Jacobi T, Kunas SL, Elsner B, Reuter B, Lueken U, et al. Identifying cbt non-response among ocd outpatients: a machine-learning approach. Psychotherapy Res. (2020) 31:52–62. doi: 10.1080/10503307.2020.1839140

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Hilbert K, Lueken U. Prädiktive analytik aus der perspektive der klinischen psychologie und psychotherapie = Predictive analytics from a mental health perspective. Verhaltenstherapie. (2020) 30:8–17. doi: 10.1159/000505302

CrossRef Full Text | Google Scholar

39. Probst T, Kleinstäuber M, Lambert MJ, Tritt K, Pieh C, Loew TH, et al. Why are some cases not on track? An item analysis of the assessment for signal cases during inpatient psychotherapy. Clin Psychol Psychotherapy. (2020) 27:559–66. doi: 10.1002/cpp.2441

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Yao L, Zhao X, Xu Z, Chen Y, Liu L, Feng Q, et al. Influencing factors and machine learning-based prediction of side effects in psychotherapy. Front Psychiatry. (2020) 11:537442. doi: 10.3389/fpsyt.2020.537442

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Andersson G, Titov N, Dear BF, Rozental A, Carlbring P. Internet-delivered psychological treatments: from innovation to implementation. World Psychiatry. (2019) 18:20–8. doi: 10.1002/wps.20610

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Chekroud AM, Bondar J, Delgadillo J, Doherty G, Wasil A, Fokkema M, et al. The promise of machine learning in predicting treatment outcomes in psychiatry. World Psychiatry. (2021) 20:154–70. doi: 10.1002/wps.20882

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Durstewitz D, Koppe G, Meyer-Lindenberg A. Deep neural networks in psychiatry. Mol Psychiatry. (2019) 24:1583–98. doi: 10.1038/s41380-019-0365-9

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Geoghegan L, Scarborough A, Wormald JCR, Harrison CJ, Collins D, Gardiner M, et al. Automated conversational agents for post-intervention follow-up: a systematic review. BJS Open. (2021) 5:zrab070. doi: 10.1093/bjsopen/zrab070

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Horn RL, Weisz JR. Can artificial intelligence improve psychotherapy research and practice? Administrat Policy Mental Health Mental Health Services Res. (2020) 47:852–5. doi: 10.1007/s10488-020-01056-9

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Huys QJM, Browning M, Paulus MP, Frank MJ. Advances in the computational understanding of mental illness. Neuropsychopharmacology. (2021) 46:3–19. doi: 10.1038/s41386-020-0746-4

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Lee Y, Ragguett R-M, Mansur RB, Boutilier JJ, Rosenblat JD, Trevizol A, et al. Applications of machine learning algorithms to predict therapeutic outcomes in depression: a meta-analysis and systematic review. J Affect Disord. (2018) 241:519–32. doi: 10.1016/.jad.2018.08.073

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Lee Y, Ragguett R-M, Mansur RB, Boutilier JJ, Rosenblat JD, Trevizol A, et al. 'Applications of machine learning algorithms to predict therapeutic outcomes in depression: A meta-analysis and systematic review': corrigendum. J Affect Disord. (2020) 274:1211–5. doi: 10.1016/j.jad.2020.02.037

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Tahan M. Artificial Intelligence applications and psychology: an overview. Neuropsychopharmacol Hung. (2019) 21:119–26. doi: 10.32598/ajnpp.4.3.210

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Tracey TJG. The scientific future of counseling psychology: five specific areas of predictions. J Couns Psychol. (2017) 64:347–8. doi: 10.1037/cou0000234

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Zale A, Lasecke M, Baeza-Hernandez K, Testerman A, Aghakhani S, Munoz RF, et al. Technology and psychotherapeutic interventions: bibliometric analysis of the past four decades. Internet Interv. (2021) 25:100425. doi: 10.1016/j.invent.2021.100425

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Bavaresco R, Barbosa J, Vianna H, Buttenbender P, Dias L. Design and evaluation of a context-aware model based on psychophysiology. Comput Methods Programs Biomed. (2020) 189:105299. doi: 10.1016/j.cmpb.2019.105299

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Bone C, Simmonds-Buckley M, Thwaites R, Sandford D, Merzhvynska M, Rubel J, et al. Dynamic prediction of psychological treatment outcomes: development and validation of a prediction model using routinely collected symptom data. Lancet Digit Health. (2021) 3:e231–e40. doi: 10.1016/S2589-7500(21)00018-2

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Bruijniks SJE, DeRubeis RJ, Lemmens LHJM, Peeters FPML, Cuijpers P, Huibers MJH. The relation between therapy quality, therapy processes and outcomes and identifying for whom therapy quality matters in CBT and IPT for depression. Behav Res Ther. (2021) 139:103815. doi: 10.1016/j.brat.2021.103815

PubMed Abstract | CrossRef Full Text | Google Scholar

55. de Mello FL, de Souza SA. Psychotherapy and artificial intelligence: a proposal for alignment. Front Psychol. (2019) 10:263. doi: 10.3389/fpsyg.2019.00263

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Zilcha-Mano S, Errazuriz P, Yaffe-Herbst L, German RE, DeRubeis RJ. Are there any robust predictors of “sudden gainers,” and how is sustained improvement in treatment outcome achieved following a gain? J Consult Clin Psychol. (2019) 87:491–500. doi: 10.1037/ccp0000401

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Ziobrowski HN, Cui R, Ross EL, Liu H, Puac-Polanco V, Turner B, et al. Development of a model to predict psychotherapy response for depression among veterans. Psychol Med. (2022) 1–10. doi: 10.1017/S0033291722000228

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Kinley JL, Reyno SM. Project for a scientific psychiatry: A neurobiologically informed, phasic, brain-based model of integrated psychotherapy. J Psychother Integr. (2016) 26:61–73. doi: 10.1037/a0039636

CrossRef Full Text | Google Scholar

59. Caspar F, Rothenfluh T, Segal Z. The appeal of connectionism for clinical psychology. Clin Psychol Rev. (1992) 12:719–62. doi: 10.1016/0272-7358(92)90022-Z

CrossRef Full Text | Google Scholar

60. Atzil-Slonim D, Juravski D, Bar-Kalifa E, Gilboa-Schechtman E, Tuval-Mashiach R, Shapira N, et al. Using topic models to identify clients' functioning levels and alliance ruptures in psychotherapy. Psychotherapy. (2021) 58:324–39. doi: 10.1037/pst0000362

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Halfon S, Doyran M, Türkmen B, Oktay EA, Salah AA. Multimodal affect analysis of psychodynamic play therapy. Psychother Res. (2020) 31:313–28. doi: 10.1080/10503307.2020.1839141

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Laskoski PB, Serralta FB, Passos IC, Hauck S. Machine-learning approaches in psychotherapy: a promising tool for advancing the understanding of the psychotherapeutic process. Braz J Psychiatry. (2019) 41:568–9. doi: 10.1590/1516-4446-2018-0295

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Villmann T, Liebers C, Bergmann B, Gumz A, Geyer M. Investigation of psycho-physiological interactions between patient and therapist during a psychodynamic therapy and their relation to speech using in terms of entropy analysis using a neural network approach. New Ideas Psychol. (2008) 26:309–25. doi: 10.1016/j.newideapsych.2007.07.010

CrossRef Full Text | Google Scholar

64. Mergenthaler E. Emotion-abstraction patterns in verbatim protocols: a new way of describing psychotherapeutic processes. J Consult Clin Psychol. (1996) 64:1306–15. doi: 10.1037/0022-006X.64.6.1306

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Jones E. Manual for the Psychotherapy Process Q-set. Unpublished Manuscript. Berkeley: University of California. (1985).

66. Lambert MJ, Burlingame GM, Umphress V, Hansen NB, Vermeersch DA, Clouse GC, et al. The reliability and validity of the outcome questionnaire. Clin Psychol Psychother. (1996) 3:249–58.

Google Scholar

67. Franke GH. Symptom-Checklist-90^®-Standard: SCL-90^®-S. Göttingen [ua]: Hogrefe. (2014).

Google Scholar

68. Miller SD, Duncan B, Brown J, Sparks J, Claud D. The outcome rating scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog measure. J Brief Therapy. (2003) 2:91–100.

Google Scholar

69. Kernberg PF, Chazan SE, Normandin L. The children's play therapy instrument (CPTI). Description, development, and reliability studies. J Psychother Pract Res. (1998) 7:196–207. doi: 10.1037/t72860-000

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Koppe G, Meyer-Lindenberg A, Durstewitz D. Deep learning for small and big data in psychiatry. Neuropsychopharmacology. (2021) 46:176–90. doi: 10.1038/s41386-020-0767-z

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Poster K, Bennemann B, Hofmann SG, Lutz W. Therapist interventions and skills as predictors of dropout in outpatient psychotherapy. Behavior Therapy. (2021) 52:1489–1501. doi: 10.1016/j.beth.2021.05.001

PubMed Abstract | CrossRef Full Text | Google Scholar

72. de Jong K, Conijn JM, Gallagher RAV, Reshetnikova AS, Heij M, Lutz MC. Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: a multilevel meta-analysis. Clin Psychol Rev. (2021) 85:102002. doi: 10.1016/j.cpr.2021.102002

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Besse P, Castets-Renard C, Garivier A, Loubes J-M. Can everyday ai be ethical? Machine learning algorithm fairness (english version). Statistiques et Société. (2018) 6:8.

Google Scholar

74. Elhai JD, Montag C. The compatibility of theoretical frameworks with machine learning analyses in psychological research. Current Opinion Psychol. (2020) 36:83–8. doi: 10.1016/j.copsyc.2020.05.002

PubMed Abstract | CrossRef Full Text | Google Scholar