No pain, no gain revisited: the impact of positive and negative psychotherapy experiences on treatment outcome

Objective: Psychotherapy may have many benefits for patients with mental health problems, but patients may also have negative experiences related to the therapy. Not much is known about these negative psychotherapy experiences and their impact on treatment outcome. The aim of this study was to examine the association between positive and negative psychotherapy experiences and treatment outcome.

Methods: A total of 130 patients participated in the study. They received treatment as usual and were assessed for positive and negative psychotherapy experiences at mid-treatment and post-treatment using the Positive and Negative Experiences of Psychotherapy Questionnaire (PNEP). Treatment outcome was measured by the Outcome Questionnaire - 45 (OQ-45). Multiple linear regression was used to analyze the data.

Results: All patients reported positive psychotherapy experiences at mid-treatment. At least one negative experience was reported by 69% of participants. After correction for baseline severity (i.e., OQ-45 at baseline) and relevant demographic variables, positive psychotherapy experiences at mid-treatment emerged as a predictor for treatment outcome. In contrast, negative psychotherapy experiences did not predict treatment outcome. However, reporting more negative experiences was associated with reporting fewer positive experiences at mid- and end of treatment.

Conclusion: Both positive and negative psychotherapy experiences commonly occur. Although in this study negative psychotherapy experiences did not predict treatment outcome in terms of symptom reduction, the findings do suggest that negative experiences may influence the way in which patients evaluate their treatment. Although positive experiences outweigh negative experiences, patients should be informed that negative experiences may also occur.

Introduction

Over the past few decades, research has evidently shown that psychotherapy is effective in reducing psychological distress and improving quality of life for many patients (Holmes et al., 2018; Wakefield et al., 2020; Cuijpers et al., 2023). However, much less is known about the potential negative effects of psychotherapy. Little attention has been paid to the possibility that not all patients will improve, and some may even deteriorate or have unwanted negative experiences during therapy (Lilienfeld, 2007; Holmes et al., 2018; Rozental et al., 2018).

The extent to which negative effects of psychotherapy exist, and the problems these effects may cause, has been the subject of much debate in the past (Barlow, 2010). Some of the earliest evidence that interventions designed to improve human behavior can also have unintended, harmful consequences, is the Cambridge-Somerville Youth Study (Powers and Witmer, 1951). This longitudinal study was conducted in the 1940s with the aim of reducing juvenile delinquency. As described by McCord (2003), in this study, pre-delinquent boys were randomly assigned to a control condition or a treatment condition in which they received counseling and individualized services. Contrary to the researchers’ expectations, follow-up showed that boys in the treatment group were more likely to have undesirable outcomes, such as involvement in criminal activity, receiving a mental health diagnosis, or even dying at a younger age. In exploring the explanations for these findings, the authors suggested one possible explanation: the summer camps may have provided an opportunity for boys to model and reinforce each other’s problem behaviors. A recent meta-scientific review focusing on potentially harmful therapies (Williams et al., 2021), examining over 70 studies, found evidence that some interventions have the potential to cause harm. Evidence of potential harm was most clearly evident for Scared Straight interventions, which aim to deter future criminal behavior by exposing at-risk youth to the dangers of prison life, and Critical Incident Stress Debriefing.

One of the first systematic analyses of the negative effects of psychotherapy was provided by Bergin (1966), who reviewed seven psychotherapy outcome studies in which no apparent differences were found between treatment and control groups. On closer examination of the data, this author found a wider range of change scores in the treatment groups than in the control groups, with the former showing both greater improvement and greater deterioration on outcome measures after therapy. These findings suggest that psychotherapy not only has the potential to cause some patients to improve in comparison to untreated patients, but also to cause a proportion of patients to deteriorate significantly more. Despite these findings, research in the following decades has continued to focus primarily on the positive effects of psychotherapy, such as symptom reduction and improved well-being. This has provided important evidence for the effectiveness of psychotherapy for many psychiatric disorders, but the potential negative effects of psychotherapy have often been overlooked (Dandachi-Fitz Gerald et al., 2023).

Berk and Parker (2009) argued that costs and benefits are conceptualized differently in pharmacotherapy and psychotherapy. For example, monitoring for negative effects is mandatory in medical research, but not in psychotherapy research (Nutt and Sharpe, 2008). A study by Vaughan et al. (2014) compared 45 randomized controlled trials of pharmacotherapy and psychotherapy (15 pharmacotherapy only trials, 15 psychotherapy only trials, and 15 combined pharmacotherapy and psychotherapy trials) on the frequency of reporting of adverse events, defined as deleterious results directly attributable to a treatment intervention. The results showed that adverse events were 9–20 times more likely to be reported in the pharmacotherapy research papers than in the psychotherapy research papers, suggesting that pharmacotherapy research pays more attention to the possibility of adverse outcomes than psychotherapy research. In line with these findings, another study reviewed 132 randomized controlled trials of psychological interventions in patients for monitoring adverse events, side effects, and deterioration. Only 28 of the trials (21%) included information on adverse events, and of these, most had incomplete descriptions of how these adverse events were defined and monitored (Jonsson et al., 2014). While there has been some advancement, as indicated by a recent systematic review noting that 60% of published preregistered psychotherapy trials explicitly reported harmful events, there is still considerable heterogeneity in the conceptualization, monitoring, and reporting of these events (Klatte et al., 2023). This diversity impedes the accurate assessment of risks and benefits associated with psychotherapy interventions.

Part of the difficulty in investigating the adverse effects of psychotherapy is the lack of consensus on the most appropriate terms and definitions to describe these effects (Parry et al., 2016). One method often used by researchers to identify a potential negative response to psychotherapy is through standard patient outcome measures assessing symptom reduction (Whipple and Lambert, 2011). Recent evidence suggests that a non-trivial minority of patients experience a worsening of symptoms despite receiving psychological treatments that have been validated and are delivered appropriately. It has been estimated that between 5 and 10% of adult patients participating in clinical trials are worse off after treatment than they were before treatment began (Lambert, 2013), with slightly higher rates (7–15%) for patients with substance use disorders (Moos, 2005). However, it is difficult to determine how many of these patients would have deteriorated regardless of treatment (Berk and Parker, 2009). Perhaps some patients are already on a negative trajectory when they enter treatment, and this deteriorating course cannot be stopped. Others may experience unpleasant life events unrelated to treatment that lead to a worsening of symptoms (Lambert, 2013). Therefore, it cannot be assumed that deterioration on outcome measures during treatment means that this outcome was caused by the treatment. Furthermore, worsening of symptoms may not be the only type of negative effect of treatment. Even when treatment is beneficial, patients may still experience negative reactions, such as feeling emotionally overwhelmed, stigmatized, or impaired social functioning (e.g., strained family relationships). Thus, negative treatment effects are not synonymous with non-improvement or deterioration in treatment outcome measures. This is underscored by qualitative research on clients’ perspectives on their experiences in psychotherapy (e.g., Ladmanová et al., 2022; Vybíral et al., 2024). Clients’ psychotherapy experiences encompass a broad spectrum, including both helpful experiences (e.g., gaining a new perspective on themselves, feeling empowered, or feeling accepted by the therapist) and hindering experiences (e.g., feeling emotionally overwhelmed, feeling confused, or lack of guidance from the therapist). Curran et al. (2019) used task analysis, integrating qualitative research findings and client accounts, to develop a model of process factors that can potentially lead to negative or harmful effects in psychotherapy. Their findings emphasize the presence of potentially harmful factors at each stage of therapy that require attention. Both clients and therapists face challenges in addressing and identifying negative events in therapy. Clients frequently find it difficult to address negative experiences (e.g., Burton and Thériault, 2020; Calabro et al., 2024), contributing to the challenge of identifying them. Meanwhile, therapists may struggle to recognize negative events during therapy sessions and their own role in them (Hannan et al., 2005; Hatfield et al., 2010; Werbart et al., 2019; Dandachi-Fitz Gerald et al., 2022). If these negative events are not recognized, discussed, and satisfactorily resolved, they can have lasting negative consequences (e.g., a higher risk of poorer outcomes, less trust in therapy) (e.g., Burton and Thériault, 2020; Calabro et al., 2024), underscoring the importance of actively seeking feedback on clients’ positive and negative experiences as part of a collaborative therapeutic working alliance.

To date, however, there is no generally accepted definition or assessment method for negative effects of psychotherapy, although some proposals for a conceptual framework have been made (Berk and Parker, 2009; Linden, 2013; Parry et al., 2016). Linden (2013) proposed a framework for identifying and classifying adverse events in psychotherapy that follows a stepwise process. Adverse events are defined as all events that occur in parallel with psychotherapy, are patient-related, and have a negative quality. Such adverse events may be related or unrelated to the psychotherapy and include adverse changes in somatic or psychological symptoms, the patient’s sense of well-being, and social or occupational functioning. An example of a side effect (an adverse reaction to correct treatment) is increased anxiety during exposure therapy, the first-line treatment for phobias. The increase in anxiety that occurs when patients are exposed to cues related to the feared situation or sensation is considered a necessary part of the treatment, helping patients to overcome their fears through corrective learning. However, if there were another treatment that could achieve the same positive effect without burdening the patient with increased anxiety, that other treatment would be preferred. By this definition, side effects can be unexpected, but they can also be expected and sometimes even intended and unavoidable effects of treatment.

There are relatively few systematic studies investigating the occurrence and prevalence of negative effects of psychotherapy. To date, systematic research has been hampered by the lack of a clear definition of negative effects and the diversity of terms used, which has led to difficulties in developing adequate instruments to measure negative effects. In a review by Herzog et al. (2019), eight instruments for assessing negative effects of psychotherapy were systematically examined for their theoretical orientation and psychometric properties. The authors concluded that these instruments collectively cover a large number of domains, without reaching a consensus on which domains are the most important ones. They also found that the psychometric properties of a number of these instruments are insufficient. It has been suggested that the desired, positive effects of psychotherapy should also be included in assessment tools in order to minimize negative priming. An instrument with only items about negative experiences could potentially contribute to a nocebo effect due to negative expectations about the occurrence of side effects (Herzog et al., 2019; Moritz et al., 2019; but see Muschalla et al., 2023). Taking into account these findings on negative priming and the lack of a golden standard instrument for measuring negative reactions to psychotherapy, Dandachi-Fitz Gerald et al. (2024) developed a new instrument incorporating several points of improvement: the Positive and Negative Experiences of Psychotherapy Questionnaire (PNEP). This self-report questionnaire is based on the PANEPS (Peth et al., 2018; Moritz et al., 2019), the NEQ (Rozental et al., 2016), a literature review, and an evaluation of both the PANEPS and the NEQ by a panel of experiential experts. The panel emphasized the importance of including not only items concerning negative, but also positive experiences in order to be more representative of their own experiences with psychotherapy (Berghs, 2020). The PNEP consists of two subscales, negative and positive psychotherapy experiences, and a question asking respondents to rate how beneficial the therapy was. The term psychotherapy experience was chosen rather than psychotherapy effect, because the latter implies a causal relationship that is difficult to establish. In a sample of 200 participants, Dandachi-Fitz Gerald et al. (2024) found that 89.5% reported at least one negative treatment experience as measured by the PNEP. This percentage is close to the upper bound of the 22 to 93% range of negative treatment experiences found in various studies (e.g., Schneibel et al., 2017; Moritz et al., 2019; Rozental et al., 2019; Gerke et al., 2020).

Negative psychotherapy experiences clearly occur. However, the relationship between the occurrence of negative psychotherapy experiences and subsequent treatment outcome is still unclear. Negative experiences that occur during psychotherapy may be a transient phenomenon related to therapeutic interventions that are experienced as negative by the patient but are helpful in the long run. From this perspective, a temporary worsening of symptoms may sometimes even be necessary for improvement to occur. Change in psychotherapy is often nonlinear, with clients feeling worse before symptoms begin to decrease (Antichi and Giannini, 2023). Alternatively, such negative psychotherapy experiences may actually prevent patients from benefiting from therapy and lead to deterioration. In a pilot study, Brakemeier et al. (2018) examined the relationship between adverse events and treatment outcome in terms of response, remission, and relapse rates in a sample of 50 patients in a multidisciplinary inpatient setting for treatment-resistant chronically depressed patients. A self-report questionnaire specifically designed for use in inpatient settings was used to measure three categories of adverse events 6 to 12 months after discharge: subjective temporary worsening of depressive symptoms during treatment, interpersonal conflict with treatment staff or other patients, and major negative life changes after discharge. The results showed that only subjective temporary worsening of symptoms during treatment was associated with treatment outcome. That is, patients who reported a temporary worsening of symptoms were less likely to achieve remission. These results tentatively suggest that certain adverse events experienced by patients may have a negative impact on treatment outcome.

The current study aimed to examine the relationship between positive and negative psychotherapy experiences, as measured by the PNEP at mid-treatment, and treatment outcome, as measured by the OQ-45 in a sample of patients with mild to moderate mental health care problems receiving treatment in an ambulant setting. We hypothesized that (1) reporting of more positive experiences at mid-treatment would be associated with a more positive treatment outcome, whereas (2) reporting of more negative experiences would be associated with a less positive treatment outcome. For exploratory purposes, we were also interested in how positive and negative psychotherapy experiences at mid-and end-treatment would relate to OQ-45 at baseline and end-treatment, and to participants’ evaluation of how beneficial their treatment had been. We also explored whether there was a difference in psychotherapy experiences at mid-treatment between the group of patients that had clinically significantly improved on the OQ-45 at end-treatment, and those that did not. Finding a relationship between negative experiences and treatment outcome would suggest the importance of monitoring negative experiences during therapy in order to identify patients at risk of deriving less benefit from therapy and to intervene to alter their potential negative treatment trajectories. To our knowledge, this is the first study to examine the association between positive and negative psychotherapy experiences and treatment outcome.

Methods

Participants

Sample size was calculated a priori using G*Power (Faul et al., 2007). Prior effect size estimates were not available because the current study is the first to examine the relationship between positive and negative psychotherapy experiences and treatment outcome. For a credible range of effect sizes (with Cohen’s $f^2$ ranging from 0.08 to 0.15), power analysis indicated that the study would be powered at 80% (two-tailed, α = 0.05) with approximately 80 participants. Participants were patients aged 18–65 who were referred to an outpatient mental health service in Maastricht, the Netherlands, for treatment of a current DSM-5 disorder. The mental health care system in the Netherlands is divided into a so-called basic and specialized mental health care domain. The basic domain is intended for patients with a mental disorder of low to moderate severity, such as mild to moderate mood or anxiety disorders, or other disorders with only mild impairment in functioning and without suicidality, psychotic symptoms, or substance addiction, who can be helped with relatively short-term treatment with a maximum of 12 sessions. The specialized mental health care domain is intended for the treatment of more complex mental health problems and more severe psychiatric disorders, such as bipolar disorder or severe depression, (complex) posttraumatic stress disorder, and personality disorders, with a greater impairment in several areas of functioning and with no limit on the duration of treatment. For pragmatic purposes, patients referred to the basic mental health care domain were included in the study, as this domain has a high number of patient enrollment and high patient flow due to the short duration of treatment. Exclusion criteria were illiteracy, indication of intellectual disability, substance use disorder, observed psychotic or manic symptoms, and acute suicide risk, which are also the exclusion criteria for the basic mental health care domain in the Netherlands. Non-Dutch speaking patients were also excluded. Ethical approval was obtained from the standing Ethical Review Committee of the Faculty of Psychology and Neuroscience of Maastricht University [ERCPN 240_112_07_2021: v02]. A total of 300 patients were screened for eligibility. After screening, 168 patients were eligible to participate, of whom 130 (77.4%) agreed to participate in the study. Only complete data sets (baseline, mid-treatment, and end-treatment) were used for this analysis, resulting in a sample size of 83 participants. Three participants were subsequently excluded from the analysis due to inattentive responding, resulting in a final sample size of N = 80. See Figure 1 for a detailed flowchart of participants. Of note, initially, participants received only an email reminder to complete the questionnaires. After noticing a fairly high non-response rate and consequently study dropout, the researchers began contacting participants by phone to remind them to complete the questionnaires, which greatly improved the response rate. The sociodemographic characteristics of the sample analyzed (N = 80) are shown in Table 1. The mean age of the participants was 33.1 years (SD = 13.5), ranging from 18 to 69 years. The mean number of treatment sessions was 10 (SD = 3.8). As shown in Table 2, the mean OQ-45 total score at baseline was 72.1, which is within the clinical range, and decreased to 57.1 at the end of treatment. In total, 40 participants (50.0%) showed a clinically significant improvement on the OQ-45 from baseline to end-treatment, while 39 (48.8%) did not show a clinically significant change, and 1 participant (1.2%) significantly deteriorated on the OQ-45. Patients who dropped out of the study did not differ significantly from completers with respect to age [t(125) = 1.54, p = 0.127], gender [X2(1) = 0.22, p = 0.636], and baseline OQ-45 [t(122) = 0.56, p = 0.575].

Figure 1

Figure 1. Flowchart of participant inclusion and exclusion.

Table 1

Table 1. Sociodemographic characteristics of the sample (N = 80).

Table 2

Table 2. Means, standard deviations, and correlations between variables.

Measures

Positive and negative experiences of psychotherapy questionnaire

The Positive and Negative Experiences of Psychotherapy Questionnaire (PNEP; Dandachi-Fitz Gerald et al., 2024) was used to measure the occurrence of positive and negative psychotherapy experiences at mid-and end-treatment assessments.

The PNEP consists of two subscales of items measuring both positive psychotherapy experiences (33 items) and negative psychotherapy experiences (36 items). Each PNEP item first provides respondents with a description of an experience (e.g., “I felt more confident about myself,” or, “I experienced more stress”) and asks them to indicate whether they experienced this. If participants respond affirmatively, they are asked to rate the intensity of the experience (5-point Likert scale: not at all intense, to some extent, reasonably, very intense, and extremely intense). Finally, participants are asked whether they attribute the experience entirely to the treatment they received, partly to the treatment and partly to other circumstances, or entirely to other circumstances. At the end of the PNEP, participants are asked to rate the extent to which they have benefited from their treatment (7-point Likert scale, strongly deteriorated, deteriorated, slightly deteriorated, unchanged, slightly improved, improved and strongly improved). First psychometric findings indicate that the PNEP is a sound instrument with adequate internal reliabilities (positive subscale: α = 0.94; negative subscale: α = 0.90) and quite good test–retest stability (positive subscale: r = 0.93; negative subscale r = 0.78) (Dandachi-Fitz Gerald et al., 2024). In the current study, Cronbach’s alpha values were 0.90 for the positive subscale and 0.81 for the negative subscale. For this study, a positive or negative experience was considered to be a treatment-related experience if (1) the item was answered affirmatively and (2) it was partly or entirely attributed to therapy. Total scores on both subscales were used to measure positive psychotherapy experiences (maximum total score of 33) and negative psychotherapy experiences (maximum total score of 36). In addition, the question asking participants to rate the extent to which they benefited from their treatment was used as a measure of subjective treatment evaluation.

Outcome questionnaire – 45 items

The outcome questionnaire – 45 items (OQ-45) (Lambert et al., 1996, 2004) is a self-report instrument designed for repeated measurement and consisting of 45 items, nine of which are reversed, asking respondents how they have felt in the past week (5-point Likert scale: never, rarely, sometimes, frequently, almost always). Higher scores reflect higher levels of dysfunction. The OQ-45 consists of three subscales designed to assess different domains of patient functioning: Symptomatic Distress, Interpersonal Relations, and Social Role. Internal consistency (Cronbach’s alpha) for the total score of the Dutch version ranges from 0.92 to 0.96 in university, community, patient, and community and patient combined samples. Test–retest reliability for the Dutch version was found to range from 0.79 to 0.82 for the total score (De Jong et al., 2008). In this study, the Cronbach’s alpha value was 0.94. For the current study, the total score was used as a measure of treatment outcome. A reliable change index (RCI) of 14 was used to determine clinically significant change from baseline to end of treatment.

Experimental check on data integrity

Five control items were added at the mid-and end-treatment assessments to check for inattentive responding (e.g., “If I read this item carefully, I will answer ‘Never’ here”) and overreporting of symptoms (e.g., “I experience such terrible headaches, that my feet hurt from it”) at mid-and end-treatment as a check on data integrity. Participants who failed on more than two of these items were excluded from the analysis.

Design and procedure

The study used a longitudinal, prospective, correlational research design to examine the association between positive and negative effects of psychotherapy and treatment outcome. Participants were recruited during routine clinical interview sessions at the outpatient mental health care service. Recruitment occurred after the clinician assessed the eligibility for study participation. Participants were provided with a study brochure containing general information about the study and an informed consent form. They were informed that they could withdraw from the study at any time without any implications for their treatment. Diagnosis, psychotropic medications, and number of treatment sessions were obtained from patient records. Participants reported their age, gender, education level, work status, and relationship status. The PNEP was administered at mid-treatment (i.e., after four sessions) and at post-treatment. The OQ-45 was administered at baseline and post-treatment. Data on treatment duration (i.e., number of sessions) were collected at post-treatment. Supplementary Figure S1 visualizes the study steps.

Transparency and openness

We report how we determined our sample size, all data exclusions, and all measures in the study, and we follow JARS (Appelbaum et al., 2018). All data and research materials are available at https://doi.org/10.34894/QZUNSW. Data were analyzed using IBM SPSS Statistics for Windows, Version 27.0. The study was preregistered on the Open Science Framework prospectively, before data were collected (view only link for peer review¹). There are no previously published works or works that are currently in press or under review stemming from the same dataset as this manuscript.

Data analysis

SPSS was used for all analyses. Data were checked for missing values and violation of assumptions (linearity, normality). Outliers were identified and examined. Study and treatment withdrawal was registered. Completers and non-completers were compared on baseline characteristics (i.e., demographics and OQ-45 at baseline). Descriptive statistics (means, standard deviations, percentages, and frequencies) were calculated to assess the percentages and levels of positive and negative treatment experiences and treatment outcome among the study participants. To test our hypotheses, a series of stepwise linear hierarchical regression analyses were conducted. First, a backward analysis was performed to identify potentially relevant covariates. Demographic variables, number of treatment sessions, treatment-unrelated positive and negative experiences at mid-treatment, and OQ-45 at baseline were entered as predictors of OQ-45 at end-treatment. Sequentially, non-significant predictors were removed. The criterion for removal was based on the significance level of the F value, which was set at 0.05. The remaining set of predictors was then used in a subsequent forward selection strategy. This forward regression analysis was performed in two steps. In the first step, the set of variables identified in the backward analysis was entered. In the next step, the two predictor variables of interest (i.e., total positive and negative psychotherapy experiences at mid-treatment) were added. For the exploratory analyses, we performed correlational analyses among variables, including PNEP positive and negative subscales at mid-and end-treatment, OQ-45 at baseline and end-treatment, and overall treatment evaluation at mid-and end-treatment. Additionally, we utilized independent samples t-test to compare the subgroup that exhibited clinically significant change on the OQ-45 from pre-to post treatment with the subgroup that did not show improvement.

Results

Positive and negative psychotherapy experiences at mid-treatment

All patients reported positive psychotherapy experiences at mid-treatment, ranging from 3 to 33 positive experiences, with a mean score of 18.2 (see Table 2). The most commonly reported experiences were related to the therapeutic relationship [e.g., feeling well informed (98.8%), understood (95.0%), and accepted (98.8%) by the therapist]. Positive experiences related to having more positive feelings, and to personal growth and acceptance were also frequently reported. The frequencies of all positive treatment experiences can be found in Supplementary Table S1.

At least one negative experience was reported by 55 (69%) of the participants. The mean number of negative experiences was 2.8, with a maximum of 13. The most commonly reported negative effects were related to experiencing more symptoms and emotional distress, such as feeling overwhelmed by emotions (35.0%), experiencing more negative thoughts and memories (26.3%), feeling more stress and/or tension (20.0%), and feeling vulnerable or unprotected (17.5%). Stigma-related experiences were also reported, such as feeling ashamed of having received treatment (10.0%), and fearing that others would find out that they had received treatment (7.5%). None of the participants reported experiencing verbal or sexual transgression by the therapist. The frequencies of all negative psychotherapy experiences can be found in Supplementary Table S2.

Correlations between variables

As expected, baseline OQ-45 was significantly correlated with OQ-45 at end-treatment. Positive and negative psychotherapy experiences at mid-treatment were significantly correlated with positive and negative psychotherapy experiences at end-treatment, respectively. Reporting of more positive experiences at mid-treatment was significantly associated with a better outcome on the OQ-45 at the end of treatment. In contrast, negative experiences mid-treatment did not correlate with treatment outcome on the OQ-45. However, reporting more negative experiences was associated with reporting less positive experiences, both at mid-and end-treatment. Also, reporting more negative experiences at mid-treatment was associated with a less favorable overall treatment evaluation both at mid-treatment and end-treatment.

Regression analysis

In a backward analysis, baseline OQ-45 and relationship status were identified as relevant predictors of OQ-45 at end-treatment. A forward stepwise linear regression analysis was then performed (see Table 3). No multicollinearity was found. In the first model, baseline OQ-45 and relationship status were entered, and in the second model, total positive and negative treatment experiences at mid-treatment were entered. Adding positive and negative treatment experiences significantly increased R². The second model was found to be significant [F(4, 75) = 23.56, p < 0.001] and explained 56% of the variance in OQ-45 at end-treatment. Both baseline OQ-45 and relationship status significantly predicted OQ-45 at end-treatment. Being in a relationship at baseline was associated with lower OQ-45 scores at end-treatment. In addition, positive psychotherapy experiences at mid-treatment were a significant predictor of OQ-45 at end-treatment, even after controlling for the effects of baseline OQ-45 and relationship status, with more positive experiences associated with lower OQ-45 scores at end-treatment. The contribution of negative psychotherapy experiences at mid-treatment to the model was not significant.

Table 3

Table 3. Hierarchical regression results for OQ-45 at end-treatment.

Comparison of psychotherapy experiences between improvers and non-improvers

We examined whether there was a difference in psychotherapy experiences at mid-treatment between the group of patients who showed clinically significant improvement on the OQ-45 at the end of treatment compared to baseline and those who did not. Participants with clinically significant improvement reported significantly more positive psychotherapy experiences at mid-treatment (M = 20.6, SD = 7.6) than participants without clinically significant improvement (M = 15.8, SD = 6.5); t(78) = 3.1, p = 0.003, d = 0.69. There was no difference in the amount of negative psychotherapy experiences at mid-treatment, t(78) = −0.9, p = 0.39, d = −0.19. No difference was found between these two groups in the types of experiences most commonly reported.

Discussion

The aim of the present study was to examine the association between positive and negative psychotherapy experiences at mid-treatment and subsequent treatment outcome. First, all participants reported positive psychotherapy experiences at mid-treatment. At least one negative psychotherapy experience was reported by 55 (69%) of the participants at mid-treatment.

With regard to our two main hypotheses, the results can be summarized as follows. First, reporting more positive psychotherapy experiences at mid-treatment was associated with a more positive treatment outcome. Patients who reported more positive experiences at mid-treatment had a more favorable treatment outcome than patients who reported fewer of these positive experiences. The most commonly reported positive experiences were feeling accepted, understood, and supported by the therapist, and feeling that the treatment was well conducted, all of which are related to the therapeutic alliance (i.e., clarity and agreement about treatment goals and tasks, and the formation of a positive emotional bond, Bordin, 1994). A stronger therapeutic alliance has been consistently associated with better treatment outcomes across a range of psychotherapies (Flückiger et al., 2018), and a recent systematic review (Baier et al., 2020) of 37 psychotherapy studies identified the therapeutic alliance as a mediator of change or “treatment driver” that plays an important role in effective psychotherapy. Other commonly reported positive experiences in the present study were related to a better understanding of self, improved ability to deal with problems, and increased self-acceptance. These findings are consistent with previous research on what changes patients value in psychotherapy beyond symptom reduction (Binder et al., 2010; Hill et al., 2013; Timulak and Keogh, 2017), and suggest that such changes are positively associated with better treatment outcomes in terms of symptom improvement at the end of treatment.

Regarding the association between negative psychotherapy experiences and subsequent treatment outcome, our second hypothesis was not supported. That is, reporting more negative psychotherapy experiences did not predict a less favorable outcome after controlling for baseline psychological distress and relationship status. Nor was an inverse association found, which is consistent with previous research (Moritz et al., 2015) and provides further evidence against the “no pain, no gain” view that such negative experiences are necessary for patients to benefit from treatment. Although no association was found between negative experiences reported at mid-treatment and treatment outcome in terms of symptom reduction on the OQ-45, we did find that reporting more negative experiences was associated with reporting fewer positive experiences, and with a less positive overall evaluation of the treatment. These results are consistent with another recent study (Dandachi-Fitz Gerald et al., 2024) and suggest that negative experiences, while having no effect on symptomatic improvement, may be related to the way in which patients evaluate their treatment. These findings are similar to those reported for adverse side effects of pharmacological treatments, which also do not appear to have a direct impact on their mechanisms of action, but can negatively affect patients’ quality of life, treatment adherence (Dibonaventura et al., 2012; Franzoi et al., 2021; Fleming et al., 2022), and treatment satisfaction (Hughes et al., 2017). Although in the present study, negative psychotherapy experiences did not affect treatment outcome in terms of symptom reduction, it is important for patients to be informed about their potential occurrence so that they can make a better decision about whether to enter treatment (Blease et al., 2016). Crawford et al. (2016) conducted a large survey of patients receiving psychological treatment, asking respondents whether they had experienced lasting negative effects from treatment. Interestingly, respondents who said that they were not given enough information about the treatment before it started were more likely to report lasting bad effects. The authors suggest that properly informed patients may have more realistic expectations about the treatment, which may modulate their perception of negative effects. In addition, recent research suggests that monitoring and discussing side effects can improve the therapeutic alliance, which in turn may have a positive effect on treatment outcome (Muschalla et al., 2023).

In our study, the four most commonly reported negative experiences were feeling emotionally overwhelmed, having more negative thoughts and memories, an increase in stress and/or tension, and feeling vulnerable and unprotected. These findings align with the study by Dandachi-Fitz Gerald et al. (2024), who found that these four negative experiences were the most commonly reported in both a patient sample and a professional sample of psychologists who had received psychotherapy as part of their training. In the present study, both positive and negative psychotherapy experiences appeared to be relatively stable over time, with the reporting of more experiences at mid-treatment being associated with the reporting of more experiences at the end of treatment. Further research using follow-up assessments is needed to investigate whether these experiences persist after the end of treatment, or whether they represent a more transient phenomenon. It is important to note that clients’ subjective reporting of negative experiences does not invariably lead to worsened treatment outcome. Some negative experiences may even contribute positively to the therapeutic process. To aid clinicians in distinguishing between negative psychotherapy experiences that facilitate treatment progress and those that impede it, more research is warranted to elucidate the relationship between client-reported experiences and treatment outcomes. For now, considering that clients’ perspectives on their therapy experiences, particularly negative ones, are often overlooked, clinicians may utilize our findings to not only inform clients but also to systematically monitor both positive and negative experiences. By doing so, clinicians can better recognize early signs of client dissatisfaction or discomfort and respond timely.

Strengths and limitations

The main strengths of this study were the use of a naturalistic setting, which allows the findings to be generalized to clinical practice, the use of a longitudinal prospective design, and the novelty of the research question. In addition, both positive and negative psychotherapy experiences were included, which is in accordance with what patients have been shown to find representative of their experience of psychotherapy (Dandachi-Fitz Gerald et al., 2024). Despite these strengths, the present study has several limitations. The PNEP is a self-report instrument and we cannot be sure how accurate people are in identifying positive and negative experiences and attributing them to a specific cause. In addition, diagnoses were not based on structured clinical interviews, such as the SCID-5, but on the therapists’ assessment following a clinical interview with the patient. Because the study used diagnosis only as a descriptive baseline characteristic, this assessment of diagnosis was considered to be acceptable. Another limitation is that patients were recruited from the basic mental health care domain. In general, patients referred to this segment have less severe mental health problems, making it difficult to generalize the present findings to other patient groups with more severe mental health problems. In addition, patients varied in terms of diagnosis and the type of treatment they received [e.g., cognitive behavioral therapy, eye movement desensitization and reprocessing (EMDR), acceptance and commitment therapy (ACT), and schema-focused therapy (SFT) interventions]. Due to the exploratory nature of the present study, this heterogeneity in diagnoses and treatment interventions is less of a concern. A future line of research might be to investigate whether specific positive and negative experiences are associated with different psychotherapies and/or with different diagnostic classifications. Another interesting line of future research might be to investigate the effects of informing both patients and therapists about the occurrence of negative psychotherapy experiences, and of monitoring for these experiences during treatment. A final limitation relates to the relatively high level of participant attrition between baseline and mid-treatment assessment, which increases the risk of attrition bias. Participants who dropped out of the study after baseline may have had more negative psychotherapy experiences or less positive treatment outcomes.

In conclusion, both positive and negative psychotherapy experiences are common and should not be overlooked. Importantly, in order for patients to provide informed consent to treatment, it is crucial that they are informed of the potential costs (i.e., treatment burden), risks, and benefits of that treatment (Blease et al., 2016; Dobler et al., 2018). Although positive experiences outnumber negative experiences, individuals referred for psychotherapy should be informed about the most common negative experiences that may also occur.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://doi.org/10.34894/QZUNSW.

Ethics statement

The studies involving humans were approved by Ethical Review Committee of the Faculty of Psychology and Neuroscience of Maastricht University. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

RV: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Data curation, Formal analysis, Investigation, Project administration, Visualization. NB: Conceptualization, Methodology, Writing – review & editing, Formal analysis, Writing – original draft. BD-F: Methodology, Writing – review & editing, Conceptualization, Resources, Supervision, Writing – original draft.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This research was partially funded by Stichting Wetenschapsbevordering Klinisch psycholoog en Klinisch neuropsycholoog (Grant number: 20-001 WF).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1378456/full#supplementary-material

Footnotes

1. ^https://osf.io/jkvg3/?view_only=55de685342b84154acbab23ad951f9fd

References

Antichi, L., and Giannini, M. (2023). An introduction to change in psychotherapy: moderators, course of change, and change mechanisms. J. Contemp. Psychother. 53, 315–323. doi: 10.1007/s10879-023-09590-x

PubMed Abstract | Crossref Full Text | Google Scholar

Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., and Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: the APA publications and communications board task force report. Am. Psychol. 73, 3–25. doi: 10.1037/amp0000191

PubMed Abstract | Crossref Full Text | Google Scholar

Baier, A. L., Kline, A. C., and Feeny, N. C. (2020). Therapeutic alliance as a mediator of change: a systematic review and evaluation of research. Clin. Psychol. Rev. 82:101921. doi: 10.1016/j.cpr.2020.101921

PubMed Abstract | Crossref Full Text | Google Scholar

Barlow, D. H. (2010). Negative effects from psychological treatments: a perspective. Am. Psychol. 65, 13–20. doi: 10.1037/a0015643

PubMed Abstract | Crossref Full Text | Google Scholar

Berghs, M.H.G.A. (2020). Evaluation of two questionnaires to measure negative effects of psychotherapy. Master’s thesis. Maastricht, the Netherlands: Maastricht University.

Google Scholar

Bergin, A. E. (1966). Some implications of psychotherapy research for therapeutic practice. J. Abnorm. Psychol. 71, 235–246. doi: 10.1037/h0023577

PubMed Abstract | Crossref Full Text | Google Scholar

Berk, M., and Parker, G. (2009). The elephant on the couch: side-effects of psychotherapy. Aust. N. Z. J. Psychiatry 43, 787–794. doi: 10.1080/00048670903107559

PubMed Abstract | Crossref Full Text | Google Scholar

Binder, P. E., Holgersen, H., and Nielsen, G. H. (2010). What is a “good outcome” in psychotherapy? A qualitative exploration of former patients' point of view. Psychother. Res. 20, 285–294. doi: 10.1080/10503300903376338

PubMed Abstract | Crossref Full Text | Google Scholar

Blease, C. R., Lilienfeld, S. O., and Kelley, J. M. (2016). Evidence-based practice and psychological treatments: the imperatives of informed consent. Front. Psychol. 7:1770. doi: 10.3389/fpsyg.2016.01170

Crossref Full Text | Google Scholar

Bordin, E. S. (1994). “Theory and research on the therapeutic working alliance: new directions” in The working alliance: Theory, research, and practice. eds. A. O. Horvath and L. S. Greenberg (Hoboken, NJ: John Wiley & Sons), 13–37.

Google Scholar

Brakemeier, E.-L., Herzog, P., Radtke, M., Schneibel, R., Breger, V., Becker, M., et al. (2018). CBASP als stationäres Behandlungskonzept der therapieresistenten chronischen Depression: Eine Pilotstudie zum Zusammenhang von Nebenwirkungen und Therapieerfolg. Psychother. Psychosom. Med. Psychol. 68, 399–407. doi: 10.1055/a-0629-7802

PubMed Abstract | Crossref Full Text | Google Scholar

Burton, L., and Thériault, A. (2020). Hindering events in psychotherapy: a retrospective account from the client's perspective. Couns. Psychother. Res. 20, 116–127. doi: 10.1002/capr.12268

Crossref Full Text | Google Scholar

Calabro, A., Cassera, L., and Aschieri, F. (2024). Hindering events in psychotherapy with adolescents: a retrospective investigation of patients’ recollection. Couns. Psychother. Res. 2024, 1–12. doi: 10.1002/capr.12740

Crossref Full Text | Google Scholar

Crawford, M. J., Thana, L., Farquharson, L., Palmer, E., Hancock, E., Bassett, P., et al. (2016). Patient experience of negative effects of psychological treatment: results of a national survey. Br. J. Psychiatry 208, 260–265. doi: 10.1192/bjp.bp.114.162628

PubMed Abstract | Crossref Full Text | Google Scholar

Cuijpers, P., Miguel, C., Ciharova, M., Harrer, M., Moir, F., Roskvist, R., et al. (2023). Psychological treatment of adult depression in primary care compared with outpatient mental health care: a meta-analysis. J. Affect. Disord. 339, 660–675. doi: 10.1016/j.jad.2023.07.011

PubMed Abstract | Crossref Full Text | Google Scholar

Curran, J., Parry, G. D., Darling, J., Mason, A. M., and Chambers, E. (2019). How does therapy harm? A model of adverse process using task analysis in the meta-synthesis of service users’ experience. Front. Psychol. 10:347. doi: 10.3389/fpsyg.2019.00347

Crossref Full Text | Google Scholar

Dandachi-Fitz Gerald, B., Houben, S., Broers, N. J., and Merckelbach, H. (2024). The positive and negative experiences of psychotherapy questionnaire (PNEP): First psychometric findings of a new instrument for monitoring clients’ experiences. J. Clin. Psychol. Psychoth. doi: 10.1002/cpp.3014

Crossref Full Text | Google Scholar

Dandachi-Fitz Gerald, B., Meijs, L., Moonen, I. M., and Merckelbach, H. (2022). No self-serving bias in therapists' evaluations of clients' premature treatment termination: an approximate replication of Murdock et al. (2010). Clin. Psychol. Psychother. 29, 972–981. doi: 10.1002/cpp.2677

PubMed Abstract | Crossref Full Text | Google Scholar

Dandachi-Fitz Gerald, B., Otgaar, H., and Merckelbach, H. (2023). “When psychotherapy fails” in Toward a science of clinical psychology: A tribute to the life and works of Scott O Lilienfeld. eds. C. L. Cobb, S. J. Lynn, and W. O’Donohue (Cham, Switzerland: Springer International Publishing), 301–319.

Google Scholar

De Jong, K., Nugter, A., Polak, M., Wagenborg, H., Spinhoven, P., and Heiser, W. (2008). De Nederlandse versie van de Outcome Questionnaire (OQ-45): Een crossculturele validatie. Psychol. Gezondh. 36, 35–45. doi: 10.1007/BF03077465

Crossref Full Text | Google Scholar

Dibonaventura, M., Gabriel, S., Dupclay, L., Gupta, S., and Kim, E. (2012). A patient perspective of the impact of medication side effects on adherence: results of a cross-sectional nationwide survey of patients with schizophrenia. BMC Psychiatry 12:20. doi: 10.1186/1471-244X-12-20

Crossref Full Text | Google Scholar

Dobler, C. C., Harb, N., Maguire, C. A., Armour, C. L., Coleman, C., and Murad, M. H. (2018). Treatment burden should be included in clinical practice guidelines. Br. M. J. 363:k4065. doi: 10.1136/bmj.k4065

Crossref Full Text | Google Scholar

Faul, F., Erdfelder, E., Lang, A.-G., and Buchner, A. (2007). G*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39, 175–191. doi: 10.3758/BF03193146

PubMed Abstract | Crossref Full Text | Google Scholar

Fleming, L., Agnew, S., Peddie, N., Crawford, M., Dixon, D., and Mac Pherson, I. (2022). The impact of medication side effects on adherence and persistence to hormone therapy in breast cancer survivors: a quantitative systematic review. Breast J. 64, 63–84. doi: 10.1016/j.breast.2022.04.010

Crossref Full Text | Google Scholar

Flückiger, C., Del Re, A. C., Wampold, B. E., and Horvath, A. O. (2018). The alliance in adult psychotherapy: a meta-analytic synthesis. Psychother. 55, 316–340. doi: 10.1037/pst0000172

Crossref Full Text | Google Scholar

Franzoi, M. A., Agostinetto, E., Perachine, M., Del Mastro, L., de Azambuja, A., Vaz-Luis, I, et al. (2021). Evidence-based approaches for the management of side-effects of adjuvant endocrine therapy in patients with breast cancer. Lancet Oncol. 22, e303–e313. doi: 10.1016/S1470-2045(20)30666-5

Crossref Full Text | Google Scholar

Gerke, L., Meyrose, A.-K., Ladwig, I., Rief, W., and Nestoriuc, Y. (2020). Frequencies and predictors of negative effects in routine inpatient and outpatient psychotherapy: two observational studies. Front. Psychol. 11, 1–12. doi: 10.3389/fpsyg.2020.02144

Crossref Full Text | Google Scholar

Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., et al. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. J. Clin. Psychol. 61, 155–163. doi: 10.1002/jclp.20108

PubMed Abstract | Crossref Full Text | Google Scholar

Hatfield, D., McCullough, L., Frantz, S. H., and Krieger, K. (2010). Do we know when our clients get worse? An investigation of therapists' ability to detect negative client change. Clin. Psychol. Psychother. 17, 25–32. doi: 10.1002/cpp.656

PubMed Abstract | Crossref Full Text | Google Scholar

Herzog, P., Lauff, S., Rief, W., and Brakemeier, E.-L. (2019). Assessing the unwanted: a systematic review of instruments used to assess negative effects of psychotherapy. Brain Behav. 9:e01447. doi: 10.1002/brb3.1447

PubMed Abstract | Crossref Full Text | Google Scholar

Hill, C. E., Chui, H., and Baumann, E. (2013). Revisiting and reenvisioning the outcome problem in psychotherapy: an argument to include individualized and qualitative measurement. Psychotherapy 50, 68–76. doi: 10.1037/a0030571

PubMed Abstract | Crossref Full Text | Google Scholar

Holmes, E. A., Ghaderi, A., Harmer, C. J., Ramchandani, P. G., Cuijpers, P., Morrison, A. P., et al. (2018). The lancet psychiatry commission on psychological treatments research in tomorrow's science. Lancet Psychiatry 5, 237–286. doi: 10.1016/S2215-0366(17)30513-8

PubMed Abstract | Crossref Full Text | Google Scholar

Hughes, S., Lacasse, J., Rogers Fuller, R., and Spaulding-Givens, J. (2017). Adverse effects and treatment satisfaction among online users of four antidepressants. Psychiatry Res. 255, 78–86. doi: 10.1016/j.psychres.2017.05.021

PubMed Abstract | Crossref Full Text | Google Scholar

Jonsson, U., Alaie, I., Parlin, T., and Arnberg, F. K. (2014). Reporting of harms in randomized controlled trials of psychological interventions for mental and behavioral disorders: a review of current practice. Contemp. Clin. Trials 38, 1–8. doi: 10.1016/j.cct.2014.02.005

PubMed Abstract | Crossref Full Text | Google Scholar

Klatte, R., Strauss, B., Flückiger, C., and Rosendahl, J. (2023). Adverse events in psychotherapy randomized controlled trials: a systematic review. Psychother. Res. 13, 1–16. doi: 10.1080/10503307.2023.2286992

PubMed Abstract | Crossref Full Text | Google Scholar

Ladmanová, M., Rihacek, T., and Timulak, L. (2022). Client-identified impacts of helpful and hindering events in psychotherapy: a qualitative meta-analysis. Psychother. Res. 32, 723–735. doi: 10.1080/10503307.2021.2003885

PubMed Abstract | Crossref Full Text | Google Scholar

Lambert, M. J. (2013). Outcome in psychotherapy: the past and important advances. Psychotherapy 50, 42–51. doi: 10.1037/a0030682

PubMed Abstract | Crossref Full Text | Google Scholar

Lambert, M. J., Burlingame, G. M., Umphress, V., Hansen, N. B., Vermeersch, D. A., Clouse, G. C., et al. (1996). The reliability and validity of the outcome questionnaire. Clin. Psychol. Psychother. 3, 249–258. doi: 10.1002/(SICI)1099-0879(199612)3:4<249::AID-CPP106>3.0.CO;2-S

PubMed Abstract | Crossref Full Text | Google Scholar

Lambert, M. J., Morton, J. J., Hatfield, D. R., Harmon, C., Hamilton, S., Shimokawa, K., et al. (2004). Administration and scoring manual for the outcome questionnaire (OQ 45.2). 3rd Edn. Wilmington, DE: American Professional Credentialling Services LLC.

Google Scholar

Lilienfeld, S. O. (2007). Psychological treatments that cause harm. Perspect. Psychol. Sci. 2, 53–70. doi: 10.1111/j.1745-6916.2007.00029.x

PubMed Abstract | Crossref Full Text | Google Scholar

Linden, M. J. (2013). How to define, find and classify side effects in psychotherapy: from unwanted events to adverse treatment reactions. Clin. Psychol. Psychother. 20, 286–296. doi: 10.1002/cpp.1765

PubMed Abstract | Crossref Full Text | Google Scholar

McCord, J. (2003). Cures that harm: unanticipated outcomes of crime prevention programs. Ann. Am. Ac. Pol. Soc. Sci. 587, 16–30. doi: 10.1177/0002716202250781

Crossref Full Text | Google Scholar

Moos, R. H. (2005). Iatrogenic effects of psychosocial interventions for substance use disorders: prevalence, predictors, prevention. Addiction 100, 595–604. doi: 10.1111/j.1360-0443.2005.01073.x

PubMed Abstract | Crossref Full Text | Google Scholar

Moritz, S., Fieker, M., Hottenrott, B., Seeralan, T., Cludius, B., Kolbeck, K., et al. (2015). No pain, no gain? Adverse effects of psychotherapy in obsessive-compulsive disorder and its relationship to treatment gains. J. Obsessive Compuls. Relat. Disord. 5, 61–66. doi: 10.1016/j.jocrd.2015.02.002

Crossref Full Text | Google Scholar

Moritz, S., Nestoriuc, Y., Rief, W., Klein, J. P., Jelinek, L., and Peth, J. (2019). It can’t hurt, right? Adverse effects of psychotherapy in patients with depression. Eur. Arch. Psychiatry Clin. Neurosci. 269, 577–586. doi: 10.1007/s00406-018-0931-1

Crossref Full Text | Google Scholar

Muschalla, B., Müller, J., Grocholewski, A., and Linden, A. (2023). Effects of talking about side effects versus not talking about side effects on the therapeutic alliance: a controlled clinical trial. Acta Psychiatr. Scand. 148, 208–216. doi: 10.1111/acps.13543

PubMed Abstract | Crossref Full Text | Google Scholar

Nutt, D. J., and Sharpe, M. (2008). Uncritical positive regard? Issues in the efficacy and safety of psychotherapy. J. Psychopharmacol. 22, 3–6. doi: 10.1177/0269881107086283

PubMed Abstract | Crossref Full Text | Google Scholar

Parry, G. D., Crawford, M. J., and Duggan, C. (2016). Iatrogenic harm from psychological therapies - time to move on. Br. J. Psychiatry 208, 210–212. doi: 10.1192/bjp.bp.115.163618

PubMed Abstract | Crossref Full Text | Google Scholar

Peth, J., Jelinek, L., Nestoriuc, Y., and Moritz, S. (2018). Unerwünschte Effekte von Psychotherapie bei depressiven Patienten-Erste Anwendung der Positive and Negative Effects of Psychotherapy Scale (PANEPS). Psychother. Psychosom. Med. Psychol. 68, 391–398. doi: 10.1055/s-0044-101952

PubMed Abstract | Crossref Full Text | Google Scholar

Powers, E., and Witmer, H. (1951). An experiment in the prevention of delinquency: The Cambridge–Somerville youth study. New York, NY: Columbia University Press.

Google Scholar

Rozental, A., Castonguay, L., Dimidjian, S., Lambert, M., Shafran, R., Andersson, G., et al. (2018). Negative effects in psychotherapy: commentary and recommendations for future research and clinical practice. BJPsych Open 4, 307–312. doi: 10.1192/bjo.2018.42

PubMed Abstract | Crossref Full Text | Google Scholar

Rozental, A., Kottorp, A., Boettcher, J., Andersson, G., and Carlbring, P. (2016). Negative effects of psychological treatments: an exploratory factor analysis of the negative effects questionnaire for monitoring and reporting adverse and unwanted events. PLoS One 11:e0157503. doi: 10.1371/journal.pone.0157503

PubMed Abstract | Crossref Full Text | Google Scholar

Rozental, A., Kottorp, A., Forsström, D., Manson, K., Boettcher, J., Andersson, G., et al. (2019). The negative effects questionnaire: psychometric properties of an instrument for assessing negative effects in psychological treatment. Behav. Cogn. Psychother. 47, 559–572. doi: 10.1017/S1352465819000018

PubMed Abstract | Crossref Full Text | Google Scholar

Schneibel, R., Wilbertz, G., Scholz, C., Becker, M., Brakemeier, E.-L., Bschor, T., et al. (2017). Adverse events of group psychotherapy in the in-patient setting - results of a naturalistic trial. Acta Psychiatr. Scand. 163, 247–258. doi: 10.1111/acps.12747

Crossref Full Text | Google Scholar

Timulak, L., and Keogh, D. (2017). The client’s perspective on (experiences of) psychotherapy: a practice friendly review. J. Clin. Psychol. 73, 1556–1567. doi: 10.1002/jclp.22532

PubMed Abstract | Crossref Full Text | Google Scholar

Vaughan, B., Goldstein, M. H., Alikalos, M., Cohen, L. J., and Serby, M. J. (2014). Frequency of reporting of adverse events in randomized controlled trials of psychotherapy vs. psychopharmacotherapy. Compr. Psychiatry 55, 849–855. doi: 10.1016/j.comppsych.2014.01.001

PubMed Abstract | Crossref Full Text | Google Scholar

Vybíral, Z., Ogles, B. M., Rihacek, T., Urbancová, B., and Gocieková, V. (2024). Negative experiences in psychotherapy from clients’ perspective: a qualitative meta-analysis. Psychother. Res. 34, 279–292. doi: 10.1080/10503307.2023.2226813

PubMed Abstract | Crossref Full Text | Google Scholar

Wakefield, S., Kellett, S., Simmonds-Buckley, M., Stockton, D., Bradbury, A., and Delgadillo, J. (2020). Improving access to psychological therapies (IAPT) in the United Kingdom: a systematic review and meta-analysis of 10-years of practice-based evidence. Br. J. Clin. Psychol. 60, 1–37. doi: 10.1111/bjc.12259

Crossref Full Text | Google Scholar

Werbart, A., Annevall, A., and Hillblom, J. (2019). Successful and less successful psychotherapies compared: three therapists and their six contrasting cases. Front. Psychol. 10:816. doi: 10.3389/fpsyg.2019.00816

Crossref Full Text | Google Scholar

Whipple, J. L., and Lambert, M. J. (2011). Outcome measures for practice. Ann. Rev. Clin. Psychol. 7, 87–111. doi: 10.1146/annurev-clinpsy-040510-143938

PubMed Abstract | Crossref Full Text | Google Scholar

Williams, A. J., Botanov, Y., Kilshaw, R. E., Wong, R. E., and Sakulak, J. K. (2021). Potentially harmful therapies: a meta-scientific review of evidential value. Clin. Psychol. 28, 5–18. doi: 10.1111/cpsp.12331

Crossref Full Text | Google Scholar