Thomas Probst
Elke Humer
Andrea Jesser
Christoph Pieh- Department for Psychosomatic Medicine and Psychotherapy, University for Continuing Education Krems, Krems, Austria
Studies report that psychotherapists overestimate their own performance (self-assessment bias). This study aimed to examine if the self-assessment bias in psychotherapists differs between therapeutic orientations and/or between social comparison groups. Psychotherapists gave subjective estimations of their professional performance (0–100 scale from poorest to best performance) compared to two social comparison groups (“all psychotherapists” vs. “psychotherapists with the same therapeutic approach”). They further rated the proportion of their patients recovering, improving, not changing, or deteriorating. In total, N = 229 Austrian psychotherapists (n = 39 psychodynamic, n = 121 humanistic, n = 48 systemic, n = 21 behavioral) participated in the online survey. Psychotherapists rated their own performance on average at M = 79.11 relative to “all psychotherapists” vs. at M = 77.76 relative to “psychotherapists with the same therapeutic approach” (p < 0.05). This was not significantly different between therapeutic orientations. A significant interaction between social comparison group and therapeutic orientation (p < 0.05) revealed a drop of self-assessement bias in social comparison group “same approach” vs. “all psychotherapists” in psychodynamic and humanistic therapists (p < 0.05). Psychotherapists overestimated the proportion of patients recovering (M = 44.76%), improving (M = 43.73%) and underestimated the proportion of patients not changing (M = 9.86%) and deteriorating (M = 1.64%), with no differences between orientations. The self-assessment bias did not differ between therapeutic orientations, but the social comparison group appears to be an important variable. A major drawback is that results have not been connected to patient-reported outcome or objectively rated performance parameters.
Introduction
Research has shown that self-reported measures of therapeutic skills and patient’s progress are systematically overestimated by psychotherapists (Macdonald and Mellor-Clark, 2015). Self-assessment bias means overestimation of one’s own professional performance relative to a social comparison group (SCG) (Walfish et al., 2012). Research on the relationship between self-perception and social perception has shown the generality of a self–other asymmetry (Pronin et al., 2004). This asymmetry arises from the observation that other people’s responses to a situation sometimes differ from our own. When trying to make sense of this difference in response, we try to draw on attributions about the underlying characteristics of the other and the other’s views and priorities. In general, we assume that our perspective on a given situation is objective and thus superior to others. Consequently, we are more ready to detect or infer a bias in the other. In essence, this corresponds to the epistemological stance of naive realism (Ross and Ward, 1996). About the self-assessment bias, this means that people are more ready to view their assessments and outcomes in a positive light than those of others. This has been shown in a variety of areas of performance (Dunning et al., 2004), including health care professionals. In the United States, a mean rating of mental health care professionals’ own performance relative to SCG of others with similar qualifications at the 80th centile was found (Walfish et al., 2012), whereas the 50th could be expected due to variability between therapists (Okiishi et al., 2003). For United Kingdom mental health care providers (Parker and Waller, 2015), a smaller self-assessment bias than in the United States study (Walfish et al., 2012) relative to SCG of others with similar qualifications was observed. Both studies asked about estimations of one’s own performance relative to one SCG “others with similar qualifications.” Festingers’ social comparison theory (Festinger, 1954), however, highlights the importance of the social comparison group (degree of closeness/divergence). Also, a study on pupils observed variations in self-assessment with the specific definition of the SCG (Blatchford, 1997). Academic self-assessment dropped from the SCG “all children of the same age” to the SCG “same class” (Blatchford, 1997). Whether self-assessment of psychotherapists changes with the SCG (“all psychotherapists” vs. “psychotherapists with the same psychotherapeutic approach”) has not been assessed so far.
Although overly positive self-views might have also favorable consequences under certain circumstances (Walfish et al., 2012), systematically overestimating skills, expertise and knowledge not only prevents therapists from professional development but also increases the risk of not recognizing that some of their patients do not benefit from psychotherapy or even begin to deteriorate (Walfish et al., 2012; Macdonald and Mellor-Clark, 2015). Mental health care professionals` perceptions of patient progress might even be more important than estimations of their abilities (Walfish et al., 2012). As even among the most effective therapists a proportion of patients deteriorate (Okiishi et al., 2006), recognizing that patients are at risk of treatment failure is essential to improve patient outcomes and decrease the proportion of patients that deteriorate or drop out (Lambert et al., 2002; Hawkins et al., 2004). Studies in the United States and United Kingdom observed that therapists overestimate recovering/improving patients and underestimate deteriorating patients compared to rates found in the outcome literature (Walfish et al., 2012; Parker and Waller, 2015). The inability to recognize patients who worsen during psychotherapy has even been observed in psychotherapists who were informed about a deterioration rate of 8% (Hannan et al., 2005).
It has been reported that in routine practice a considerable proportion of patients does not recover and some even experience reliable deterioration (Hansen et al., 2002; Westbrook and Kirk, 2005). Inaccurate self-assessments might be one potential reason, as psychotherapists who estimate their own skills above average, are likely less willing to monitor their patient’s outcomes and to take measures to improve their skills and outcomes (Walfish et al., 2012; Parker and Waller, 2015). Thus, it is essential to determine potentials flaws in self-assessment in psychotherapists to provide implications for the practice of psychotherapy on the necessity to implement tools to repair potential self-assessment biases (Walfish et al., 2012). Feedback-informed treatment has been suggested as helpful for psychotherapists to become aware of patients who deteriorate during treatment and thus could serve as a countermeasure for self-assessment bias (Hannan et al., 2005). Whether self-assessment bias differs among therapeutic orientations and thus whether feedback-informed treatment should be prioritized in therapeutic orientations that are specifically prone toward self-assessment bias has not been studied so far. The current study aimed to examine the self-assessment bias in a sample of Austrian psychotherapists. Austria has a wide range of established psychotherapy approaches, which can be classified into four orientations (psychodynamic, humanistic, systemic, and behavioral) (Heidegger, 2017). The main difference between the orientations resides in their understanding of the human psyche and their theories regarding the origin of mental disorders (DiTomasso et al., 2009; Pocock, 2015; Schlippe Von and Schweitzer, 2015; Cabaniss, 2016). Accordingly, different interventions are used in the treatment, and to some extent, the setting differs, e.g., the frequency of sessions and the duration of psychotherapy. Against the background of the differences as well as similarities between the four orientations, this study aimed to investigate if the self-assessment bias in psychotherapists differs between the four therapeutic orientations and/or between social comparison groups (i.e., estimating their own performance relative to “all other psychotherapists” vs. “psychotherapists with the same therapeutic approach”). We hypothesized that irrespective of the therapeutic orientation psychotherapists overestimate their own professional performance and patient progress. Based on the human tendency to assume that people who have a similar view of the world will handle a situation similar to what we do and that people whose views differ from our own are more likely to be biased, it seems likely that differences will emerge when comparing SCGs. Thus, we further hypothesized that the self-assessment bias is reduced in a social comparison group of more closeness.
Materials and methods
Study design
The present study is based on an online survey with N = 238 Austrian psychotherapists, conducted between February and April 2021 [more details in (Humer et al., 2021)]. Psychotherapists received the link to the anonymized online survey by e-mails from the study team and the Austrian Federal Association for Psychotherapy (Vienna, Austria) supported recruitment for this study by sending their members an information e-mail. The study was conducted following the American Association for Public Opinion Research (AAPOR) reporting guidelines (Pitt et al., 2021) and according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Danube University Krems (protocol code EKGZ23-2018-2021).
In Austria, psychotherapy is a profession regulated by law. Psychotherapy training is offered by institutions accredited by the Austrian Federal Ministry of Social Affairs, Health, Care, and Consumer Protection. Training is regulated by the Austrian psychotherapy law and comprises two parts: first 765 h of lessons in theory and 550 h of practical training, second 300 advanced lessons in theory and 1,600 h of practical training. In Austria, a total of 23 psychotherapeutic approaches are accredited (Heidegger, 2017), which can be categorized into four psychotherapeutic orientations (psychodynamic, humanistic, systemic, and behavioral).
Measures
The online survey included information on gender, age, years in the profession, and therapeutic approach. Self-assessment of their professional performance as well as their estimates on therapeutic outcomes were assessed with the two measures applied in the aforementioned studies (Walfish et al., 2012; Parker and Waller, 2015). First, psychotherapists were asked to respond to the following two questions:
1. How would you rate the quality of your psychotherapeutic sessions on average, in comparison to all other psychotherapists in Austria of all 23 psychotherapeutic approaches who have similar qualifications/experience as you?
2. How would you rate the quality of your psychotherapeutic sessions on average, in comparison to all other psychotherapists in Austria who are trained in the same psychotherapeutic approach as you are and have similar qualifications/experience as you?
Psychotherapists were asked to rate both questions on a 0–100 scale, with 0 = poorest, 50 = average, and 100 = best. Extending the research conducted by Walfish et al. (2012) and Parker and Waller (2015), who asked about estimations of one’s performance relative to solely one SCG (“others with similar qualification”), two different SCGs were investigated in the study at hand. The reason for including two different SCGs is based on previous studies highlighting the importance of the social comparison group (degree of closeness/divergence) in self-assessments (Festinger, 1954; Blatchford, 1997). Second, psychotherapists were asked to rate (0–100% scales) the proportion of their patients recovering, improving, not changing, or deteriorating. The following definitions were provided: recovering: no symptoms at the end of treatment and no need for further psychotherapeutic treatment; improving: significant reduction of symptoms at end of treatment, but still some problems; not changing: no change at end of treatment; deteriorating: significant increase in symptoms at the end of treatment.
Statistics
To analyze the first measure, a repeated measure ANOVA was applied (between-subject effect “orientation” 4-levels, within-subject effect “SCG” 2-levels “all” vs. “same”). To analyze the second measure (% of recovered, improved, unchanged, and deteriorated patients), univariate ANOVAs (between-subject factor “orientation” 4-levels) were used.
Results
Of the total N = 238 participating psychotherapists (Humer et al., 2021), nine psychotherapists were excluded from further analyses as they could not be assigned to one of the four orientations. The sample characteristics of the remaining N = 229 psychotherapists are presented in Table 1. More than half of the psychotherapists were trained in humanistic approaches (n = 121), while behavioral psychotherapists represented the smallest group (n = 21). More than three-quarters were female. Their mean age was 50.66 (SD = 9.90) years and they were on average 12.70 (SD = 9.39) years in the profession. There were no significant differences between the therapeutic orientations in gender, age, and professional experience. Results of both measures of the self-assessment bias are summarized in Table 2. All measures of self-assessment bias were not affected by the therapeutic orientation (main effect “orientation” > 0.05). No one rated their own performance below the average of 50. The main effect for “SCG” [F(1,225 = 4.888; p = 0.028] in the first measure showed that, in general, self-assessment bias was smaller for the same approach vs. all psychotherapists. Bonferroni-corrected post-hoc tests to explore the interaction “orientation x SCG” [F(3,225 = 4.427, p = 0.005] revealed that psychodynamic (p = 0.004) and humanistic psychotherapists (p = 0.035) had a drop of self-assessment bias from SCG “all approaches” to “same approach.” Psychotherapists estimated that on average 44.76% of their patients recover and 43.73% improve. The proportion of patients not changing was estimated to amount to 9.86%. A mean proportion of 1.64% of the treated patients was estimated to deteriorate.
Table 1. Sample characteristics and comparison between the four therapeutic orientations in gender, age, and professional years.
Table 2. Results for the two measures of the self-assessment bias.
Discussion
Results suggest that irrespective of the therapeutic orientation, psychotherapists rate their own professional performance above average when compared to colleagues. This overestimation is more pronounced when they compare their performance with all other psychotherapists as compared to those with the same psychotherapeutic orientation. Moreover, data suggest that psychotherapists overestimate their effectiveness in terms of overestimating positive patient outcomes and underrecognizing patients that deteriorate during treatment.
Compared to other countries, evidence-based practice is still not very popular within the psychotherapy community in Austria (Nussbaumer-Streit et al., 2022). Not all of the 23 psychotherapeutic approaches accredited in Austria fulfill the criteria to be evidence-based. Also, academic education is not a prerequisite for the psychotherapeutic profession in Austria (Heidegger, 2017). Compared to other countries, the belief that overall competence is more important than science and that psychotherapy is more of an art than a science is more common among Austrian psychotherapists (Nussbaumer-Streit et al., 2022). Despite the less favorable attitude toward evidence-based practice in Austrian psychotherapists, the extent of the self-assessment bias found in the present study corresponds to findings of previous studies, especially those from the United States (Walfish et al., 2012; Parker and Waller, 2015). Karpen (2018) argued that the psychological mechanisms underlying biased self-assessment occur on an unconscious level. He concluded that strategies aiming at addressing the bias directly, such as informing people about typical biases, aim at the wrong direction. Instead, he opted for strategies that prevent the unconscious biasing mechanisms from operating. For instance, video-informed feedback from educators and colleagues as well as guidance on improvement strategies are effective in reducing self-assessment bias (Lane and Gottlieb, 2004).
In line with our hypothesis and the results by Blatchford (1997), self-assessment bias was reduced in an SCG of more closeness. Psychodynamic and humanistic therapists in particular assessed their performance closer to the performance of therapists of the same orientation than in comparison to all other therapists. This might be related to different reasons. First, differences might be related to methodological issues. Due to the differences in sample size between the four orientations, the significant effect observed in humanistic psychotherapists for the differences between self-assessment relative to the same approach vs. all psychotherapists should be interpreted with caution. As larger sample sizes enable statistical tests to detect smaller differences between groups, the significance is likely attributable to the relatively big sample size (n = 121). Absolute differences between self-assessment relative to the same approach vs. all psychotherapists were even numerically lower as compared to systemic or behavioral therapists. Second, psychotherapists likely know psychotherapists from one’s own orientation better than colleagues from other orientations, which likely led to a more realistic comparison with their skills. This finding is in line with results observed in students, showing a drop in self-assessment bias in a SCG of more closeness (Blatchford, 1997). Third, ingroup-outgroup dynamics might also play a role. Ingroup members see themselves as more similar and rate themselves as less different than outgroup members (Mussweiler and Bodenhausen, 2002).
Results also confirmed our hypothesis that irrespective of the therapeutic orientation psychotherapists overestimate their patient’s progress. On average psychotherapists estimated that an average of 1.64% of their patients leave treatment worse off than they began treatment. In contrast, it has been estimated that in 5 to 10% of adult patients participating in clinical trials, therapies have negative than rather positive consequences for the patients (Lambert and Ogles, 2004). In routine care and youths, the situation seems to be even more problematic, with deterioration rates as high as 24% (Warren et al., 2010). Results of the present study further highlight, that psychotherapists often overestimate their own patient’s progress and overlook worsening states. Monitoring patient progress and using outside feedback provides an integral part of evidence-based practice in mental health and might help psychotherapists to become aware of patients who deteriorate during treatment (Jensen-Doss et al., 2018). A growing body of evidence supports the notion that feedback-informed treatment improves performance of psychotehrapists and treatment outcomes (Miller et al., 2016).
As overestimation of professional performance and patient outcomes were observed in all therapeutic orientations to a similar extent, the results of the present study imply that it is not necessary to prioritize feedback-informed treatment implementation in specific therapeutic orientations.
Future research should compare the therapists’ self-assessments of their performance with their actual patient-reported outcomes. In addition, more social comparison groups could be investigated (e. g., gender, age, training institution, etc). Limitations of this study are that no pre-post or follow-up patient-reported outcomes were assessed and the missing objectively rated performance parameters. Therefore it is also possible that a potential self-selection bias toward more skilled psychotherapists led to an accurate reflection of their performance levels rather than a self-assemsent bias. A further drawback is the relatively small sample size, especially in the subgroups.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by Ethics Committee of the Danube University Krems, Austria. The patients/participants provided their written informed consent to participate in this study.
Author contributions
TP recruited the participants, designed the study, performed the statitical analyses, and wrote the manuscript. EH, AJ, and CP co-designed the study and reviewed the manuscript. All authors contributed to the article and approved the submitted version.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Blatchford, P. (1997). Students’ self assessment of academic attainment: accuracy and stability from 7 to 16 years and influence of domain and social comparison group 1. Educ. Psychol. 17, 345–359. doi: 10.1080/0144341970170308
DiTomasso, R. A., Golden, B. A., and Morris, H. J. (2009). Handbook of Cognitive Behavioral Approaches in Primary Care. New York, USA: Springer Publishing Company.
Dunning, D., Heath, C., and Suls, J. M. (2004). Flawed self-assessment: implications for health, education, and the workplace. Psychol. Sci. Public Interest 5, 69–106. doi: 10.1111/j.1529-1006.2004.00018.x
Festinger, L. (1954). A theory of social comparison processes. Hum. Relat. 7, 117–140. doi: 10.1177/001872675400700202
Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., et al. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. J. Clin. Psychol. 61, 155–163. doi: 10.1002/jclp.20108
Hansen, N. B., Lambert, M. J., and Forman, E. M. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clin. Psychol. Sci. Pract. 9, 329–343. doi: 10.1093/clipsy.9.3.329
Hawkins, E. J., Lambert, M. J., Vermeersch, D. A., Slade, K. L., and Tuttle, K. C. (2004). The therapeutic effects of providing patient progress information to therapists and patients. Psychother. Res. 14, 308–327. doi: 10.1093/ptr/kph027
Heidegger, K.-E. (2017). The situation of psychotherapy in Austria. Available at: http://www.europsyche.org/situation-of-psychotherapy-in-various-countries/austria/ (Accessed October 03, 2022).
Humer, E., Haid, B., Schimböck, W., Reisinger, A., Gasser, M., Eichberger-Heckmann, H., et al. (2021). Provision of psychotherapy one year after the beginning of the COVID-19 pandemic in Austria. IJERPH 18:5843. doi: 10.3390/ijerph18115843
Jensen-Doss, A., Haimes, E. M. B., Smith, A. M., Lyon, A. R., Lewis, C. C., Stanick, C. F., et al. (2018). Monitoring treatment Progress and providing feedback is viewed Favorably but rarely used in practice. Admin. Pol. Ment. Health 45, 48–61. doi: 10.1007/s10488-016-0763-0
Karpen, S. C. (2018). The Social Psychology of Biased Self-Assessment. Am. J. Pharm. Educ. 82:6299. doi: 10.5688/ajpe6299
Lambert, M. J., Whipple, J. L., Vermeersch, D. A., Smart, D. W., Hawkins, E. J., Nielsen, S. L., et al. (2002). Enhancing psychotherapy outcomes via providing feedback on client progress: a replication. Clin. Psychol. Psychother. 9, 91–103. doi: 10.1002/cpp.324
Lambert, M. J., and Ogles, B. M. (2004). “The efficacy and effectiveness of psychotherapy” in Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change. 5th Edn. ed. M. J. Lambert (New York, USA: Wiley), 139–193.
Lane, J. L., and Gottlieb, R. P. (2004). Improving the interviewing and self-assessment skills of medical students: is it time to readopt videotaping as an educational tool? Ambul. Pediatr. 4, 244–248. doi: 10.1367/A03-122R1.1
Macdonald, J., and Mellor-Clark, J. (2015). Correcting psychotherapists’ Blindsidedness: formal feedback as a means of overcoming the natural limitations of therapists: correcting psychotherapists’ Blindsidedness. Clin. Psychol. Psychother. 22, 249–257. doi: 10.1002/cpp.1887
Miller, S. D., Bargmann, S., Chow, D., Seidel, J., and Maeschalck, C. (2016). “Feedback-informed treatment (FIT): improving the outcome of psychotherapy one person at a time,” in Quality Improvement in Behavioral Health. eds. W. O’Donohue and A. Maragakis (Cham, Switzerland: Springer International Publishing), 247–262.
Mussweiler, T., and Bodenhausen, G. V. (2002). I know you are, but what am I? Self-evaluative consequences of judging in-group and out-group members. J. Pers. Soc. Psychol. 82, 19–32. doi: 10.1037/0022-3514.82.1.19
Nussbaumer-Streit, B., Jesser, A., Humer, E., Barke, A., Doering, B. K., Haid, B., et al. (2022). A web-survey assessed attitudes toward evidence-based practice among psychotherapists in Austria. Sci. Rep. 12:9374. doi: 10.1038/s41598-022-13266-2
Okiishi, J. C., Lambert, M. J., Eggett, D., Nielsen, L., Dayton, D. D., and Vermeersch, D. A. (2006). An analysis of therapist treatment effects: toward providing feedback to individual therapists on their clients’ psychotherapy outcome. J. Clin. Psychol. 62, 1157–1172. doi: 10.1002/jclp.20272
Okiishi, J., Lambert, M. J., Nielsen, S. L., and Ogles, B. M. (2003). Waiting for supershrink: an empirical analysis of therapist effects. Clin. Psychol. Psychother. 10, 361–373. doi: 10.1002/cpp.383
Parker, Z. J., and Waller, G. (2015). Factors related to psychotherapists’ self-assessment when treating anxiety and other disorders. Behav. Res. Ther. 66, 1–7. doi: 10.1016/j.brat.2014.12.010
Pitt, S. C., Schwartz, T. A., and Chu, D. (2021). AAPOR reporting guidelines for survey studies. JAMA Surg. 156:785. doi: 10.1001/jamasurg.2021.0543
Pocock, D. (2015). A philosophy of practice for systemic psychotherapy: the case for critical realism: the case for critical realism. J. Fam. Ther. 37, 167–183. doi: 10.1111/1467-6427.12027
Pronin, E., Gilovich, T., and Ross, L. (2004). Objectivity in the eye of the beholder: divergent perceptions of bias in self versus others. Psychol. Rev. 111, 781–799. doi: 10.1037/0033-295X.111.3.781
Ross, L., and Ward, A. (1996). “Naive realism in everyday life: Implications for social conflict and misunderstanding” in Values and knowledge. eds. E. S. Reed, E. Turiel, and T. Brown (New York, USA: Lawrence Erlbaum Associates, Inc.), 103–135.
Schlippe Von, A., and Schweitzer, J. (2015). Lehrbuch der Systemischen Therapie und Beratung II. Das störungsspezifische Wissen. Göttingen, Germany: Vandenhoeck and Ruprecht.
Walfish, S., McAlister, B., O’Donnell, P., and Lambert, M. J. (2012). An investigation of self-assessment bias in mental health providers. Psychol. Rep. 110, 639–644. doi: 10.2466/02.07.17.PR0.110.2.639-644
Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., and Burlingame, G. M. (2010). Youth psychotherapy change trajectories and outcomes in usual care: community mental health versus managed care settings. J. Consult. Clin. Psychol. 78, 144–155. doi: 10.1037/a0018544
Keywords: psychotherapy, self-assessment, self-assessment bias, performance, therapeutic orientations
Citation: Probst T, Humer E, Jesser A and Pieh C (2022) Attitudes of psychotherapists towards their own performance and the role of the social comparison group: The self-assessment bias in psychodynamic, humanistic, systemic, and behavioral therapists. Front. Psychol. 13:966947. doi: 10.3389/fpsyg.2022.966947
Edited by:
Henriette Loeffler-Stastka, Medical University of Vienna, AustriaReviewed by:
Anton-Rupert Laireiter, University of Salzburg, AustriaJohanna Schröder, Medical School Hamburg, Germany
Copyright © 2022 Probst, Humer, Jesser and Pieh. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Thomas Probst, thomas.probst@donau-uni.ac.at