Convergent and Discriminant Validity of the Test of Passive Aggression in a Psychotherapy Outpatient Sample

Background: While most clinical aggression questionnaires focus on the assessment of active aggression, the recently developed Test of Passive Aggression (TPA) assesses both self-directed (TPA-SD) and other-directed passive aggression (TPA-OD). Reliability and factorial validity of the TPA have been demonstrated in a clinical sample, while previous evaluations of convergent and discriminant validity were limited to student samples. The current study aimed at addressing this gap by demonstrating convergent and discriminant validity of the TPA in an outpatient sample.

Methods: Eighty-two patients admitted to an outpatient psychotherapy unit at Saarland University, Germany, participated in the preregistered study with an assessment of self-reported passive aggression, impulsivity, anger expression, self-compassion, self-esteem, and auto-aggressive mindset. Analyses used regression models with robust maximum likelihood estimations.

Results: Self-directed passive aggression showed a significant association with self-compassion, auto-aggressive mindset, self-esteem, and internal anger expression supporting the convergent validity of TPA-SD. Results on discriminant validity of TPA-SD were heterogenous at the first sight, revealing small associations of self-directed passive aggression with anger control but medium associations with impulsivity. However, exploratory analysis showed that the medium association with impulsivity was driven by the non-behavioral impulsivity dimension “inattention” and that both behavioral impulsivity dimensions (“motor-impulsivity” and “unplanned behavior”) demonstrated only weak associations with TPA-SD. Validity of TPA-OD was not supported by the current study.

Conclusion: Our findings provide evidence for the validity of the TPA-SD to outpatient samples. Future studies will need to analyze construct validity based on a nomological network using larger and more diverse samples.

Introduction

Aggressive behavior is defined as any behavior harming oneself or others in active or passive ways (Allen and Anderson, 2017). Active-aggression harms by engagement in active behavior (e.g., cutting oneself) and passive-aggression by omission (e.g., neglecting own needs; Buss, 1961). In comparison to the general population, the prevalence of both other-directed (e.g., violence: 2% vs. 10%; Swanson et al., 2015) and self-directed aggression (e.g., non-suicidal self-harm: 4% vs. 21%; Briere and Gil, 1998) is higher in individuals with mental disorders. Other-directed aggression shows its strongest associations with psychosis and substance abuse (Fazel et al., 2018), and self-directed aggression demonstrates its strongest relationship with depressive disorder as well as borderline personality disorder (Favril et al., 2020). However, both forms of aggression are related to a broad range of mental disorders, including anxiety disorders as well as trauma- and stressor-related disorders (Oram et al., 2014; Bentley et al., 2015). Although, aggressive behavior is more prevalent in inpatients, mild to moderate other-directed aggression and non-suicidal self-harm are also common in outpatients (de Klerk et al., 2011; Genovese et al., 2017). Currently, neither psychotherapeutic interventions for the reduction of other-directed nor self-directed aggression, demonstrate sufficient evidence (Rossouw and Fonagy, 2012; Stephens-Lewis et al., 2019). Thus, patients with mental disorders are a relevant population for research into aggressive behavior. However, previous research remained almost exclusively restricted to active aggression. The recently developed Test of Passive Aggression (TPA) is the first clinical assessment of self-directed (TPA-SD) and other-directed (TPA-OD) passive aggression (Schanz et al., 2021). Convergent as well as discriminant validity, and retest-reliability of the TPA have been demonstrated in a student sample. Additionally, the two-factor structure of the TPA was supported by exploratory structural equation modelling, using data of 307 patients with heterogeneous diagnoses in an inpatient setting. However, to date, support for its convergent and discriminant validity from a patient sample is missing. The current study aimed to address this gap by investigating convergent and discriminant validity of the TPA in an outpatient sample.

Constructs for Evaluating the Validity of the TPA

Self-directed aggressive behavior is assumed to be closely related to self-conscious emotions (Laye-Gindhu and Schonert-Reichl, 2005) and self-criticism (Gilbert et al., 2010). In line with this assumption, Schanz et al. (2021) found an association between TPA-SD and an auto-aggressive mindset in a student sample, which requires a replication in an outpatient sample.

Impulsivity, a disposition for unreflected and spontaneous behavior, is a predictor for other-directed (Bresin, 2019) and self-directed (Hamza et al., 2015) active-aggressive behavior. By contrast, passive-aggression harms by omission and should thus occur independently from impulsivity (Buss, 1961; Parrott and Giancola, 2007). This assumption has also been supported by our previous study.

Anger can result in different behavioral responses: internal anger expression, external anger expression as well as anger control (Spielberger et al., 1988). Previous studies showed positive relationships between higher internal anger expression and self-directed aggression, higher external anger expression and increased other-directed aggression, as well as higher anger control (AC) and decreased aggressive behavior (Parrott and Giancola, 2004; Roberton et al., 2015; Kuzucu, 2016; Lievaart et al., 2016). With respect to internal anger expression and external anger expression, a replication of this pattern is expected for both TPA scales. However, passive aggression is assumed to be rather independent from anger control, for the same reason described above for impulsivity.

Self-esteem describes a stable and trait-like evaluation of one’s own worth (Kuster and Orth, 2013). Whereas previous results regarding the association between self-esteem and other-directed active aggression are heterogeneous (Ostrowsky, 2010), the results regarding the association between lower self-esteem and higher levels of self-directed active aggression are homogenous (Forrester et al., 2017). Therefore, a medium association of TPA-SD and self-esteem is expected.

Self-compassion is defined as a kind and mindful way to deal with oneself in challenging times (Neff, 2016). As such, self-compassion can be interpreted as the opposite of a self-directed aggressive attitude. This notion is supported by previous studies that demonstrated a strong negative association between self-directed aggressive behavior and self-compassion (Xavier et al., 2016; Jiang et al., 2017). Correspondingly, an at least moderate association between TPA-SD and self-compassion is expected.

Even though, for a final prove of construct validity a nomological network would be needed (Cronbach and Meehl, 1955), the current study will provide first insights into convergent and discriminant validity of the TPA. To this end, the sum of the presence of expected correlations (i.e., at least medium associations of TPA-SD with an auto-aggressive mindset, internal anger expression, self-esteem, and self-compassion as well as at least medium associations of TPA-OD with external anger expression) and the absence of unexpected relationships (i.e., at most small associations of both TPA scales with anger control and impulsivity) will provide support for the TPA’s validity and form a foundation for its further investigation.

Materials and Methods

Participants and Procedure

Patients were recruited subsequent to their initial consultation at the Center for Behavior Therapy of the Saarland University and the Institute for Postgraduate Studies in Psychotherapy Saarbruecken. All patients were adults (age≥18years) and gave written informed consent according to latest revision of the Declaration of Helsinki. Eighty-two patients [62.20% female, M_age=39.84 (SD=13.34)] were enrolled in the study. Diagnoses were based on the Structured Clinical Interview for Mental Disorders for DSM-5 – Clinical Version (SCID-5-CV; Beesdo-Baum et al., 2019). Affective disorders (44.87%) were the most prevalent main diagnosis, followed by anxiety disorders (25.64%) and trauma- and stress-related disorders (17.95%) with 39% of the patients being diagnosed with at least two mental disorders. The study was preregistered in the German Clinical Trial Register (www.drks.de, DRKS00017321).

Measurements

Passive aggression was measured using the TPA. The TPA comprises two scales assessing self-directed (TPA-SD) and other-directed passive aggression (TPA-OD), with 12 items each. For English translations of all TPA items, see Schanz et al. (2021). TPA-SD exhibited good (α=0.82) and TPA-OD acceptable (α=0.78) internal consistency in the current sample.

Impulsivity was assessed with the 15-item short-form of the Barratt Impulsiveness Scale (BIS-15; Spinella, 2007). The BIS-15 comprises three subscales (inattention, motor impulsivity, and unplanned behavior). The global score and all subscales showed good internal consistency (α=0.80–0.85) in the present study.

Behavioral responses to anger were assessed using the Stait-Trait-Anger-Expression-Inventory-2 (STAXI-2). The STAXI-2 assesses internal anger expression (AX-I), external anger expression (AX-O) as well as AC using 26 items. An initially assumed differentiation in internal and external AC was not empirically supported in previous studies (Rohrmann, 2013). In the present sample, all STAXI-2 scales demonstrated acceptable to good internal consistency (α=0.77–0.84).

The auto-aggression scale of the Short Questionnaire for Assessing Factors of Aggression (K-FAF; Heubrock and Petermann, 2008) was used for assessing an auto-aggressive mindset (i.e., self-conscious emotions and self-criticism). It consists of nine items and demonstrated good internal consistency (α=0.81) in the current sample.

Self-compassion was assessed using the 24-item Self-Compassion Scale (SCS; Neff, 2016; Coroiu et al., 2018). In the current sample, the score of the SCS demonstrated good internal consistency (α=0.90).

Self-esteem was assessed with the Multidimensional Self-Esteem Scale (MSES; Schütz and Sellin, 2006), a German adaptation of the Multidimensional Self Concept Scale (Rotatori, 1994), comprising 32 items. In the present study, the MSES showed high internal consistency (α=0.92).

Data Analyses

All analyses were performed using the lavaan package (Rosseel, 2012) in RStudio (RStudio Team, 2019). For the evaluation of convergent and discriminant validity, bivariate associations between passive aggressive behavior and all relevant measures were analyzed using regression analyses with robust maximum likelihood estimation. Additionally, on an exploratory basis, we performed multiple regression analyses with robust maximum likelihood estimation to analyze unique association between passive aggressive behavior and each study variable under mutual control for other study variables, age and gender. Effect sizes were interpreted following the guidelines proposed by Cohen (2013); β≥0.10: small; β≥0.30: medium; and β≥0.50: strong. A post hoc confirmatory factor analysis was conducted using the lavaan package, evaluating the two-factor structure of the TPA. Due to multivariate non-normality, the analysis was performed using diagonally weighted least squares (DWLS) estimation procedure and robust error estimators (Mîndrilã, 2010). Results of this analysis should interprete with caution as the current sample was not recruited in order to examine the factorial structure of the TPA (Worthington and Whittaker, 2006). However, as the TPA constitutes a newly developed instrument, we decided to include the analysis to provide the reader with all available information on TPA’s psychometric properties.

Results

Results of factor analysis are presented in Table 1. Whereas, Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), and model test [Χ²(251)=278.44; p=0.113] supported the two-factor structure of the TPA, Standardized Root Mean Square Residual (SRMR) did not.¹ Associations between the passive-aggression subscales and study variables are presented in Table 2. With respect to TPA-SD, all associations were in line with our hypotheses, except for the association with global impulsivity. An exploratory analysis of the subscales of the BIS-15 revealed a weak association between TPA-SD and unplanned behavior (β=0.22; p=0.047) as well as with motor impulsivity (β=0.19; p=0.088), but a medium-sized relationship between TPA-SD and inattention (β=0.45; p<0.001). In line with our hypotheses, other-directed passive aggression did not account for a significant amount of variance in impulsivity and AC. However, in contrast to our assumptions, the association between other-directed passive aggression and external anger expression (AX-O) was only small. Controlling the associations of TPA-SD with study variables for TPA-OD and vice versa, did not result in a relevant change of results (see Table 2). Moreover, results also remained stable when controlling for age and gender. In a multiple regression analysis including all study variables, only an auto-aggressive mindset (K-FAF) shared a unique amount of variance with self-directed passive aggression (TPA-SD), β=0.36, t(73)=2.67; p=0.009. For TPA-OD, in a multiple regression analysis including all study variables, only self-compassion (SCS) accounted for a unique amount of variance, β=−0.31, t(73)=−2.18; p=0.033.

TABLE 1

Table 1. Results of confirmatory factor analysis.

TABLE 2

Table 2. Descriptive statistics of study variables and associations with Test of Passive Aggression (TPA) scales.

Discussion

Self-directed passive aggression harms oneself by omission of need satisfaction and disengagement in positive experiences in consequence of negative self-evaluations (Schanz et al., 2021). Consequently, we assumed the TPA-SD scale to be negatively associated with measures of positive self-evaluation (i.e., self-esteem) and with measures assessing a kind and mindful way to deal with oneself (i.e., self-compassion) and to be associated positively with measures of negative self-evaluation (i.e., auto-aggressive mindset). Additionally, we expected the TPA-SD scale to be positively associated with other measures of self-directed aggression (i.e., internal anger expression, AX-I). These hypotheses were confirmed by the current study supporting the convergent validity of the TPA-SD scale.

With respect to discriminant validity of the TPA-SD scale, results were heterogeneous. In line with our hypotheses, the relationship between TPA-SD and anger control were small and non-significant. However, TPA-SD and impulsivity – measured using the global score of the BIS-15 – showed a medium-sized association. Further exploratory analyses revealed only weak associations between TPA-SD and the behavior associated subscales of the BIS-15 (i.e., unplanned behavior and motor-impulsiveness). By contrast, the in attention subscale of the BIS-15 measures a lack of focus and does not assess any behavioral engagement at all (Meule et al., 2011). Therefore, the medium-sized association between TPA-SD and BIS-15 global score does not contradict the discriminant validity of the TPA-SD. This notion needs to be considered when investigating the validity of the TPA based on a nomological network, which should include heterogeneous relationships between self-directed passive aggression and dimensions of impulsivity.

Individuals with passive-aggressive personality try to harm others by omission instead of active engagement (Parrott and Giancola, 2007). Thus, we assumed other-directed passive aggression to be moderately associated with measures of external anger-expression, but not with measures of impulsivity or impulse control. While the evidence for the latter hypothesis was convincing, the association between other-directed passive aggression and external anger-expression was only weak. Thereby, contradicting previous results that demonstrated a medium to strong association between TPA-OD and active aggression measured with the K-FAF (Schanz et al., 2021). However, it remains unclear if these diverging results may be accounted for by methodological differences, differences in sample characteristics, or differences in assessed aggression types. Thus, future studies need to assess a broad range of operationalizations of aggression in different samples to further explore the underlying latent constructs.

The current study was conducted for evaluating convergent and discriminant validity of the TPA by examining bivariate associations with related constructs. Sample size was limited by data collection in a clinical outpatient sample but followed recommendations for the development of psychometric scales (Hobart et al., 2012; Streiner and Kottner, 2014). However, this approach may neglect the robustness of single coefficient estimations. Simulation studies suggest that such estimations require larger sample sizes, i.e., N≥250 (Schönbrodt and Perugini, 2013). Therefore, our findings need to be replicated to provide more robust estimates of regression and correlation coefficients. Moreover, such studies would also allow for the use of more advanced methods (e.g., structural equation modeling) to investigate construct validity. Therefore, future studies exploring the psychometric properties of the TPA in clinical samples of larger size are needed.

In sum, this study enlarges the knowledge on psychometric properties of the TPA-SD by providing evidence for validity in an outpatient sample. The validity of TPA-OD was not supported by the current study. Building on the findings of the present study, future studies in large-scale clinical samples should further investigate the validity of the TPA-SD as the first clinical assessment of self-directed passive aggression to provide a comprehensive psychometric evaluation. These studies should particularly focus on the relationship between self-directed passive aggression and different dimensions of impulsivity differing in behavioral consequences.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Ethics Committee of Saarland University. The participants provided their written informed consent to participate in this study.

Author Contributions

CS developed the study design, recruited the participants, analyzed the data, and prepared the materials and first draft of the manuscript. ME, SS, and TM participated in the development of the study design and reviewed the materials and manuscript. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^As DWLS estimations that were used due to the multivariate non-normality of the data assume data to be ordinally scaled, SRMR may underestimate the model fit (Mîndrilã, 2010).

References

Allen, J. J., and Anderson, C. A. (2017). “Aggression and violence: definitions and distinctions,” in The Wiley Handbook of Violence and Aggression. ed. P. Sturmey (Blackwell: Wiley), 1–14.

Google Scholar

Beesdo-Baum, K., Zaudig, M., and Wittchen, H. U. (2019). SCID-5-CV. Strukturiertes Klinisches Interview für DSM-5-Störungen—Klinische Version. Göttingen: Hogrefe.

Google Scholar

Bentley, K. H., Cassiello-Robbins, C. F., Vittorio, L., Sauer-Zavala, S., and Barlow, D. H. (2015). The association between nonsuicidal self-injury and the emotional disorders: a meta-analytic review. Clin. Psychol. Rev. 37, 72–88. doi: 10.1016/j.cpr.2015.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Bresin, K. (2019). Impulsivity and aggression: a meta-analysis using the UPPS model of impulsivity. Aggress. Violent Behav. 48, 124–140. doi: 10.1016/j.avb.2019.08.003

CrossRef Full Text | Google Scholar

Briere, J., and Gil, E. (1998). Self-mutilation in clinical and general population samples: prevalence, correlates, and functions. Am. J. Orthop. 68, 609–620. doi: 10.1037/h0080369

PubMed Abstract | CrossRef Full Text | Google Scholar

Buss, A. H. (1961). The Psychology of Aggression. New York: Wiley.

Google Scholar

Cohen, J. (2013). Statistical Power Analysis for the Behavioral Sciences. New York: Academic press.

Google Scholar

Coroiu, A., Kwakkenbos, L., Moran, C., Thombs, B., Albani, C., Bourkas, S., et al. (2018). Structural validation of the self-compassion scale with a German general population sample. PLoS One 13:e0190771. doi: 10.1371/journal.pone.0190771

PubMed Abstract | CrossRef Full Text | Google Scholar

Cronbach, L. J., and Meehl, P. E. (1955). Construct validity in psychological tests. Psychol. Bull. 52:281. doi: 10.1037/h0040957

PubMed Abstract | CrossRef Full Text | Google Scholar

de Klerk, S., van Noorden, M. S., van Giezen, A. E., Spinhoven, P., den Hollander-Gijsman, M. E., Giltay, E. J., et al. (2011). Prevalence and correlates of lifetime deliberate self-harm and suicidal ideation in naturalistic outpatients: the Leiden routine outcome monitoring study. J. Affect. Disord. 133, 257–264. doi: 10.1016/j.jad.2011.03.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Favril, L., Yu, R., Hawton, K., and Fazel, S. (2020). Risk factors for self-harm in prison: a systematic review and meta-analysis. Lancet Psychiatry 7, 682–691. doi: 10.1016/S2215-0366(20)30190-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Fazel, S., Smith, E. N., Chang, Z., and Geddes, J. R. (2018). Risk factors for interpersonal violence: an umbrella review of meta-analyses. Br. J. Psychiatry 213, 609–614. doi: 10.1192/bjp.2018.145

PubMed Abstract | CrossRef Full Text | Google Scholar

Forrester, R. L., Slater, H., Jomar, K., Mitzman, S., and Taylor, P. J. (2017). Self-esteem and non-suicidal self-injury in adulthood: a systematic review. J. Affect. Disord. 221, 172–183. doi: 10.1016/j.jad.2017.06.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Genovese, T., Dalrymple, K., Chelminski, I., and Zimmerman, M. (2017). Subjective anger and overt aggression in psychiatric outpatients. Compr. Psychiatry 73, 23–30. doi: 10.1016/j.comppsych.2016.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Gilbert, P., McEwan, K., Irons, C., Bhundia, R., Christie, R., Broomhead, C., et al. (2010). Self-harm in a mixed clinical population: the roles of self-criticism, shame, and social rank. Br. J. Clin. Psychol. 49, 563–576. doi: 10.1348/014466509X479771

PubMed Abstract | CrossRef Full Text | Google Scholar

Hamza, C. A., Willoughby, T., and Heffer, T. (2015). Impulsivity and nonsuicidal self-injury: a review and meta-analysis. Clin. Psychol. Rev. 38, 13–24. doi: 10.1016/j.cpr.2015.02.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Heubrock, D., and Petermann, F. (2008). Kurzfragebogen zur Erfassung von Aggressivitätsfaktoren: K-FAF. Göttingen: Hogrefe.

Google Scholar

Hobart, J. C., Cano, S. J., Warner, T. T., and Thompson, A. J. (2012). What sample sizes for reliability and validity studies in neurology? J. Neurol. 259, 2681–2694. doi: 10.1007/s00415-012-6570-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Jiang, Y. Q., You, J. N., Ren, Y. X., Sun, R., Liao, S. Y., Zhu, J. J., et al. (2017). Brief report: a preliminary comparison of self-compassion between adolescents with nonsuicidal self-injury thoughts and actions. J. Adolesc. 59, 124–128. doi: 10.1016/j.adolescence.2017.05.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuster, F., and Orth, U. (2013). The long-term stability of self-esteem: its time-dependent decay and nonzero asymptote. Personal. Soc. Psychol. Bull. 39, 677–690. doi: 10.1177/0146167213480189

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuzucu, Y. (2016). Do anger control and social problem-solving mediate relationships between difficulties in emotion regulation and aggression in adolescents? Educ. Sci. Theory Pract. 16, 849–866. doi: 10.12738/estp.2016.3.0003

CrossRef Full Text | Google Scholar

Laye-Gindhu, A., and Schonert-Reichl, K. A. (2005). Nonsuicidal self-harm among community adolescents: understanding the “whats” and “whys” of self-harm. J. Youth Adolesc. 34, 447–457. doi: 10.1007/s10964-005-7262-z

CrossRef Full Text | Google Scholar

Lievaart, M., Franken, I. H. A., and Hovens, J. E. (2016). Anger assessment in clinical and nonclinical populations: further validation of the state-trait anger expression Inventory-2. J. Clin. Psychol. 72, 263–278. doi: 10.1002/jclp.22253

PubMed Abstract | CrossRef Full Text | Google Scholar

Meule, A., Vögele, C., and Kübler, A. (2011). Psychometrische evaluation der deutschen Barratt impulsiveness scale–Kurzversion (BIS-15). Diagnostica 57, 126–133. doi: 10.1026/0012-1924/a000042

CrossRef Full Text | Google Scholar

Mîndrilã, D. (2010). Maximum likelihood (ML) and diagonally weighted least squares (DWLS) estimation procedures: a comparison of estimation bias with ordinal and multivariate non-normal data. Int. J. Digit. Soc. 1, 60–66. doi: 10.20533/ijds.2040.2570.2010.0010

CrossRef Full Text | Google Scholar

Neff, K. D. (2016). The self-compassion scale is a valid and theoretically coherent measure of self-compassion. Mindfulness 7, 264–274. doi: 10.1007/s12671-016-0560-6

CrossRef Full Text | Google Scholar

Oram, S., Trevillion, K., Khalifeh, H., Feder, G., and Howard, L. M. (2014). Systematic review and meta-analysis of psychiatric disorder and the perpetration of partner violence. Epidemiol. Psychiatr. Sci. 23, 361–376. doi: 10.1017/S2045796013000450

PubMed Abstract | CrossRef Full Text | Google Scholar

Ostrowsky, M. K. (2010). Are violent people more likely to have low self-esteem or high self-esteem? Aggress. Violent Behav. 15, 69–75. doi: 10.1016/j.avb.2009.08.004

CrossRef Full Text | Google Scholar

Parrott, D. J., and Giancola, P. R. (2004). A further examination of the relation between trait anger and alcohol-related aggression: the role of anger control. Alcohol. Clin. Exp. Res. 28, 855–864. doi: 10.1097/01.alc.0000128226.92708.21

PubMed Abstract | CrossRef Full Text | Google Scholar

Parrott, D. J., and Giancola, P. R. (2007). Addressing “The criterion problem” in the assessment of aggressive behavior: development of a new taxonomic system. Aggress. Violent Behav. 12, 280–299. doi: 10.1016/j.avb.2006.08.002

CrossRef Full Text | Google Scholar

Roberton, R., Daffern, M., and Bucks, R. S. (2015). Beyond anger control: difficulty attending to emotions also predicts aggression in offenders. Psychol. Violence 5:74. doi: 10.1037/a0037214

CrossRef Full Text | Google Scholar

Rohrmann, S. (2013). Das State-Trait-Ärgerausdrucks-Inventar-2: STAXI-2; Dt. Adaption Des State-Trait Anger Expression Inventory-2 V. ed. C. D. Spielberger (Bern: Huber).

Google Scholar

Rosseel, Y. (2012). Lavaan: an R package for structural equation Modeling. J. Stat. Softw. 48, 1–36. doi: 10.18637/jss.v048.i02

CrossRef Full Text | Google Scholar

Rossouw, T. I., and Fonagy, P. (2012). Mentalization-based treatment for self-harm in adolescents: a randomized controlled trial. J. Am. Acad. Child Adolesc. Psychiatry 51, 1304–1313.e3. doi: 10.1016/j.jaac.2012.09.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Rotatori, A. F. (1994). Multidimensional self concept scale. Meas. Eval. Couns. Dev. 26:265.

Google Scholar

RStudio Team (2019). RStudio: Integrated Development for R. RStudio Inc., Boston, MA. Available at: https://www.rstudio.com/ (Accessed May 31, 2021).

Google Scholar

Schanz, C. G., Equit, M., Schäfer, S. K., Käfer, M., Mattheus, H. K., and Michael, T. (2021). Development and psychometric properties of the test of passive-aggression. Front. Psychol. 12:579183. doi: 10.3389/fpsyg.2021.579183

PubMed Abstract | CrossRef Full Text | Google Scholar

Schönbrodt, F. D., and Perugini, M. (2013). At what sample size do correlations stabilize? J. Res. Pers. 47, 609–612. doi: 10.1016/j.jrp.2013.05.009

CrossRef Full Text | Google Scholar

Schütz, A., and Sellin, I. (2006). Multidimensionale Selbstwertskala: MSWS. Göttingen: Hogrefe.

Google Scholar

Spielberger, C. D., Krasner, S. S., and Solomon, E. P. (1988). “The experience, expression, and control of anger,” in Individual Differences, Stress, and Health Psychology. Contributions to Psychology and Medicine. ed. M. P. Janisse (New York: Springer).

Google Scholar

Spinella, M. (2007). Normative data and a short form of the Barratt impulsiveness scale. Int. J. Neurosci. 117, 359–368. doi: 10.1080/00207450600588881

PubMed Abstract | CrossRef Full Text | Google Scholar

Stephens-Lewis, D., Johnson, A., Huntley, A., Gilchrist, E., McMurran, M., Henderson, J., et al. (2019). Interventions to reduce intimate partner violence perpetration by men who use substances: a systematic review and meta-analysis of efficacy. Trauma Violence Abuse doi: 10.1177/1524838019882357 [Epub ahead of print].

CrossRef Full Text | Google Scholar

Streiner, D. L., and Kottner, J. (2014). Recommendations for reporting the results of studies of instrument and scale development and testing. J. Adv. Nurs. 70, 1970–1979. doi: 10.1111/jan.12402

PubMed Abstract | CrossRef Full Text | Google Scholar

Swanson, J. W., McGinty, E. E., Fazel, S., and Mays, V. M. (2015). Mental illness and reduction of gun violence and suicide: bringing epidemiologic research to policy. Ann. Epidemiol. 25, 366–376. doi: 10.1016/j.annepidem.2014.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Worthington, R. L., and Whittaker, T. A. (2006). Scale development research: a content analysis and recommendations for best practices. Couns. Psychol. 34, 806–838. doi: 10.1177/0011000006288127

CrossRef Full Text | Google Scholar

Xavier, A., Pinto-Gouveia, J., and Cunha, M. (2016). The protective role of self-compassion on risk factors for non-suicidal self-injury in adolescence. Sch. Ment. Heal. 8, 476–485. doi: 10.1007/s12310-016-9197-9

CrossRef Full Text | Google Scholar