Climate of Accountability, Respect, and Ethics Survey (CARES): development and validation of an organizational climate survey

Background: This research describes the development and validation of the CARES Climate Survey, a 22-item measure designed to assess interpersonal dimensions of work-unit climates. Dimensions of work-unit climates are identified through work-unit member perceptions and include civility, interpersonal accountability, conflict resolution, and institutional harassment responsiveness.

Methods: Two samples (N = 1,384; N = 868) of academic researchers, including one from the North American membership of the American Geophysical Union (AGU), and one from a large research-intensive university, responded to the CARES and additional measures via an online survey.

Results: We demonstrate content validity of the CARES measure and confirm structural validity through exploratory and confirmatory factor analyses which yielded four dimensions of interpersonal climate. In addition, we confirm the CARES internal reliability, construct validity, and excellent sub-group invariance.

Conclusions: The CARES is a brief, psychometrically sound instrument that can be used by researchers, institutional leaders, and other practitioners to assess interpersonal climates in organizational work-units.

Originality/value: This is the first study to develop and validate such a measure of interpersonal climates specifically in research-intensive organizations, using rigorous psychometric methods, grounded in both theory and prior research on work-unit climates.

Introduction

Organizational members who are skilled and competent in work-related tasks is a necessary, but not sufficient condition for a highly functioning organization. Creating and maintaining the highest levels of organizational effectiveness, productivity, and integrity requires members' full engagement with their work. To support full engagement, organizational members must experience psychological safety, feel respected and valued, and have a sense of positive interpersonal interactions (Bryer, 2020; Frazier et al., 2017; Knapp et al., 2014; Komisarof, 2022). Because most organizational members must interact with others to get their work done, the nature, competence, and integrity of their interpersonal interactions influences the quality and quantity of work performed as well as relevant psychological outcomes for the members themselves (Liden et al., 2000; Reich and Hershcovis, 2011). Organizational cultures and climates have been identified as important facilitators or modifiers of interpersonal interactions, both positively and negatively (Andersson and Pearson, 1999; Willness et al., 2007) and correlate with the integrity of creative efforts such as research in academic settings; with positive climates being associated with higher integrity behavior and negative climates being associated with lower integrity behavior (Crain et al., 2013). Work units that provide climates of respect and integrity also support the best possible outcomes for individual organizational members (Feldblum and Lipnic, 2016; Frone, 2000; NASEM, 2018), their performance (Carr et al., 2003), and their creativity (Tu et al., 2019). Thus, interpersonal climates in organizations are key to the success of its members, and the ability to measure such climates can be a powerful strategy for minimizing negative interpersonal interactions and maximizing their quality and integrity (Walsh and Magley, 2019).

Effective leadership in organizations must include a focus on interpersonal aspects of the workplace and psychologically safe workplaces (Cross et al., 2019; Haven et al., 2021; Valantine et al., 2021). A primary goal of organizational administration and leadership is to facilitate members' ability to produce their best work. Yet, the nature of large institutions can leave leaders somewhat removed from the day-to-day interactions in the organizational sub-units over which they have charge. Without valid and useful information about the interactions in units across their organizations, how can these leaders know how well their organizational settings or climates support their members and foster the highest integrity of work with effectiveness, accountability, and respect? The purpose of our work here is to develop and validate a psychometrically sound survey to assess interpersonal work-unit climates, particularly within but not limited to academic research-focused environments. Members of our team have long been measuring research integrity climate and civility in organizations (Martinson et al., 2013; Walsh et al., 2011) but realized the lack of validated tools for assessing micro-climates relevant to lab/team level interpersonal dimensions that are important to maintaining accountable, ethical, respectful working environments.

There are two fairly distinct categories of prior work on organizational climate, one that has emphasized global or generic climate, and a second that has emphasized a strategic focus on specific aspects of organizational climates such as service, safety, justice, diversity, civility, and integrity. While multiple of these foci overlap to some extent with interpersonal climate, we have not identified any climate instrument that adequately captures interpersonal climate, despite the fact that many climate measures contain large numbers of scale items. Many existing climate assessment tools also either downplay or entirely ignore the role of power and power-differentials in organizations as factors that affect organizational member behavior and outcomes. Yet, in academic research environments, supervision dynamics, particularly inefficient supervision has been identified as an important barrier to fostering responsible research climates (Haven et al., 2020; Kis et al., 2022). Moreover, in much prior work the voices of women and those of various under-represented groups are not much represented. The development of the climate measure described here has been informed by a comprehensive understanding of interpersonal climate and an effort to understand how that climate is manifest among members of workgroups in organizations.

Contextual factors, what we refer to as organizational climate, have been identified as precursors to problematic interpersonal behavior (Andersson and Pearson, 1999; Pearson et al., 2000). Time-pressures, resource uncertainties, and strong status hierarchies can create opportunities for disrespectful interpersonal interactions between organizational members that over time can become entrenched as workplace norms (Pearson et al., 2001), in turn, leading to disengagement, disaffection, and behaviors that run counter to desired organizational outcomes (Pearson and Porath, 2005).

Among many problematic interpersonal behaviors, sexual harassment has perhaps received the greatest attention and has been the focus of perhaps the most organizational research and policy (NASEM, 2018). It is also a type of behavior in organizations for which laws exist as sanctions against its occurrence. Importantly, sexual harassment has also been contextualized in a way that suggests it is better understood not primarily as sexualized behavior, but as a misuse and abuse of power, that often takes sex-based forms. In their study of 8th Circuit Federal Court employees, Lim and Cortina (2005) document significant correlations between sexualized harassment and incivility, and even stronger correlations between incivility and the construct of gender harassment, the most frequent sub-type of sexualized harassment which reflects sex-related put-downs and sexism. Others have widened the window of discourse to include toxic leadership, bullying, and harassment, and to distinguish behaviors oriented toward expressing dominance over others from strictly sexually oriented behavior (Berdahl et al., 2018; Berdahl and Bhattacharyya, 2021; Glick et al., 2018). Many of these authors have noted that disrespectful, hostile, and intimidating behavior may be targeted toward numerous groups of organizational members based on multiple dimensions of their identities, often intersectionally.

While organizational climates have often been demonstrated to contribute to many types of undesirable interpersonal behavior, efforts to develop more positive organizational climates can help sustain prosocial norms in the workplace that promote members' sense of safety, respect, and belonging (Feldblum and Lipnic, 2016; NASEM, 2018; Walsh and Magley, 2019). Interpersonal collaboration, itself predicated on genuine respect between co-workers, has been shown to be more important than a sense of purpose in maximizing employee engagement (Cross et al., 2019). Moreover, the quality of interpersonal aspects of work have also been identified as a key component of worker wellbeing (Lovejoy et al., 2021). Our work here is concerned with the kinds of detrimental interpersonal behaviors that have previously been characterized by Andersson and Pearson (1999) as “low-intensity deviant behavior with ambiguous intent to harm the target, in violation of workplace norms for mutual respect” (p. 457).

Concerns about interpersonal misbehavior in research organizations is directly connected to, though not limited to, concerns about research integrity. In recent years, the mistreatment of individuals (e.g., sexual harassment), has been increasingly recognized as undermining excellence in research, and being a form of research misconduct in and of itself. Examples of this include the 2018 NASEM Report on Harassment in STEMM (NASEM, 2018), the inclusion of “discrimination, harassment (including sexual harassment), and bullying” as forms of research misconduct in the American Geophysical Union (AGU)'s 2017 Integrity and Ethics Policy in its Code of Conduct (https://www.agu.org/-/media/Files/Learn-About-AGU/AGU_Scientific_Integrity_and_Professional_Ethics_Policy_document.pdf), the September 2018 announcement by the NSF of new reporting requirements by institutions of sexual harassment on the part of NSF research grantees (Mervis and Kaiser, 2018), and a subsequent, comparable strengthening of polices toward sexual harassment on the part of NIH (Kaiser, 2019). Given the potential importance of organizational climate, it is unfortunate that interpersonal organizational climate has generally received less attention than other organizational factors that influence the quality of outputs and members' wellbeing. Based on our review of existing measures, there has not been a comprehensive instrument proposed or developed to assess the broader scope of interpersonal organizational climates with which we are concerned. Until now, while specific measures of incivility climate have been developed— Civility Norms Questionnaire-Brief (CNQ-B) (Walsh et al., 2011) and the Survey of Organizational Research Climates (SOURCE), for assessing research integrity climates in academic research settings (Martinson et al., 2013)— we have found no instrument designed specifically to assess climates of organizational work-units that includes the constellation of civility, respect, misuses of power (e.g., harassment, bullying), and conflict resolution. It should be noted that instruments exist for assessing sexual assault and harassment in academic settings (Cantor et al., 2015), but these fall short of the goal of broadly assessing interpersonal climate. On the one hand, extant instruments fail to assess organizational climates, rather querying direct personal experiences, and on the other hand, lack complete coverage of important dimensions of these climates. To be clear, we did not seek to develop a measure of the frequency or prevalence of interpersonal interactions whether respectful or harassing. We have specifically avoided inclusion of experientially framed survey items that tap concepts such as harassment frequency.

In defining organizational climate, we follow the definition offered by Ehrhart, Schneider and Macey that it is “the shared meaning organizational members attach to the events, policies, practices, and procedures they experience and the behaviors they see being rewarded, supported, and expected” (Ehrhart et al., 2013, p. 115). The factors identified by Ehrhart et al. as markers of organizational climate are the observable (and therefore reportable) aspects of organizational life that point to the deeper organizational culture. Thus, we sought to develop a measure with items that focus on such observable and reportable features of members' working environments. Emergent dimensions of interpersonal climate addressed by these items include a broad spectrum from the positive (psychological safety and civility) to the negative (bullying, harassment, assault).

Although not encompassing of a broad understanding of interpersonal climate, we expect the psychological safety experienced by individuals, and the protection of a sense of psychological safety through accountability for misbehavior, respectful conflict resolution, and institutional level supports for positive interpersonal interactions, are formative of a positive interpersonal workgroup climate. Psychological safety is considered present when members hold perceptions that they are able to engage in interpersonal risk taking within the workgroup without fear of negative personal or professional consequences (Edmondson, 1999; Kahn, 1990) and is instrumental in the willingness to share ideas and act in concert with others (Frazier et al., 2017). Critical aspects to the perceptions of psychological safety include a sense of support and respect from other workgroup members, the creation of expectations and norms through leader behavior, and the expectation that institutions place importance on these group norms and interpersonal relationship expectations (Frazier et al., 2017). Therefore, we sought to develop a measure that includes items reflecting all three perceptual levels of contribution to interpersonal climates—co-workgroup member interaction climate, leader influence, and institutional practices and processes.

Finally, it is important to note that workgroup-level interpersonal climate as conceptualized here is a necessarily broad concept which encompasses relevant theoretical aspects and levels of influence, because members do not cognitively map all potential aspects nor hold each with the same level of salience. In order to create a maximally useful and applicable measure, capturing the totality of interpersonal climates, and that is sufficiently brief to be employable in most organizational research studies, we engaged in a predominately inductive process that captured a comprehensive representation of interpersonal organizational climate. In this article we describe the development of this new measure and evaluate its psychometric properties including content validity, structural validity internal reliability, as well as construct validity and factorial test invariance for multiple characteristics of respondent subgroups (i.e., race, gender, career stage).

Method and results

Scale development overview

In developing the interpersonal climate survey we relied on broadly accepted scale development methods (e.g., Hinkin, 1998; Rahim and Magner, 1995) across multiple phases, including literature review, item generation, content validation, exploratory and confirmatory factor analyses, and nomological validity estimates. This process produced a 22-item scale and revealed four distinct dimensions of the scale. It should be noted that while we believe this measure which we named the Climate of Accountability, Respect and Ethics Survey (CARES) is suitable for use in a broad range of organizational settings, the empirical work on which we report here was conducted primarily in research settings, including, but not limited to universities. Please refer to the flowchart in Figure 1 for a map of our work-flow.

Figure 1

Figure 1. CARES development and validation flowchart.

Item generation

We worked with a professional librarian to conduct a search of online databases including PubMed, Sociological Abstracts, Web of Science, Academic Search Premier, and Business Source Premier for existing scales and surveys covering our conceptual dimensions of interest using specific search-terms including harassment, assault, bullying, intimidation, psychological safety, civility, and organizational climate. We combined those results with additional manuscripts identified by team members and other content experts resulting in a final collection of 198 articles, manuscripts, and reports. All abstracts and summaries of these materials were reviewed by three team members (BCM, JS, CRT) who identified 45 sources as relevant for a full-text review. This review resulted in an initial pool of 618 scale items covering a breadth of conceptual dimensions (including psychological safety, civility, bullying, harassment, and assault) across the domains of policies, procedures, leadership practices, socialization, norms, expectations, and institutional resources. Those items were subjected to an iterative two-rater process to sort and organize each by dimension and domain. We culled items with substantially duplicate content and evaluated the remaining items to ensure topic coverage, reducing the pool to 132 items covering all conceptual domains and which were not substantially duplicative. See Supplementary materials 2, 3 files for further description of our process here, and the artifacts of that process.

This pool of 132 items provided a conceptual landscape of domains of focus for existing scales. Four-members of our research team (BCM, JS, VM, CRT) extensively reviewed this pool of items, discussing as a team to reach consensus to further reduce the number of candidate items by culling experiential and poorly worded items, recrafting experientially worded items into climate-focused items, creating new items where concepts were found to be missing, and reframing item wording as necessary. This final stage of initial item generation produced 61 items for further consideration addressing climate across all conceptual domains.

Subject matter expert review

The authors invited 35 subject matter experts (SME's) to review and provide feedback on the 61 candidate items. The invited pool consisted of 23 women and 12 members of minority racial groups. SME's were chosen based on one or more of the following criteria: (1) expert scholars in organizational psychology, sexual misconduct, harassment, psychological safety, or professionalism in academia, (2) panel members of the 2018 NASEM Report on Sexual Harassment (NASEM, 2018), (3) panel members of the 2019 NIH Working Group report on sexual harassment (Kaiser, 2019), (4) academic professionals (e.g., journal editors) with experience and interest in interpersonal working conditions, and (5) graduate students or postdoctoral research fellows. Of the invited pool, 18 SME's provided quantitative ratings indicating the extent to which each item was relevant to workplace climate as well as extensive qualitative feedback. See Supplemental material 1 for further details about the SME identification and recruitment process, as well as details regarding the characteristics of the SME candidate pool and ultimate participants, what input we requested from them and how that was processed, as well as the instructions and data-collection forms provided to SMEs. The authors engaged in extensive analysis and discussion of these results using a consensus approach that led to substantial revisions to most items and a further vetting process which produced a final list of 46 items which we carried forward for large-scale data collection and psychometric analysis.

Sample 1 factor analysis participants

We invited individuals from the North American membership of the American Geophysical Union (AGU), an international nonprofit scientific professional association of earth and space scientists, who were active researchers to complete online surveys designed to assess their work-unit climates. See Supplemental material 4 for a copy of the survey instrument as it was implemented in Qualtrics. Participation was voluntary and participants were assured of the confidentiality of the study. All study activities reported in this paper were approved by the University of Illinois Urbana-Champaign Institutional Review Board. A total of 27,753 members were invited, and among 2,572 who consented to participate (9%), 2,253 reported being actively engaged in research and therefore eligible to participate (87% of those consenting). We omitted a further 869 due to suspect or missing responses, ultimately resulting in 1,384 usable surveys, representing 61% of the eligible, consenting participants. Responses were considered missing if half or more of the climate response items remained unanswered. Responses were considered suspect if all items in a climate scale were answered with the same response value. The demographic breakdown was as follows: 45% female, 50% male, and 4.6% not answering; 0.2% Native Hawaiian or Pacific Islander, 0.3% American Indian or Alaskan Native, 1% Black or African American, 5% Hispanic or Latino, 8% Asian, 73% white, 6% not listed or multiracial, and 6% not answering; 16% students, 29% early career researchers, 27% mid-career researchers, 23% late career researchers, and 21% not answering. We divided the Sample 1 data set into two random samples of 692 individuals each (Sample 1a and Sample 1b in Figure 1) to conduct exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). When evaluating model fit for the purposes of factor analysis, the Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), root mean squared error of approximation (RMSEA), and standardized root mean squared residual (SRMR) of the models were considered. Cutoff values of 0.95 for CFI and TLI, 0.06 for RMSEA, and 0.08 for SRMR indices were used as indications of relatively good fit (Hu and Bentler, 1999). All analyses were conducted using R version 4.0.3 utilizing maximum likelihood (ML) estimation unless otherwise noted.

Measures

Response options for all candidate items were scaled from 1 (Not at All) to 5 (Completely) with an additional “No Basis for Judging” option. We modeled the response scales here from those we previously employed in the SOURCE (Martinson et al., 2013). Scale reliabilities for the CARES dimensions and for the overall scale are reported for the CFA sample.

Workgroup climate (CARES)

The interpersonal climate of the workgroup was measured with the 22-item CARES climate scale developed as described below and items for which are found in Appendix 1.

Research integrity climate (SOURCE)

Participants rated the research integrity climate of their workgroup and institution with the 28-item SOURCE climate scale from Martinson et al. (2013). An example item from the SOURCE is “How true is it that pressure to publish has a negative effect on the integrity of research in your department?” (α = 0.95, ω = 0.96).

Integration

Participants rated their feeling of being integrated into their department with a single-item, “How integrated within your department do you feel?”

Continuance commitment

Participants rated their level of continuance commitment to their organization using an abridged 3-item version of the continuance scale from Allen and Meyer (1990). An example item is “It would be very hard for me to leave my organization right now, even if I wanted to (α = 0.78, ω = 0.78).

Exploratory factor analysis—sample 1a

We subjected items resulting from the subject matter expert review portion of development to an EFA using principal components analysis with an unspecified number of factors and utilizing the first split half of Sample 1 (Sample 1a in Figure 1). Four factors with eigenvalues >1 were revealed. These results were supported by a parallel analysis of an uncorrelated matrix representing random sampling error where factors generated by the EFA with eigenvalues greater than the corresponding eigenvalue from the parallel matrix represent underlying components. The results were further supported by an optimal coordinates analysis where factors with eigenvalues which diverge from (i.e., exceed) a regression line from the last eigenvalue through the preceding eigenvalue also returned four factors. Finally, we verified these results through visual inspection of the scree plot.

Because of anticipated intercorrelation among factors, we employed oblique rotation to interpret loadings of the items onto each factor. We interpreted each factor based on items which loaded onto that factor with weights above 0.5 and with no cross loading of more than 0.3 onto other factors. To create a reliable scale, we eliminated items with overly ambiguous loading patterns or that loaded poorly on their primary factor. We identified the CARES scale as one construct good internal reliability (α = 0.95, ω = 0.96) consisting of four dimensions identified after rotation and consisting of the following number of items: civility climate (a climate of respectful and supportive treatment of other workgroup members consisting of 7 items with internal reliability α = 0.90, ω = 0.92), interpersonal accountability climate (a climate where hostile, intimidating, and unhealthy interpersonal behaviors are not tolerated consisting of 7 items with internal reliability α = 0.93, ω = 0.94), conflict resolution climate (a climate where leaders expect and facilitate respectful resolution of interpersonal conflict consisting of 3 items with internal reliability α = 0.71, ω = 0.76), and institutional harassment responsiveness (institutional level policies and behaviors which unambiguously communicate that the institution does not tolerate harassment consisting of 5 items with internal reliability α = 0.94, ω = 0.94).

Confirmatory factor analysis—sample 1b

The next step in validating the 22-item workgroup climate scale was to perform a CFA on the second split half of Sample 1, which produces assessments of goodness of fit and can be used to confirm previously hypothesized models. The CFA was based on the covariance matrix and used maximum likelihood estimation, and the results are shown in Table 1.

Table 1

Table 1. Sample 1b confirmatory factor analyses (second split half).

Consistent with our EFA results, our hypothesized model was a 4-factor model with seven items each for the civility and interpersonal accountability climate dimensions, three items for the conflict resolution climate dimensions, and five items for the institutional harassment responsiveness dimension. The hypothesized model produced a chi-squared value of 747.281 (df = 203, p < 0.01), and the fit indices indicated a good overall fit (CFI = 0.95; RMSEA = 0.062; SRMR = 0.043). We tested several alternative models, and all failed to produce a better fit than the four-factor model. Thus, the hypothesized model derived from the EFA was confirmed. The means, standard deviations, and correlations for the second split half of Sample 1 can be found in Table 2.¹

Table 2

Table 2. Sample 1 means, standard deviations, and correlations among variables.

Members of workgroups may perceive working environments differently based on individual identity, group membership, career rank, and other factors. It is therefore instructive to test the current instrument for measurement invariance across groups to ensure that members of different groups perceive scale items similarly. Evaluating measurement invariance across groups aids in establishing the efficacy of the instrument to investigate differences in perception across groups in addition to perceptions of workgroup members in general. We conducted multigroup confirmatory factor analysis to test the invariance of parameters of nested models across groups where subsequent sets of parameters are allowed to vary across groups in one model and constrained to be constant across groups in the nested model. If model fit is similar in both the unconstrained and constrained models, we can conclude that the model is invariant across groups for the targeted parameter. We evaluated the CARES instrument sequentially across four levels of invariance: configural invariance (indicating that the model contains the same number of factors with the same items loading onto each factor across groups), metric invariance (indicating that item factor loadings are equivalent across groups), scalar invariance (indicating that slope intercepts are equivalent across groups), and strict invariance (indicating that the error variance is equivalent across groups). Achieving scalar invariance indicates that both the model structure and observed mean scores are comparable across groups (Steenkamp and Baumgartner, 1998; Steinmetz, 2013). Strict invariance is unnecessary to establish meaningful measurement comparisons across groups (Luong and Flake, 2023; Steenkamp and Baumgartner, 1998), however, we include those results here for interest and completeness.

We tested invariance across several groups. First, we examined invariance for those who reported themselves as belonging to either the majority or minority gender in their workgroup, and the majority or minority race in their workgroup. We had enough participants to further examine invariance across the four intersections of these categories. We further examined the scale dependent on whether participants reported their sex as female or male. We did not have enough of each category to examine invariance across all reported racial groups. We grouped responses into the racial groups of BIPOC and non-BIPOC, which provided sufficient responses in each group to examine scale invariance across groups. Finally, we examined scale invariance across individuals who reported themselves as being in four different career stages (student, early career, mid-career, and experienced). Measurement invariance is confirmed when ΔCFI < 0.010 between nested models (Cheung and Rensvold, 2002). Results are presented in Table 3 and demonstrate the strongly consistent performance of the scale across groups, showing configural, metric, and scalar invariance across all groups and intersections of groups of interest collected in this sample and strict invariance across most of those groups. These results provide substantial confidence in the use of the CARES scale to meaningfully interpret and compare perceptions of participants from many different groups.

Table 3

Table 3. Sample 1b multi-group confirmatory factor analyses (second split half).

The scale showed good configural, metric, and scalar invariance of fit across all respondent groups. Together, these findings support a multidimensional conceptualization of the CARES climate scale, which is durable across member groups, allowing for reliable comparison across groups. Given these findings, we proceeded to explore the ability of workgroup climate to account for variance in group member outcomes (i.e., workgroup integration, organizational citizenship behavior, work unit diversity climate, work unit experiences, continuance commitment to one's employer).

Sample 2 confirmatory factor analysis

Participants

We invited 5,489 individuals from a large, R1, public university in the Midwest of the United States. Our primary purpose for this was to conduct confirmatory factor analyses in a sample of researchers that was more comprehensive of scientific disciplines not represented in the AGU membership sample (Sample 1). Participation was voluntary and participants were assured of the confidentiality of the study. A total of 1,302 consented and reported on a screening item that they were involved in research activities in the university. Similar to sample 1, of these, 549 were omitted due to suspect or missing responses, resulting in a usable sample of 868 participants (a 16% participation rate). The demographic breakdown was as follows: 39% female, 55% male, and 6% not answering; 0.1% American Indian or Alaskan Native, 21% Asian, 3% Black or African American, 6% Hispanic or Latino, 0% native Hawaiian or Pacific Islander, 59% white, 5% not listed or multiracial, and 12% not answering; 34% graduate students, 18% research support including post-doctoral researchers, 41% tenure track faculty, and 8% not answering; 49% early career, 32% mid-career, 18% late career, and 1% not answering.

Measures

Scale reliabilities for the CARES dimensions and for the overall scale are reported for the CFA sample.

Workgroup climate (CARES)

The interpersonal climate of the workgroup was measured with the 22-item CARES climate scale developed here (α = 0.94, ω = 0.97). The reliabilities of the dimension of the scale were: civility climate (α = 0.90, ω = 0.93), interpersonal accountability climate (α = 0.94, ω = 0.96), conflict resolution climate (α = 0.79, ω = 0.84), and institutional harassment responsiveness (α = 0.95, ω = 0.96).

Research integrity climate (SOURCE)

Participants rated the research integrity climate of their workgroup and institution with the 28-item SOURCE climate scale (Martinson et al., 2013) (α = 0.94, ω = 0.95).

Sample 2 results

Confirmatory factor analysis

As with Sample 1b, we validated the 22-item workgroup climate scale by conducting a CFA against the data obtained in this sample with the results reported in Table 4. The hypothesized model produced a chi-square value of 753.356 (df = 203, p < 0.01), and the fit indices indicated a good overall fit (CFI = 0.97; RMSEA = 0.056; SRMR = 0.034). We tested several alternative models, and all failed to produce a better fit than the four-factor model. Thus, the hypothesized model obtained with Sample 1b was confirmed.

Table 4

Table 4. Sample 2 confirmatory factor analyses.

We proceeded to evaluate the CARES instrument sequentially at the four levels of invariance outlined previously across the same breakdown of subgroups as we did with Sample 1b. for several groups available for analysis in this sample. Measurement invariance was confirmed when ΔCFI < 0.010 between nested models (Cheung and Rensvold, 2002). Results are presented in Table 5 and demonstrate the strongly consistent performance of the scale across groups, showing configural, metric, and scalar invariance across all groups and intersections of groups of interest collected in this sample and strict invariance across most of those groups. As with Sample 1b, these results provide substantial confidence in the use of the CARES instrument to meaningfully interpret and compare perceptions of participants from many different groups.

Table 5

Table 5. Sample 2 multi-group confirmatory factor analyses.

Because of the multi-level nature of influences on workgroup climates, we sought to evaluate the potential for aggregation of scale scores at the group level. Given that the data from Sample 2 was secondary, we did not have the ability to fully specify the information collected. Even though the data did not contain sufficient information to analyze grouping at the workgroup level (the level for which we expect the largest group-level effect for a climate measure), we were able to analyze the data at a higher, unit level of analysis where each unit contained several workgroups. Even though analysis at this level is less than ideal and is expected to result in an underestimation of group-level effects, we proceeded to calculate ICC(1), ICC(2) (Bliese, 2000),

$ICC\left(1\right),\rho =\frac{{\sigma }_{u_0}^2}{{\sigma }_{u_0}^2+{\sigma }_r^2};ICC(2),{\lambda }_j=\frac{{\eta }_j\rho }{1+({\eta }_j- 1)\rho }$

and r_wg(j) (James et al., 1993) values for the overall CARES scale and each dimension. The results are found in Table 6.

Table 6

Table 6. Sample 2 group level clustering indices.

Construct validation

While the nature of this work was to some extent exploratory, we had some expectations about how the CARES would relate to other measures. For some of our hypotheses we expected strong positive correlations (>0.50), while for other hypotheses we expected weaker correlations (< 0.30). For one measure, we expected a weak negative correlation.

When individuals feel a sense of relationship and inclusion with the organization and with coworkers, those members will also experience an increased sense of integration with the organization (Blau, 1960). Following this logic, we expect that a positive interpersonal climate which encourages healthy conflict resolution, establishes respectful interactions as a workgroup norm, promotes fair treatment of all members, and discourages hostile and unhealthy behavior will, in turn, result in a feeling of integration in the workplace.

Hypothesis 1: Interpersonal climate is strongly positively related (>0.50) to a feeling of integration in the workplace.

Continuance commitment is the commitment felt by employees not as a result of an affective or normative desire to stay with an organization, but rather an intention to remain with an organization only because of a perceived lack of viable alternatives (Allen and Meyer, 1990; Meyer and Allen, 1984). Employees who have remained in an organization are likely to express a sense of continuance commitment in the absence of more salient drivers of a desire to remain with the organization. Negative work environments, such as those created by abusive supervision, are positively related to continuance commitment (Tepper, 2000). Climates which are exemplified by a positive sense of justice are negatively related to continuance commitment. We, therefore, expect a positive interpersonal climate exemplified by a sense of fair and respectful interactions will be negatively related to continuance commitment.

Hypothesis 2: Interpersonal climate is weakly negatively (<-0.30) related to continuance commitment.

Our expectation of a weak negative correlation between interpersonal climate and continuance commitment is based on our recognition that continuance commitment is driven by many factors extrinsic to the organizational climate and often specific to the personal constraints individual organizational members may be under due to circumstances beyond their current position.

Organizational citizenship behaviors (OCB's) are voluntary behaviors which are intended to benefit the organization and coworkers (Organ, 1988, 1997) and result from group norms (Ehrhart and Naumann, 2004). Therefore, workgroups who experience the group norm of a positive interpersonal climate—whose members are supportive and respectful of each other and do not have to guard against others unjustly taking credit for accomplishments, where affective and cognitive interpersonal ties are strengthened, and where members feel supported by both leaders and the institution—will include members who are more likely to engage in higher levels of organizational citizenship behaviors. Furthermore, individuals who operate within a positive interpersonal climate are expected to create positive affect and cognition directed toward coworkers and the organization and will engage in increased levels of OCB (Frazier et al., 2017; Organ and Konovsky, 1989). Such individuals will also develop a sense of social exchange and reciprocity with the foci of that climate (i.e., the institution, leaders, and workgroup members) leading to increased levels of citizenship behaviors (Settoon et al., 1996).

Hypothesis 3: Interpersonal climate is strongly positively (>0.50) related to organizational citizenship behaviors.

A positive interpersonal climate which includes expressions of interpersonal respect and civility and which guards against hostile, demeaning, and harassing behaviors at the levels of the institution, workgroup leadership, and coworker interactions will foster a climate where individuals are not excluded or harassed for exhibiting diverse aspects. Rather, it will foster a climate which is receptive to differences among its members resulting in increased levels of work unit diversity (WUD).

Hypothesis 4: Interpersonal climate is strongly positively (>0.50) related to work unit diversity.

Organizational climates have been shown to have strong and positive influences on corresponding individual behaviors within organizations (Schneider et al., 2013) which provides the basis for our expectations that climate is related to interpersonal experiences within an organization. Consequently, we expect that a positive interpersonal climate will be positively associated with workgroup member behavior-related experiences (WUE) exemplified by low levels of incivility and high levels of inclusion, cohesion, and citizenship.

Hypothesis 5: Interpersonal climate is strongly positively (>0.50) related to work unit experiences.

A positive interpersonal climate is an important component of creating a workgroup with an enduring level of professional integrity. The current study was conducted in research environments and, therefore, we here focus on research integrity. Research integrity is partially exemplified by fair and respectful leadership behaviors, respectful and supportive interpersonal interactions, and open and fair behaviors. As discussed above, these behaviors are fostered by a positive interpersonal climate emphasizing the importance of interpersonal respect and fairness, supportive leadership behaviors, and related institutional policies and practices.

Hypothesis 6: Interpersonal climate is strongly positively (>0.50) related to research integrity climate.

Scale evaluation

Participants

We invited 376 members from Sample 1 who had completed a previous survey related to this research and consented to be invited to participate in this follow-up survey (Sample 1c in Figure 1). Participation was voluntary and participants were assured of the confidentiality of the study. Similar to previous samples, we omitted 33 responses due to suspect or missing responses, resulting in a usable sample of 343 completed surveys (a 91% participation rate). The demographic breakdown was as follows: 50% female, 45% male, and 5% not answering; 0.9% American Indian or Alaskan Native, 4% Asian, 1% Black or African American, 3% Hispanic or Latino, 0.3% Native Hawaiian or Pacific Islander, 69% white, 6% not listed or multiracial, and 16% not answering; 20% students, 26% early career researchers, 26% mid-career researchers, 25% late career researchers, and 4% not answering.

Measures

In addition to the workgroup climate measure (CARES), research integrity climate measure (SOURCE), integration item, and continuance commitment measure, the following listed constructs were collected in this survey. Responses to all measures were scaled from 1 (Not at All) to 5 (Completely) with an additional “No Basis for Judging” option.

Organizational citizenship behaviors (OCB)

Participants rated the helping behaviors of members of their workgroup using the 5-items helping scale from Ehrhart (2004). An example item is “Members of my work unit help others who have heavy workloads” (α = 0.94, ω = 0.95).

Work unit diversity climate (WUD)

Participants rated the diversity climate in their workgroup using the 4-item work unit diversity scale (McKay et al., 2008). An example item is “Leaders in my work unit demonstrate a visible commitment to diversity” (α = 0.92, ω = 0.95).

Work unit experiences (WUE)

Participants rated their individual experiences in their workgroup using the 13-item work unit environment scale developed by Vicki Magley (personal communication), and which includes items from the Cortina et al., Workplace Incivility Scale (Cortina et al., 2001), two items from the Martin and Hine measure of uncivil workplace behavior (Martin and Hine, 2005), and question items assessing cohesion, inclusion, and citizenship generated de-novo by Magley or modified by her from existing items to have an experiential/behavioral framing. An example item is “During the past year, were you ever in a situation in which any of your work unit members showed you genuine concern and courtesy” (α = 0.92, ω = 0.95).

We further assessed the validity of the CARES instrument by examining the relationship between it and measures of constructs at both the individual and work-unit levels which are expected to correlate with work-unit climate. The means, standard deviations, and correlations for these scales can be found in Table 7. Supporting Hypothesis 1, workgroup members' feelings of integration in the workplace correlated moderately to strongly positively with interpersonal climate as measured by the CARES instrument (r = 0.55) and its dimensions (r = 0.35 to 0.51). Supporting Hypothesis 2, continuance commitment correlated moderately negatively with interpersonal climate as measured by the CARES instrument (r = −0.20) and its dimensions (r = −0.11 to −0.23). Supporting Hypothesis 3, OCB correlated moderately to strongly positively with interpersonal climate as measured by the CARES instrument (r = 0.58) and its dimensions (r = 0.21 to 0.60). Supporting Hypothesis 4, WUD correlated moderately to strongly positively with interpersonal climate as measured by the CARES instrument (r = 0.72) and its dimensions (r = 0.35 to 0.76). Supporting Hypothesis 5, WUE correlated moderately to strongly positively with interpersonal climate as measured by the CARES instrument (r = 0.66) and its dimensions (r = 0.26 to 0.73). Finally, supporting Hypothesis 6, research integrity in the workgroup correlated moderately to strongly positively with interpersonal climate as measured by the CARES instrument (r = 0.78) and its dimensions (r = 0.56 to 0.79).

Table 7

Table 7. Sample 2 means, standard deviations, and correlations among variables.

Discussion

Our goal in developing the CARES instrument is to provide leaders of organizations with a tool to support the effective promotion of high-quality work climates. It does so by providing organizational leaders the ability to collect reliable data to benchmark baseline conditions of working climates across organizational sub-units, identify well-functioning units, and identify those areas needing improvement. Such local and specific evidence can facilitate recognition of high-performing sub-units from which best practices can be gleaned and distributed throughout the organization. It can further inform and facilitate the development and deployment of appropriately targeted interventions for areas of concern. When leadership efforts to create organizational change are tailored to the specific concerns identified in local settings by organizational members, they are likely to be more effective than the more typical “blanket” policy edicts. The latter are typically and generically applied to all organizational members and units, are often perceived as unwanted bureaucratic compliance requirements, and thus regularly miss the mark, because they are not informed by the specific concerns of organizational members in specific organizational sub-units.

As a newly developed measure of interpersonal climates in organizations, the CARES instrument provides a novel tool that is useful to leaders in understanding dimensions of their organizational climates that are recognized as important to organizational and individual outcomes, productivity, wellbeing, and success. This detailed understanding of the climate at the level of the workgroup provides organizations with the ability to identify and respond to both exemplary and concerning administrative sub-units and to understand the climate within those units in a multi-dimensional manner.

As hypothesized, exploratory and confirmatory factor analyses identified multiple dimensions addressing unique but correlated aspects of interpersonal climates in organizations. Moreover, these four CARES dimensions of interpersonal climate correlate positively with important organizational outcomes such as workgroup integration, organizational citizenship behavior, work unit diversity climate, and work unit experiences, and negatively with important individual outcomes such as continuance commitment to one's employer. Further, we found moderate to strong correlations between work unit interpersonal climate and research integrity climate. Multi-group confirmatory factor analyses established the invariance of scale measures across a broad range of salient sub-groups. This speaks to questions of intersectionality, suggesting that different groups perceive and respond to the survey items using the same frames of reference, or cognitive schema.

Contributing to the strength and practicality of the CARES instrument is an extensive and broad-based initial literature review, our refinements of survey items to cover a broad spectrum of dimensions and features of interpersonal climates, our subsequent enlistment of subject matter experts to provide critical qualitative input that led to further item modifications, and our quantitative analyses with two independent samples for exploratory and confirmatory factor analyses. It is worth noting that, although the initial theorizing and development of the CARES instrument explicitly considered psychological safety as an important component to healthy interpersonal interactions, this aspect of interpersonal climate did not emerge from the data as a unique dimension. Instead, we observe that each of the dimensions which did emerge is likely to contribute in part to a perception of psychological safety and other beneficial proximal and distal climate outcomes among workgroup members. We encourage future research which examines additional outcomes of a comprehensive, interpersonal climate including perceptions of psychological safety.

We also view it as strength that our measures intentionally eschew the evaluation of organizational member attitudes and intentions, specifically because our theory of change does not require change in these factors for there to be changes in behavior, and in the organizational climate itself. It is our view that the most powerful and lasting source of organizational change is found in changing organizational climate which, in turn, shapes members attitudes, intentions, behaviors, and downstream outcomes such as individual and organizational health and performance. It is possible that changes in expectations of organizational members might itself lead to changes in attitudes, but it is not a requirement for there to be positive change. These strengths notwithstanding, there are also some limitations to this work that warrant mention.

Both financial and time limitations precluded our development and utilization of survey samples for this work that would have allowed us to cluster individual respondents within meaningful organizational sub-units (e.g., something like a two-stage, clustered sample, with individual respondents nested within work-units). This means that we have not been able here to assess the strength of climate measures in terms of perceptions of them being ^*shared^* by respondents within and across organizational sub-units. This also precluded our ability to assess whether shared perceptions of climate were more strongly correlated with measured outcomes than individual perceptions of them. While we remain confident in the importance of the quality of interpersonal organizational climates as factors influencing important organizational and individual outcomes, it is worth noting that such measures of climate clearly do not capture all aspects of interpersonal behavior that may be of interest and concern. Specifically, such tools may not always illuminate the existence of “bad-apple” individuals within a work-unit, particularly if that individual's behavior is targeted on one or a small number of individuals and not perceived or observed by others in the work-unit and has not yet risen to the level where it negatively impacts work-unit climate. As noted recently by Dumitrescu (2022), “We do not know how someone behaves toward those less powerful than they unless we catch them unaware, or unless their targets trust us enough to tell us.” In addition to the bad-apple issue, it is likely that climate surveys may miss identifying work-units that are so highly toxic that their members simply do not feel safe speaking up about the issues in any context, no matter guarantees of anonymity, for fear of being identified and retaliated against. While we believe the CARES is valid and useful in a broad range of organizational types, our work here has been confined to research oriented organizational settings. Further work is needed to examine utility of the CARES in a wide range of organizational settings. And finally, although a 22-item survey is a notably brief instrument in order to robustly capture a multidimensional understanding of workgroup climate, it is prudent to recognize that respondent burden is an ever-present concern, especially in organization-wide surveys of members. As such, future work should consider the feasibility of even more abbreviated versions of the CARES scale, either retaining its dimensionality or focusing on capturing a unidimensional measure of workgroup climate consistent with this current instrument.

One potential mechanism through which organizational climate is thought to influence organizational member behavior is its influence on intrinsic motivation. Gagné and Deci (2005) theorize that humans have a trio of fundamental psychological needs: experiencing competence, autonomy, and relatedness. Aspects of social context, including organizational climates in work settings, can support or undermine meeting these basic needs, primarily through whether these organizational features support or undermine intrinsic motivation of organizational members (Gagné and Deci, 2005). Intrinsic motivation has been shown to correlate positively with many desired domains in workplaces, such as perceived organizational support, optimism, job satisfaction, wellbeing and self-reported physical health, while correlating negatively with turnover intentions, and psychological distress (Gagné et al., 2010). If organizational climates undermine members' feelings of competence, autonomy, or relatedness, this in turn can threaten the development and continuance of individual intrinsic motivation. These ideas have been most fully articulated in self-determination theory (Deci et al., 2017; Deci and Ryan, 2000).

Importantly, these theoretical expectations are supported by extensive evidence documenting associations between negative interpersonal experiences (e.g., incivility, sexual harassment) and both individual and organizational outcomes. Such outcomes may include decreased job satisfaction (Lim and Cortina, 2005), worker motivation, commitment, and organizational citizenship behaviors (Johnson and Indvik, 2001), increased work withdrawal and turnover intentions (Cortina et al., 2001; Griffin, 2010; Miner-Rubino and Reed, 2010), reduced psychological wellbeing (Cortina et al., 2001; Johnson and Indvik, 2001; Lim and Lee, 2011), counterproductive work behavior (Penney and Spector, 2005) and work performance (Caza and Cortina, 2007; Porath and Erez, 2009).

Implications for use

The CARES assesses multiple dimensions of interpersonal climates in organizations, focused on civility, interpersonal accountability, harassment, and conflict resolution in organizational sub-units. Although intercorrelated (see Table 2), each of these dimensions is distinct, and it is inappropriate to use and treat the CARES as a unidimensional construct, nor is it developed or intended to be used as such. It is not meant to be a general-purpose sexual harassment survey, examples of which there are many, nor is it meant to serve as a DEI assessment tool, though there is some conceptual overlap with both of these types of instruments. In developing the CARES items, we also realized that some dimensions of concern in the interpersonal climates in organizations (e.g., sexual harassment responsiveness) are most frequently subject to policies and actions taken at organizational levels beyond the immediate work-unit, and that are, therefore, influenced by factors more properly seen as residing in the control of leaders of the larger institutional body. As such, we felt it important to retain items addressing these dimensions, even though they yield results that extend beyond the work-unit.

The survey tool is not an intervention per se, rather based on our experience with administering organizational climate surveys, we envision it as a means of surfacing issues of concern that may not be on the radar of organizational leaders yet warrant their attention proactively. Similarly, we envision it as a means of surfacing positive exemplars in an organization which should equally be brought to the attention of organizational leaders. We further expect that this instrument will be useful in assessing organizational change interventions, though its utility in this regard remains to be tested in future work. Our results here support the utility of the CARES survey across a broad range of disciplines, and types of organizations (including traditional academic research government, and industry settings).

Conclusion

The importance of climates to individual and organizational outcomes has long been recognized. The CARES instrument answers a critical need to more richly understand multiple dimensions of organizational climate—civility, interpersonal accountability, conflict resolution, and institutional harassment responsiveness—and to discern these aspects of climate at a level of meaningful action—that of the workgroup. This instrument has been developed and demonstrated to be robust across a rich variety of organizational members, providing organizations and their leaders confidence in a consistent and actionable measure of the climate which exists in each of their work-units. These attributes suggest that the CARES survey offers both researchers and practitioners a powerful new tool in the work of understanding and improving organizational climates.

Data availability statement

The CARES instrument has been licensed through the Creative Commons 4.0 license, as follows: This work is licensed under the creative commons attribution-noncommercial-noderivatives 4.0 international license. To view a copy of this license, visit http://creativecommons.Org/licenses/by-nc-nd/4.0/ or send a letter to creative commons, PO box 1866, mountain view, CA 94042, USA.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by University of Illinois at Urbana-Champaign Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The Ethics Committee/Institutional Review Board waived the requirement of written informed consent for participation from the participants or the participants' legal guardians/next of kin because human subject participation was limited to a survey that was not deemed to include sensitive information from respondents, and from which responses were never connected to direct individual identifiers. As such, the handling IRB deemed this research protocol as not requiring written informed consent.

Author contributions

BM: Conceptualization, Data curation, Investigation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing. JS: Conceptualization, Data curation, Formal analysis, Methodology, Validation, Writing – original draft, Writing – review & editing. VM: Conceptualization, Investigation, Methodology, Validation, Writing – review & editing. CT: Conceptualization, Data curation, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing. CG: Funding acquisition, Project administration, Resources, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This project has been supported by funding from the Alfred P. Sloan Foundation, via Grant #2019-12294; Co-PIs: Billy M. Williams (AGU) and C. K. Gunsalus, JD (UIUC—NCPRE). After approval of funding, and for the duration of the project funding period, the funder received semi-annual updates from our research team and our team received suggestions from their advisory body. The funder did not otherwise have direct input on the research after the funding decision was made.

Acknowledgments

We wish to acknowledge Billy M. Williams' excellent leadership of the overarching project of which the work presented here was one component.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Gen AI was used in the creation of this manuscript.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frma.2025.1516726/full#supplementary-material

Footnotes

1. ^The results for the full AGU data set are not appreciably different than those for the second split half reported here and are available upon request from the corresponding author.

2. ^Items in this dimension are asked in a negatively valanced fashion, assessing the extent to which interpersonal accountability climate is absent or lacking in their work-unit. Items are reverse coded for this dimension as it is presented in Tables 2 and 7, to reflect the presence of interpersonal accountability climate.

References

Allen, N. J., and Meyer, J. P. (1990). The measurement and antecedents of affective, continuance and normative commitment to the organization. J. Occup. Psychol. 63, 1–18. doi: 10.1111/j.2044-8325.1990.tb00506.x

PubMed Abstract | Crossref Full Text | Google Scholar

Andersson, L. M., and Pearson, C. M. (1999). Tit for tat? The spiraling effect of incivility in the workplace. Acad. Managem. Rev. 24, 452–471. JSTOR. doi: 10.2307/259136

Crossref Full Text | Google Scholar

Berdahl, J. L., and Bhattacharyya, B. (2021). Four ways forward in studying sex-based harassment. Equal. Divers. Inclus. doi: 10.1108/EDI-03-2021-0071

Crossref Full Text | Google Scholar

Berdahl, J. L., Cooper, M., Glick, P., Livingston, R. W., and Williams, J. C. (2018). Work as a masculinity contest. J. Soc. Issues 74, 422–448. doi: 10.1111/josi.12289

Crossref Full Text | Google Scholar

Blau, P. M. (1960). A theory of social integration. Am. J. Sociol. 65, 545–556. doi: 10.1086/222785

Crossref Full Text | Google Scholar

Bliese, P. D. (2000). “Within-group agreement, non-independence, and reliability: implications for data aggregation and analysis,” in Multilevel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New Directions (Hoboken, Nj: Jossey-Bass), 349–381.

Google Scholar

Bryer, A. (2020). Making organizations more inclusive: the work of belonging. Organiz. Stud. 41, 641–660. doi: 10.1177/0170840618814576

Crossref Full Text | Google Scholar

Cantor, D., Fisher, B., Chibnall, S. H., Bruce, C., Townsend, R., Thomas, G., et al. (2015). Report on the AAU Campus Climate Survey on Sexual Assault and Sexual Misconduct. Rockville: Westat, 302.

Google Scholar

Carr, J. Z., Schmidt, A. M., Ford, J. K., and DeShon, R. P. (2003). Climate perceptions matter: a meta-analytic path analysis relating molar climate, cognitive and affective states, and individual level work outcomes. J. Appl. Psychol. 88, 605–619. doi: 10.1037/0021-9010.88.4.605

PubMed Abstract | Crossref Full Text | Google Scholar

Caza, B. B., and Cortina, L. M. (2007). From insult to injury: Explaining the impact of incivility. Basic Appl. Soc. Psych. 29, 335–350. doi: 10.1080/01973530701665108

Crossref Full Text | Google Scholar

Cheung, G. W., and Rensvold, R. B. (2002). Evaluating goodness-of-hit indexes for testing measurement invariance. Struct. Equ. Model. 9, 233–255. doi: 10.1207/S15328007SEM0902_5

Crossref Full Text | Google Scholar

Cortina, L. M., Magley, V. J., Williams, J. H., and Langhout, R. D. (2001). Incivility in the workplace: incidence and impact. J. Occup. Health Psychol. 6, 64–80. doi: 10.1037/1076-8998.6.1.64

Crossref Full Text | Google Scholar

Crain, A. L., Martinson, B. C., and Thrush, C. R. (2013). Relationships between the survey of organizational research climate (SORC) and self-reported research practices. Sci. Eng. Ethics 19, 835–850. doi: 10.1007/s11948-012-9409-0

PubMed Abstract | Crossref Full Text | Google Scholar

Cross, R., Edmondson, A., and Murphy, W. (2019). “A noble purpose alone won't transform your company,” in MITSloan Managment Review, 61. Available at: https://mitsmr.com/2qBXmb2 (accessed November 29, 2021).

Google Scholar

Deci, E. L., Olafsen, A. H., and Ryan, R. M. (2017). Self-determination theory in work organizations: The state of a science. Annu. Rev. Organ. Psychol. Organ. Behav. 4, 19–43. doi: 10.1146/annurev-orgpsych-032516-113108

Crossref Full Text | Google Scholar

Deci, E. L., and Ryan, R. M. (2000). The “what” and “why” of goal pursuits: human needs and the self-determination of behavior. Psychol. Inq. 11, 227–268. doi: 10.1207/S15327965PLI1104_01

Crossref Full Text | Google Scholar

Dumitrescu, I. (2022). “The frenzied folly of professorial groupthink,” in The Chronicle of Higher Education. Available at: https://www.chronicle.com/article/the-frenzied-folly-of-professorial-groupthink

Google Scholar

Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Adm. Sci. Q. 44, 350–383. doi: 10.2307/2666999

Crossref Full Text | Google Scholar

Ehrhart, M. G. (2004). Leadership and procedural justice climate as antecedents of unit-level organizational citizenship behavior. Pers. Psychol. 57, 61–94. doi: 10.1111/j.1744-6570.2004.tb02484.x

Crossref Full Text | Google Scholar

Ehrhart, M. G., and Naumann, S. E. (2004). Organizational citizenship behavior in work groups: A group norms approach. J. Appl. Psychol. 89, 960–974. doi: 10.1037/0021-9010.89.6.960

PubMed Abstract | Crossref Full Text | Google Scholar

Ehrhart, M. G., Schneider, B., and Macey, W. H. (2013). Organizational Climate and Culture: An Introduction to Theory, Research, and Practice. London: Routledge.

Google Scholar

Feldblum, C. R., and Lipnic, V. A. (2016). “Select task force on the study of harassment in the workplace,” in Equal Employment Opportunity Commission. Available at: https://www.eeoc.gov/june-2016-report-co-chairs-select-task-force-study-harassment-workplace

Google Scholar

Frazier, M. L., Fainshmidt, S., Klinger, R. L., Pezeshkan, A., and Vracheva, V. (2017). Psychological safety: a meta-analytic review and extension. Pers. Psychol. 70, 113–165. doi: 10.1111/peps.12183

Crossref Full Text | Google Scholar

Frone, M. R. (2000). Interpersonal conflict at work and psychological outcomes: testing a model among young workers. J. Occup. Health Psychol. 5, 246–255. doi: 10.1037/1076-8998.5.2.246

PubMed Abstract | Crossref Full Text | Google Scholar

Gagné, M., and Deci, E. L. (2005). Self-determination theory and work motivation. J. Organ. Behav. 26, 331–362. doi: 10.1002/job.322

Crossref Full Text | Google Scholar

Gagné, M., Forest, J., Gilbert, M.-H., Aub,é, C., Morin, E., and Malorni, A. (2010). The motivation at work scale: validation evidence in two languages. Educ. Psychol. Meas. 70, 628–646. doi: 10.1177/0013164409355698

PubMed Abstract | Crossref Full Text | Google Scholar

Glick, P., Berdahl, J. L., and Alonso, N. M. (2018). Development and validation of the masculinity contest culture scale. J. Soc. Issues 74, 449–476. doi: 10.1111/josi.12280

PubMed Abstract | Crossref Full Text | Google Scholar

Griffin, B. (2010). Multilevel relationships between organizational-level incivility, justice and intention to stay. Work & Stress 24, 309–323. doi: 10.1080/02678373.2010.531186

Crossref Full Text | Google Scholar

Haven, T., Pasman, H. R., Widdershoven, G., Bouter, L., and Tijdink, J. (2020). Researchers' perceptions of a responsible research climate: a multi focus group study. Sci. Eng. Ethics 26, 3017–3036. doi: 10.1007/s11948-020-00256-8

PubMed Abstract | Crossref Full Text | Google Scholar

Haven, T. L., Bouter, L., Mennen, L., and Tijdink, J. K. (2021). Superb supervision: a pilot study on training supervisors to convey responsible research practices onto their phd students. ArXiv. doi: 10.31222/osf.io/dxyng

PubMed Abstract | Crossref Full Text | Google Scholar

Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organ. Res. Methods 1, 104–121. doi: 10.1177/109442819800100106

Crossref Full Text | Google Scholar

Hu, L., and Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria vs. new alternatives. Struct. Equ. Model. 6, 1–55. doi: 10.1080/10705519909540118

Crossref Full Text | Google Scholar

James, L. R., Demaree, R. G., and Wolf, G. (1993). An assessment of within-group interrater agreement. J. Appl. Psychol. 78, 306–309. doi: 10.1037/0021-9010.78.2.306

Crossref Full Text | Google Scholar

Johnson, P. R., and Indvik, J. (2001). Slings and arrows of rudeness: Incivility in the workplace. J. Managem. Dev. 20, 705–714. doi: 10.1108/EUM0000000005829

Crossref Full Text | Google Scholar

Kahn, W. A. (1990). Psychological conditions of personal engagement and disengagement at work. Acad. Managem. J. 33, 692–724. doi: 10.2307/256287

Crossref Full Text | Google Scholar

Kaiser, J. (2019). NIH prepares to toughen harassment rules. Science 364, 1117–1118. doi: 10.1126/science.364.6446.1117

PubMed Abstract | Crossref Full Text | Google Scholar

Kis, A., Tur, E. M., Lakens, D., Vaesen, K., and Houkes, W. (2022). Leaving academia: PhD attrition and unhealthy research environments. PLoS ONE 17:e0274976. doi: 10.1371/journal.pone.0274976

PubMed Abstract | Crossref Full Text | Google Scholar

Knapp, J. R., Smith, B. R., and Sprinkle, T. A. (2014). Clarifying the relational ties of organizational belonging: understanding the roles of perceived insider status, psychological ownership, and organizational identification. J. Leaders. Organiz. Stud. 21, 273–285. doi: 10.1177/1548051814529826

Crossref Full Text | Google Scholar

Komisarof, A. (2022). A new framework of workplace belonging: instrument validation and testing relationships to crucial acculturation outcomes. J. Int. Intercult. Commun. 15, 311–332. doi: 10.1080/17513057.2021.1897152

Crossref Full Text | Google Scholar

Liden, R. C., Wayne, S. J., and Sparrowe, R. T. (2000). An examination of the mediating role of psychological empowerment on the relations between the job, interpersonal relationships, and work outcomes. J. Appl. Psychol. 85, 407–416. doi: 10.1037/0021-9010.85.3.407

PubMed Abstract | Crossref Full Text | Google Scholar

Lim, S., and Cortina, L. M. (2005). Interpersonal mistreatment in the workplace: The interface and impact of general incivility and sexual harassment. J. Appl. Psychol. 90, 483–496. doi: 10.1037/0021-9010.90.3.483

PubMed Abstract | Crossref Full Text | Google Scholar

Lim, S., and Lee, A. (2011). Work and nonwork outcomes of workplace incivility: does family support help? J. Occup. Health Psychol. 16, 95–111. doi: 10.1037/a0021726

PubMed Abstract | Crossref Full Text | Google Scholar

Lovejoy, M., Kelly, E. L., Kubzansky, L. D., and Berkman, L. F. (2021). Work redesign for the 21st century: promising strategies for enhancing worker wellbeing. Am. J. Public Health 111, 1787–1795. doi: 10.2105/AJPH.2021.306283

PubMed Abstract | Crossref Full Text | Google Scholar

Luong, R., and Flake, J. K. (2023). Measurement invariance testing using confirmatory factor analysis and alignment optimization: A tutorial for transparent analysis planning and reporting. Psychol. Meth. doi: 10.1037/met0000441. [Epub ahead of print].

PubMed Abstract | Crossref Full Text | Google Scholar

Martin, R. J., and Hine, D. W. (2005). Development and validation of the uncivil workplace behavior questionnaire. J. Occup. Health Psychol. 10, 477–490. doi: 10.1037/1076-8998.10.4.477

PubMed Abstract | Crossref Full Text | Google Scholar

Martinson, B. C., Thrush, C. R., and Crain, A. L. (2013). Development and validation of the survey of organizational research climate (sorc). Sci. Eng. Ethics 19, 813–834. doi: 10.1007/s11948-012-9410-7

PubMed Abstract | Crossref Full Text | Google Scholar

McKay, P. F., Avery, D. R., and Morris, M. A. (2008). Mean racial-ethnic differences in employee sales performance: the moderating role of diversity climate. Pers. Psychol. 61, 349–374. doi: 10.1111/j.1744-6570.2008.00116.x

Crossref Full Text | Google Scholar

Mervis, J., and Kaiser, J. (2018). NSF issues sexual harassment policy as NIH promises action. Science 361, 1299–1300. doi: 10.1126/science.361.6409.1299

PubMed Abstract | Crossref Full Text | Google Scholar

Meyer, J. P., and Allen, N. J. (1984). Testing the “side-bet theory” of organizational commitment: Some methodological considerations. J. Appl. Psychol. 69, 372–378. doi: 10.1037/0021-9010.69.3.372

PubMed Abstract | Crossref Full Text | Google Scholar

Miner-Rubino, K., and Reed, W. D. (2010). Testing a moderated mediational model of workgroup incivility: The roles of organizational trust and group regard. J. Appl. Soc. Psychol. 40, 3148–3168. doi: 10.1111/j.1559-1816.2010.00695.x

Crossref Full Text | Google Scholar

NASEM (2018). Sexual Harassment of Women: Climate, Culture, and Consesquences in Academic Sciences, Engineering, and Medicine. Washington, DC: National Academies Press.

Google Scholar

Organ, D. W. (1988). Organizational Citizenship Behavior: The Good Soldier Syndrome. Lexington: Lexington Books/DC Heath and Com.

PubMed Abstract | Google Scholar

Organ, D. W. (1997). Organizational citizenship behavior: It's construct clean-up time. Human Perform. 10:85. doi: 10.1207/s15327043hup1002_2

Crossref Full Text | Google Scholar

Organ, D. W., and Konovsky, M. (1989). Cognitive vs. affective determinants of organizational citizenship behavior. J. Appl. Psychol. 74, 157–164. doi: 10.1037/0021-9010.74.1.157

PubMed Abstract | Crossref Full Text | Google Scholar

Pearson, C. M., Andersson, L. M., and Porath, C. L. (2000). Assessing and attacking workplace incivility. Organ. Dyn. 29, 123–137. doi: 10.1016/S0090-2616(00)00019-X

PubMed Abstract | Crossref Full Text | Google Scholar

Pearson, C. M., Andersson, L. M., and Wegner, J. W. (2001). When workers flout convention: a study of workplace incivility. Hum. Relat. 54, 1387–1419. doi: 10.1177/00187267015411001

Crossref Full Text | Google Scholar

Pearson, C. M., and Porath, C. L. (2005). On the nature, consequences and remedies of workplace incivility: No time for nice? Think again. Acad. Managem. Perspect. 19, 7–18. doi: 10.5465/ame.2005.15841946

Crossref Full Text | Google Scholar

Penney, L. M., and Spector, P. E. (2005). Job stress, incivility, and counterproductive work behavior (CWB): the moderating role of negative affectivity. J. Organ. Behav. 26, 777–796. doi: 10.1002/job.336

Crossref Full Text | Google Scholar

Porath, C. L., and Erez, A. (2009). Overlooked but not untouched: How rudeness reduces onlookers' performance on routine and creative tasks. Organ. Behav. Hum. Decis. Process. 109, 29–44. doi: 10.1016/j.obhdp.2009.01.003

Crossref Full Text | Google Scholar

Rahim, M. A., and Magner, N. R. (1995). Confirmatory factor analysis of the styles of handling interpersonal conflict: First-order factor model and its invariance across groups. J. Appl. Psychol. 80, 122–132. doi: 10.1037/0021-9010.80.1.122

PubMed Abstract | Crossref Full Text | Google Scholar

Reich, T. C., and Hershcovis, M. S. (2011). “Interpersonal relationships at work,” in APA Handbook of Industrial and Organizational Psychology, Vol 3: Maintaining, Expanding, and Contracting the Organization (Washington, D.C.: American Psychological Association), 223–248.

Google Scholar

Schneider, B., Ehrhart, M. G., and Macey, W. H. (2013). Organizational climate and culture. Annu. Rev. Psychol. 64, 361–388. doi: 10.1146/annurev-psych-113011-143809

PubMed Abstract | Crossref Full Text | Google Scholar

Settoon, R. P., Bennett, N., and Liden, R. C. (1996). Social exchange in organizations: Perceived organizational support, leader – member exchange, and employee reciprocity. J. Appl. Psychol. 81, 219–227. doi: 10.1037/0021-9010.81.3.219

Crossref Full Text | Google Scholar

Steenkamp, J.-B. E. M., and Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research 25, 78–90. doi: 10.1086/209528

Crossref Full Text | Google Scholar

Steinmetz, H. (2013). Analyzing observed composite differences across groups: Is partial measurement invariance enough? Methodology 9, 1–12. doi: 10.1027/1614-2241/a000049

Crossref Full Text | Google Scholar

Tepper, B. J. (2000). Consequences of abusive supervision. Acad. Managem. J. 43, 178–190. doi: 10.2307/1556375

Crossref Full Text | Google Scholar

Tu, Y., Lu, X., Choi, J. N., and Guo, W. (2019). Ethical leadership and team-level creativity: Mediation of psychological safety climate and moderation of supervisor support for creativity. J. Busin. Ethics 159, 551–565. doi: 10.1007/s10551-018-3839-9

Crossref Full Text | Google Scholar

Valantine, H. A., Le Fauve, C. E., Morris, K. A., and Riley, W. T. (2021). Ending sexual harassment in science: Designing and administering a survey that can lead to an improved organizational climate. Acad. Med. 97, 364–369. doi: 10.1097/ACM.0000000000004491

PubMed Abstract | Crossref Full Text | Google Scholar

Walsh, B. M., and Magley, V. J. (2019). Don't forget the role of civility interventions in workplace sexual harassment. Ind. Organ. Psychol. 12, 39–41. doi: 10.1017/iop.2019.5

Crossref Full Text | Google Scholar

Walsh, B. M., Magley, V. J., Reeves, D. W., Davies-Schrils, K. A., Marmet, M. D., and Gallus, J. A. (2011). Assessing workgroup norms for civility: the development of the civility norms questionnaire-brief. J. Bus. Psychol. 27, 407–420. doi: 10.1007/s10869-011-9251-4

Crossref Full Text | Google Scholar

Willness, C. R., Steel, P., and Lee, K. (2007). A meta-analysis of the antecedents and consequences of workplace sexual harassment. Pers. Psychol. 60, 127–162. doi: 10.1111/j.1744-6570.2007.00067.x

Crossref Full Text | Google Scholar

Appendix 1

CARES climate survey items

The prompt for the scale is “These questions are about the interpersonal climate of your primary work unit”. Response options include: 1 = “Not at All”, 2 = “Somewhat”, 3 = “Moderately”, 4 = “Very”, 5 = “Completely”. In addition, a “No Basis for Judging” response option is provided for all items.

Civility Climate (work-unit level)

1. How consistently do members of your work unit give appropriate credit for work effort and contributions?

2. How consistently do leaders in your work unit model respectful interpersonal behavior toward everyone, regardless of their identity or demographic characteristics?

3. Do members of your work unit value providing everyone equitable opportunities, regardless of their identity or demographic characteristics?

4. To what extent do leaders in your work unit create an environment where occasional errors are not a big deal?

5. To what extent are respectful interactions the norm in your work unit?

6. To what extent do leaders in your work unit create an environment that supports your personal and professional wellbeing?

7. To what extent are members of your work unit encouraging and supportive of each other?

Interpersonal Accountability Climate² (work-unit level)

8. To what extent could your work unit be characterized as hostile and intimidating?

9. Do people in your work unit get away with hostile and intimidating behaviors?

10. Do members of your work unit use their power to intimidate, coerce, or demean others?

11. Do leaders in your work unit allow or create an environment that ignores rude or uncivil behavior?

12. Are top performers in your work unit allowed to get away with bad behavior?

13. To what extent are subtle slights, insults, or disrespectful comments tolerated in your work unit?

14. To what extent do performance pressures in your work unit lead to unhealthy interpersonal behavior?

Conflict Resolution Climate (work-unit level)

15. To what extent are leaders in your work unit helpful in resolving interpersonal disputes between coworkers, when they arise?

16. Do leaders in your work unit facilitate effective processes for handling disputes, either formally or informally?

17. How much are interpersonal behaviors considered as a component of performance reviews?

Institutional Harassment Responsiveness (institution level)

18. In cases of sexual harassment, does your institution hold everyone to the same standard regardless of who is being accused, who is reporting, or the nature of the complaint?

19. Does your institution take sexual harassment seriously?

20. How confident are you that your institution would hold leaders accountable who allowed sexual harassment?

21. Effective processes to report hostile and intimidating behavior should be well-defined, accessible, and safe to use. How effective are such processes at your institution?

22. Does your institution take visible actions to prevent sexual harassment (e.g., going beyond things like required trainings)?