Development of a Work Climate Scale in Emergency Health Services

An adequate work climate fosters productivity in organizations and increases employee satisfaction. Workers in emergency health services (EHS) have an extremely high degree of responsibility and consequent stress. Therefore, it is essential to foster a good work climate in this context. Despite this, scales with a full study of their psychometric properties (i.e., validity evidence based on test content, internal structure and relations to other variables, and reliability) are not available to measure work climate in EHS specifically. For this reason, our objective was to develop a scale to measure the quality of work climates in EHS. We carried out three studies. In Study 1, we used a mixed-method approach to identify the latent conceptual structure of the construct work climate. Thus, we integrated the results found in (a) a previous study, where a content analysis of seven in-depth interviews obtained from EHS professionals in two hospitals in Gibraltar Countryside County was carried out; and (b) the factor analysis of the responses given by 113 EHS professionals from these same centers to 18 items that measured the work climate in health organizations. As a result, we obtained 56 items grouped into four factors (work satisfaction, productivity/achievement of aims, interpersonal relationships, and performance at work). In Study 2, we presented validity evidence based on test content through experts' judgment. Fourteen experts from the methodology and health fields evaluated the representativeness, utility, and feasibility of each of the 56 items with respect to their factor (theoretical dimension). Forty items met the inclusion criterion, which was to obtain an Osterlind index value greater than or equal to 0.5 in the three aspects assessed. In Study 3, 201 EHS professionals from the same centers completed the resulting 40-item scale. This new instrument produced validity evidence based on the internal structure in a second-order factor model with four components (RMSEA = 0.079, GFI = 0.97, AGFI = 0.97, CFI = 0.97; NFI = 0.95, and NNFI = 0.97); absence of Differential Item Functioning (DIF) in 80% of the items; reliability (α = 0.96); and validity evidence based on relations to other variables, specifically the test-criterion relationship (ρ = 0.680). Finally, we discuss further developments of the instrument and its possible implications for EHS workers.

Introduction

One of the main priorities of modern organizations is fostering a positive work climate because it promotes greater productivity, satisfaction, stability, and commitment to the organization (Lozano-Lozano et al., 2013; Lee et al., 2016; Meneghel et al., 2016). There are three main types of definitions for the work climate construct: (1) those based on objective and structural characteristics of organizations (Schneider et al., 2013), (2) those that emphasize individual psychological features (Zadow et al., 2017), and (3) those focused on both organizational and individual levels. This last perspective emphasizes workers' perception of the structure and processes occurring in work groups (Schulz et al., 2017). This work is based on the third group of definitions because it is the most frequently used and the most complete, as it considers both organizational and individual points of view.

Independent of the type of definition, there is no consensus on what the main components that form the construct work climate are. In this sense, we find that measurement instruments are based on various numbers of components as follows: three: clarity, support and challenge (Stringer, 2002; Perry et al., 2005) and job satisfaction, organizational commitment, and motivation to continue working (Zacher and Yang, 2016); four: authority, efficiency, innovation, and adaptation (Payne and Mansfield, 1978); five: communication, work conditions, job involvement, self-realization, and supervision (Torres and Zegarra, 2015) and culture, climate, burnout, engagement, and psychosomatization (Uribe-Prado et al., 2015); six: organizational clarity, rewards, decisions, leadership, social interaction, and opening (Gómez, 2004); seven: ability, recognition, internal organization, satisfaction, information received, knowledge of management, and goals and management responsiveness (García et al., 2010); eight: authority, motivation, communication, influence, decision, planning, control, and performance (Likert, 1967) and compensation and justice, teamwork, quality and effectiveness, communication, environmental sustainability, trust, security, and support (Zenteno and Durán, 2016) and relationships, management style, sense of belonging, remuneration, availability of resources, stability, clarity and consistency in management, and shared values (Fernández-Argüelles et al., 2015); nine: structure, responsibility, reward, challenge, relationships, cooperation, standards, conflict, and identity (Litwin and Stringer, 1968); ten: involvement, cohesion, support, autonomy, organization, pressure, clarity, control, innovation, and comfort (Moos and Insel, 1974); and twelve: conflict, equipment, failure, cohesion, autonomy, management, stress, cooperation, social life, lack of shared social life, logging, and marginality (Delgado-Sánchez et al., 2006) and relationship with the boss, work environment, desire for changes, work satisfaction, capacity for making decisions, tolerance, communication and support, opportunity for training, flexibility of schedules, satisfaction with benefits, resources, and stress (Rojas et al., 2011).

The disparity of the definitions of the construct work climate inspires the use of diverse instruments to measure it (e.g., Insel and Moos, 1974). Additionally, there are other factors that increase the variability of the instruments to measure work climate: (1) the different levels of specification of work climate, such as an overwork climate (Mazzetti et al., 2016) or competitive work group climate (Fletcher and Nusbaum, 2010); and (2) the adaptation of instruments to specific work contexts and professions, such as civil servants (Popoola, 2016).

In the specific area of health services, we found some consolidated models, such as that proposed by Perry et al. (2005), where a work climate is understood as a part of the organization mainly generated by individual human behavior and interactions. Each work team has its own climate defined as the quality of the internal environment experienced by its members that influences their behavior. It is said that a change in the work climate of an organization implies changes in other aspects in that organization, such as the effectiveness or the quality of the care given to patients (Castaneda and Scanlan, 2015; Safi et al., 2016).

When establishing models of work climate in health services into specific components, we find a huge disparity in aspects such as work satisfaction, which is understood as the feeling of contentment by health workers with their job; this aspect correlates negatively with absenteeism and professional abandonment (Mendoza-Llanos, 2015); interpersonal relationships (the way the different components of a work group relate), productivity (the degree to which the work group is able to achieve their professional goals efficiently) and performance of the health staff (performing daily tasks), components that are positively correlated (Brown and Calnan, 2016); or commitment (level of involvement) established with the patient to achieve their professional goals (Chiang et al., 2017).

Referring to the tools used to measure the work climate in health services, we find several different instruments (Elmi et al., 2017), some of which are specific to concrete professions, such as nursing (Olsen et al., 2017) or anesthesiology (Rama-Maceiras et al., 2012), and certain fields such as mental health (Ehrhart et al., 2014).

There are substantial differences in the characteristics of work in the various departments of hospitals. In many cases, emergency health services (EHS) establish initial contact with the patient (Gill et al., 2017) and serve a large number of patients (Hunt et al., 2006) who improperly use the service on some occasions (Carret et al., 2007) and, in general, have a high incidence of morbidity (Billings and Raven, 2013). EHS are usually structured as three main areas: (1) the emergency department, an area for the triage and prioritization of care needs, observation and assessment of the patient, and referral to other hospital services (Godoi et al., 2016); (2) the intensive care unit, which usually has an isolation zone where healthcare professionals care for patients who, due to serious health problems, require critical care (Leung and Gomersall, 2016); and (3) the emergency ambulance service, which is a mobile unit where the emergency has occurred and, in some cases, avoids transport to the overloaded emergency departments (Wankhade and Mackway-Jones, 2015). Patients who receive care in the EHS are in extremely serious condition, so workers suffer from a high degree of responsibility and consequent stress (Estryn-Behar et al., 2011). They are subject to a number of external pressures, such as the need for short waiting times, and they experience detrimental impacts to their quality of life (including their own health) due to shift work (Vedaa et al., 2016), even occasionally suffering from posttraumatic stress disorder (Arora et al., 2013; Bragard et al., 2015) because of their daily experience with situations requiring critical decisions in a matter of seconds (Borg Xuereb et al., 2016). It has been verified that emergency personnel work with higher levels of stress than other health professionals and that they are a unique population with greater autonomy (Johnston et al., 2016). In this context, it is necessary to design an instrument adapted to these specific characteristics.

On the one hand, an adequate work climate in EHS allows workers to successfully share the common purpose and responsibility of professional teams, where each member clearly understands their role and combines their skills and knowledge to provide better care for their patients (Ajeigbe et al., 2013). On the other hand, an inadequate work climate in this context can cause significant stress in professionals (Laposa et al., 2003) and personal dissatisfaction, which can have an impact on the quality of health care provided (Hooper et al., 2010), increase the perception of fatigue and distress at work (Adriaenssens et al., 2011), and affect labor productivity (Engelen et al., 2016).

Measuring a work climate would be useful as a first step before acting to improve it to detect weaknesses in the work group that need strengthening. Additionally, in an indirect way, the work climate affects other important variables, such as alcohol consumption in workers (Carreño et al., 2006); safety climate (Sexton et al., 2016); motivation (Li et al., 2016); patient outcomes and nurses' occupational health (Taylor et al., 2011); and job involvement, effort, and performance (Brown and Leigh, 1996).

Some scales to measure work climate in EHS are already available in the literature (Davenport et al., 2007). However, such scales show poor psychometric properties and present only results about their reliability and, in some cases, validity evidence based on relations to other variables (Biggs et al., 2016). We also found proposals for scales that, in addition to reliability, provide data regarding validity evidence based on internal structure, such as the Safety Attitudes Questionnaire (Sexton et al., 2016) applied in different contexts (Patterson et al., 2010), but they do not provide data about validity evidence based on test content or relations to other variables. Other scales applied to EHS propose to measure something other than work climate, such as the Perceptions of Safety Climate and Adherence to Safe Work Practices (Eliseo et al., 2012), which aims to measure safety perception in emergency room conditions, or those designed exclusively to measure the work environment for nursing staff (Swiger et al., 2017).

Due to the poor consensus in the definition and measure of work climate and due to the fact that there is no specific instrument to measure this construct in EHS with adequate psychometric properties, the aim of this study was to develop an instrument to measure the quality of the work climate in EHS specifically. For this purpose, we carried out our research in three stages (American Educational Research Association et al., 2014): (1) we identified the latent conceptual structure of work climate using a mixed-method approach based on the information obtained from in-depth interviews using the grounded theory (Lozano-Lozano et al., 2013) and factor analysis; (2) we presented validity evidence of the resultant instrument based on the test content through experts' judgment; and (3) we assessed the psychometric properties of the final version of the instrument: we carried out studies on reliability; validity evidence based on internal structure, specifically Confirmatory Factor Analysis (CFA) and Differential Item Functioning (DIF); and validity evidence based on relations to other variables, specifically test-criterion relationships.

Study 1: Identification of the Latent Conceptual Structure of Work Climate: A Mixed-Method Approach

Method

Participants

One hundred thirteen EHS workers from two different hospitals in Gibraltar Countryside County, chosen by incidental sampling, participated voluntarily. The inclusion criteria for the sample selection integrated (a) currently working in EHS when the study was carried out, and (b) having worked in EHS for at least 6 months. In total, 59.2% were women and 40.8% were men (age: M = 37.68 and SD = 8.79). Fifty percent were nurses, 22.1% were nursing assistants, 21.2% were doctors, 5.3% were orderlies, and 1.4% were administrative officers; 59.3% worked in the emergency department, 37.2% in the critical care unit and 3.5% in the emergency ambulance service. Concerning the type of contract, 36.1% had an indefinite contract, 34.1% had an interim contract, 27.8% had a temporary contract, and 2% had a contract for work and services. They had a mean of 12.83 years of experience in their profession (SD = 7.97); 51% had more than 5 years of experience in their profession, 21.1% had 2–5 years, and 23.8% had <2 years of experience. They had a mean of 9.38 years of experience in the current workplace (SD = 8.48); 52.4% had more than 5 years of experience, 28.8% had 2–5 years of experience, and 18.8% <2 years of experience.

Instruments

We used a list of 38 items to measure work climate in EHS specifically, which were obtained from a previous study (Lozano-Lozano et al., 2013). To produce this list, all EHS workers in the two hospitals in Gibraltar Countryside County (including doctors, nurses, nursing assistants, orderlies, administrative officers, security stuff and caretakers) were asked to voluntarily participate in answering in-depth individual interviews related to aspects such as the operation of the service, the organization, job satisfaction, the needs of the service, communication, productivity, relationship with authorities, conflicts and their resolutions, innovation and training. Eighteen EHS professionals responded: 9 doctors, 5 nurses, and 4 nursing assistants. After excluding the interviews that did not provide additional information to the built category system, the sample was formed by four doctors and three nurses. The inductive-deductive process proposed by Strauss and Corbin (1998) was followed to extract the information gathered from these interviews. Supplementary Table 1 presents the list of 38 items and their origins (codes and coding families).

Additionally, the questionnaire we used to measure the work climate in EHS was formed by 18 items rated from 1 (not at all) to 5 (to a great degree) on a five-point rating scale, available in Perry et al. (2005) to measure work climate in organizations in general, which was translated into Spanish according to the International Test Commission Guidelines for translating and adapting tests (International Test Commission, 2005; Barbero-García et al., 2008). Specifically, the following back-translation method was applied to the original English version: (a) the original version was translated into Spanish by a bilingual three-component expert group, (b) the new version was again translated into English by another bilingual translator who was not among those who formed the expert group in the first stage, and (c) the discrepancies that arose were discussed and the appropriate corrections of the new version were made. Table 1 presents the items in English. The Spanish translation is available in Supplementary Data 1. We chose these specific items for three main reasons (Perry et al., 2005): (1) the authors based their construction on a consolidated theoretical model (Stringer, 2002); (2) they presented evidence of adequate reliability, construct validity, and invariance across gender, management status, and educational level; and (3) although they are context independent, their validation was carried out in a public health organization (similar to our context, the EHS).

TABLE 1

Table 1. Factor weights of the questionnaire to measure work climate (obtained from Perry et al., 2005).

Procedure

Survey methodology

We contacted the management of the EHS of Gibraltar Countryside County (two different hospitals) and presented the reasons for the research and the characteristics of the study. Once we obtained the corresponding authorization, the 18-item questionnaire obtained from Perry et al. (2005) was administered to the staff of the EHS (250 participants). To ensure anonymity, the workers completed the questionnaire in a room in the hospital and, once finished, they deposited it in a ballot box.

With the data gathered, we carried out an exploratory factor analysis (EFA) (Pérez-Gil et al., 2000). SPSS 24 was used to store the data; PRELIS and LISREL 9.2 were used to carry out the data analysis. First, we created a polychoric correlation matrix between all of the variables that came into the analysis (Holgado-Tello et al., 2010). Second, we checked whether such a matrix met the assumptions necessary to be able to develop an EFA (Yela, 1957) by calculating the Kaiser-Meyer-Olkin (KMO) test and Bartlett's test of sphericity. Third, we performed an EFA using a principal components factor extraction and Kaiser's varimax method for orthogonal rotation.

Mixed-method approach

With the survey methodology, we calculated the factor structure of the 18 items obtained from Perry et al. (2005) in a specific EHS context. To obtain more detailed information about the important aspects that form a work climate in this specific context, we integrated these 18 items considering their factor structure and the 38 items obtained from in-depth interviews (Lozano-Lozano et al., 2013) considering their coding families.

The reliability of the assignment item—factor was checked using SPSS 24 in two ways: (a) by intracoder, where one author, JALL, completed the assignment and repeated the same task after 1 month; and (b) by intercoder, where two different authors, SCM and SSC, independently performed the same task. In both cases, Cohen's kappa (κ) coefficient was calculated.

Results

Survey Methodology

Of the potential 250 participants, 113 answered, which is a participation rate of 45.2%. Assumptions to develop an EFA were accepted: the KMO was 0.851, and Bartlett's test of sphericity resulted in ${\chi }_{(153)}^2$ = 804.174, p < 0.001.

The EFA provided a four-factor solution that explained 61.33% of the common variance. Factor 1 (F1) Work satisfaction produced an eigenvalue of 6.57 and explained 36.54% of the common variance. F2 Productivity/achievement of aims produced an eigenvalue of 1.93 and explained 10.79% of the common variance. F3 Interpersonal relationships produced an eigenvalue of 1.34 and explained 7.44% of the common variance. Finally, F4 Performance at work produced an eigenvalue of 1.17 and explained 5.47% of the common variance.

Table 1 shows the highest factor loading for each item. All values were higher than 0.48.

Mixed-Method Approach

Table 2 presents the scale used to measure the work climate in EHS after combining the results obtained with the in-depth interviews and the survey methodology. The assignment item—factor produced an intracoder reliability κ of 0.922 and an intercoder reliability κ of 0.879. Both results were considered adequate.

TABLE 2

Table 2. Resultant work climate scale in emergency health services after finishing the mixed-method approach (Study 1, in-depth interviews—survey) and Osterlind indexes obtained in Study 2, validity evidence based on test content.

F1 Work satisfaction refers to feelings evoked in workers by their job and their conditions: self-confidence due to the experience, the adequacy of the workday or time for each patient, contentment with relationships with other professionals outside the group and patients and their relatives, pride, success, cohesion, or nervousness facing new circumstances. F1 is formed using 15 items, six from the survey method (items 1–6) and nine from the in-depth interviews (items 7–15, corresponding to items 1–9 in Supplementary Table 1).

F2 Productivity/achievement of aims refers to the perception of workers having everything they need to do their job or, on the contrary, lacking what they need to achieve their goals: understanding of the relevance, capabilities, and specialization of others; the value of working in a group; motivation and fulfillment of expectations; recognition of their work as a group; self-improvement; infrastructure; training and the characteristics and functioning of their service; patients' characteristics fitting with their specialization and knowledge of such characteristics; protocols; and coordination with other hospital services. F2 is formed by 20 items, four from the survey method (items 16–19) and 16 from the in-depth interviews (items 20–35, corresponding to items 10–25 in Supplementary Table 1).

F3 Interpersonal relationships refer to the feelings when workers relate to other members of the group and aspects that influence such feelings: the quality of the communication, their relationship, the level of comfort, their friendship, their conflicts, being recognized for their individual contributions, having the resources they need, following a plan, participating in decision-making, and productivity. Thirteen items, five from the survey method (items 36–40) and eight from the in-depth interviews (items 41–48, corresponding to items 26–33 in Supplementary Table 1) form F3.

Finally, F4 Performance at work includes everything related to the development of workers' job placement: the perceived importance of their job and capacity to decide how to improve their performance; the skills and knowledge they use; and their knowledge of their tasks and others' tasks, their individual and group limitations, and their patients. F4 is formed by eight items, three from the survey method (items 49–51) and five from the in-depth interviews (items 52–56, corresponding to items 34–38 in Supplementary Table 1).

Study 2: Validity Evidence of the Resultant Instrument Based on Test Content

Method

Participants

We requested 27 experts' collaboration. The inclusion criterion was to have more than 3 years of experience in social and health sciences methodology and/or in EHS as a health professional. Fourteen experts answered, which is considered a moderate number of participants (10 ≤ N ≤ 30) for a content validity study (Prieto and Muñiz, 2000).

Eight experts (57.1%) were men and six (42.9%) were women. Their mean age was 48.83 (SD = 8.99). Regarding their professions, nine (64.3%) were professors specializing in methodology, design, psychometrics and/or data analysis, four (28.6%) were physicians from EHS, and one (7.1%) was a senior technician in clinical analysis and emergency and intensive care medicine. They were in their professions a mean of 23.67 years (SD = 9.46).

Instruments

The questionnaire used to obtain validity evidence based on test content through experts' judgment was composed of the 56 items obtained in the previous mixed-method study ordered in the four factors or dimensions found (see Supplementary Data 2). Each item presented three five-point Likert scales (Sanduvete-Chaves et al., 2013) to measure their representativeness (R), utility (U), and feasibility (F) (Chacón-Moscoso et al., 2016). Additionally, there was a final open-format question to receive comments for improvements for the proposed scale. Supplementary Data 3 presents the Spanish version of the questionnaire used to obtain validity evidence based on test content, which was completed by native Spanish speakers.

Procedure

The questionnaire to obtain validity evidence based on test content was sent by e-mail to 27 experts. After the third request, a total of 14 experts gave their answers. Anonymity was assured.

The Osterlind index of congruence (1998) was used to quantify the consensus between experts regarding the adequacy item—factor (theoretical dimension) (Glück et al., 2013). The formula used was

$\begin{array}{l}{I}_{ik}=\frac{(N-1){\displaystyle \sum _{j=1}^n}{X}_{ijk}+N{\displaystyle \sum _{j=1}^n}{X}_{ijk}-{\displaystyle \sum _{j=1}^n}{X}_{ijk}}{2(N-1)n}\end{array}$

where N = the number of dimensions of the instrument; X_ijk = each score given by each expert to each item referring to each aspect (R, U, and F); and n = the number of experts. The results could be from −1 to +1, 0 being the highest possible level of disagreement between experts. The inclusion criterion was to produce at least 0.5 (Osterlind, 1998) in the three aspects evaluated (R, U, and F).

Results

Considering that we requested assessments from 27 experts and 14 answered, we obtained a 51.9% rate of participation.

Forty items met the inclusion criterion; 16 did not meet it. Table 2 presents the Osterlind indexes for each item and each aspect (R, U, and F). The most frequently undervalued aspect was F: from the 16 items excluded, 14 were rated under 0.5 in this aspect.

Additionally, several experts proposed a reorganization of some items with respect to its factor assignment through the open-format question: four and three experts, respectively, suggested including item 49, referring to the perceived importance of their work, and item 50, referring to the development of their skills and knowledge at work, in F1 Work satisfaction instead of F4 Performance at work. Four experts suggested including item 2, referring to the quality of work and item 3, referring to the existence of a common purpose, in F2 Productivity/achievement of aims instead of F1. Two experts proposed including item 51, referring to the knowledge of what is expected in their work, in F2 instead of F4 Performance at work; and vice versa, five experts proposed including item 17, referring to the understanding of each other's capabilities, in F4, instead of F2.

The four authors of this work considered these proposals individually. After debating, we accepted all the suggestions given by consensus, based on the adequacy item—factor (dimension), from a substantive point of view.

Study 3: Reliability and Validity Evidence Based on Internal Structure and Relations to Other Variables

Method

Participants

Two hundred and one EHS professionals from the same two hospitals in Study 1 participated voluntarily. The inclusion criteria for the sample selection included (a) working in EHS when the study was carried out, and (b) having worked in EHS for at least 6 months. In total, 61.7% were women and 38.3% were men (age: M = 41.6 and SD = 10.23). Of these, 40.3% were nurses, 33.3% were doctors, 22.4% were nursing assistants, and 4% were orderlies. Sixty-seven percent worked in the emergency department, 30.5% to the critical care unit, and 2.5% to the emergency ambulance service; 44.6% had a temporary contract, 26.3% had an indefinite contract, 17.1% had an interim contract, 10.9% were residents, and 1.1% had a contract for work and services. Regarding their years of experience in their profession, they had a M of 15.12 (SD = 9.64); 49.2% had more than 5 years in their profession, 22.4% had 2–5 years of experience, and 28.4% had <2 years of experience. Regarding the number of years in their current workplace, the mean was 8.52 (SD = 8.31); 81.6% had more than 5 years of experience, 13.8% had 2–5 years of experience, and 4.6% had <2 years of experience.

Instruments

The scale used was formed via instructions for participants and the items that met the inclusion criterion in Study 2 (content validity), grouped into factors per expert suggestion (see Supplementary Data 4 for the English version and Supplementary Data 5 for the Spanish version). Each item was valued from 1 (strongly disagree) to 5 (strongly agree).

Furthermore, we added one extra omnibus item: As a whole, the work climate of my work group is good; in the Spanish version, De manera global, el clima laboral de mi grupo de trabajo es bueno; translation carried out according to the International Test Commission recommendations (International Test Commission, 2005; Barbero-García et al., 2008), which was also valued from 1 (strongly disagree) to 5 (strongly agree) and was used as a criterion in Study 3. This item was used as a criterion based on the following reasons (Holgado-Tello et al., 2015): (a) it can be considered an appropriate direct measure to relate to another indirect measure (the resulting scale) in the sense that both refer to the same construct (work climate in EHS); (b) the Likert-type scale of the item permits obtaining a linear monotonic function with respect to the attitude measured as the item characteristic curve, and individual differences in participants' attitude provoke variation in responses; and (c) no instrument with its psychometric properties tested that measures work climate in EHS was available as a criterion.

Procedure

The information was gathered using Google for Work applications: Drive, Forms, and Spreadsheets. The scale was administered by two procedures: (a) using a laptop with internet access in the hospital and (b) sending emails to all the EHS professionals with the link to the scale; in this case, they answered outside the work context.

SPSS 24 was used to store the data and calculate the internal consistency of the test, the average discrimination index, and validity evidence based on a test-criterion relationship and needed assumption tests. PRELIS and LISREL 9.2 were used to estimate the polychoric correlation matrix to verify bivariate normality and to carry out the CFA.

The internal consistency of the items was calculated using Cronbach's alpha coefficient, following criteria established by George and Mallery (2003). Values > 0.9 were considered excellent, 0.8–0.9 good, and 0.7–0.8 acceptable. Following criteria by Tavakol and Dennick (2011), values equal to or higher than 0.7 were considered appropriate.

The average discrimination index was also calculated: values >0.4 were considered excellent (Sabri, 2013), 0.3–0.4 good, and 0.2–0.3 adequate (Barbero-García, 1993). Obtaining adequate values in internal consistency and on the average discrimination index was considered an essential requirement before proceeding to study the factor structure.

To evaluate the factor structure of the scale, we estimated the polychoric correlations and the asymptotic variance-covariance matrix. A Pearson correlation matrix was not estimated because the items were in ordinal scales, and therefore, the responses cannot be treated as if they were quantitative because all participants situated at different points of the interval may be assigned the same score; the use of Pearson correlations for ordinal scales would undervalue the real correlations (Holgado-Tello et al., 2010). In these cases, polychoric correlations are the most consistent and robust estimator (Morata-Ramírez and Holgado-Tello, 2013).

The use of the polychoric correlation matrix is only appropriate if we previously accept the assumption of bivariate normal distribution. For this purpose, we calculated the chi-square test (χ²) and the percentage of tests that rejected the null hypothesis of bivariate normality for each pair of correlations, assuming a 95% confidence level and the Bonferroni correction, calculating the value of α to use in the comparison of each contrast with the formula α/c (α = 0.05 corresponding to a 95% confidence level and c as the number of contrasts [c = (number of items x number of items – 1)/2]. Due to the sensitivity of χ² in large samples, we also calculated the root mean square error of approximation (RMSEA). We concluded that the parameter estimation was not significantly affected when RMSEA values did not exceed 0.1 (Hooper et al., 2008).

Once we tested the assumption of bivariate normal distribution, we then tested the resultant model from Studies 1 and 2 using a CFA (Bagozzi and Yi, 2012). The tested model was a second-order factor model that measures work climate in EHS formed by four factors (Table 2): F1 Work satisfaction (items 1–10); F2 Productivity/achievement of aims (items 11–30); F3 Interpersonal relationships (items 31–35); and F4 Performance at work (items 36–40).

The estimation method used was the unweighted least squares, which is appropriate for polychoric correlations and ordinal variables distributed asymmetrically (Jöreskog, 2003; Morata-Ramírez et al., 2015).

The lambda parameter corresponding to the relationship of the first item with each factor was fixed at 1 to (a) solve the problem of identification of the model and (b) establish the measurement scale of the latent variables.

The standardized factor loadings were calculated. Additionally, several fit indices were used to reach conclusions about the adequacy of the model: (a) the χ² test, where the acceptance of the null hypothesis (p ≥ 0.05) implied a good fit of the model; (b) the consistent Akaike information criterion (CAIC) with which the model was considered appropriate when the value of the index was closer to the value for the saturated model than the independent one (the smaller the values, the better the fit) (Bandalos, 1993); (c) the root mean square error of approximation (RMSEA) (Hooper et al., 2008), where values lower than 0.05 were considered a good fit, values between 0.08 and 0.1 a reasonable fit, and values greater than 0.1 unfit (Browne et al., 1993); (d) the goodness-of-fit index (GFI); (e) the adjusted goodness-of-fit index (AGFI) (Hooper et al., 2008); (f) the comparative fit index (CFI) (Byrne, 1998); (g) the normed fit index (NFI); and (h) the non-normed fit index (NNFI) (Hoe, 2008). Indices (d)–(h) were interpreted as indicators of good fit if the values were above 0.9 (Bendayan et al., 2013).

We also studied DIF to obtain additional validity evidence based on the internal structure of the scale. EASY-DIF (González et al., 2011) was used to perform the Mantel-Haenszel procedure for ordinal items. The DIF was calculated by gender; seniority at work (0–15 years and 16–40 years of experience); age (18–42 and 43–65 years old); and type of employment relationship (eventual and permanent). The matching method used was the minimum cell frequency.

Additionally, we calculated the validity evidence based on the test-criterion relationship, correlating the global score (X, the sum of the scores given in the 40 items) and those obtained in each factor with the score given in the omnibus item 41: As a whole, the work climate of my work group is good (criterion Y) (Holgado-Tello et al., 2015). Previously, we tested several assumptions to check that the use of the Pearson correlation coefficient (r), a parametric test, was adequate: (a) normality using the Kolmogorov Smirnov test, where p > 0.05 implied the acceptance of the assumption (Chakravarti et al., 1967); (b) a test of linearity, where linearity p < 0.05 implied the acceptance of the assumption (Field, 2000); and (c) independence of errors using Durbin Watson (d), a statistic that ranged from 0 to 4 and was expected to produce a value close to 2; thus, values between 1.5 and 2.5 implied acceptance of the assumption (Kutner et al., 2004). If one of the three assumptions is rejected, we would opt for a non-parametric correlation test (Spearman's correlation coefficient, ρ). Both the Pearson and Spearman would be interpreted as validity evidence based on a test-criterion relationship when a p < 0.05 showed a statistically significant relationship between the global scores and Y.

Results

Participation

From the potential 250 participants, 201 completed the scale, producing a participation percentage of 80.4%; in particular, 199 (79.6%) were recorded in the hospital, and two (0.8%) by email outside the work context.

Internal Consistency

The global internal consistency was excellent (α = 0.96). Factors also produced appropriate values, all over 0.7; F1 produced a good result (α = 0.848); F2 and F3 produced excellent results (α = 0.943 and 0.907, respectively); and F4 produced acceptable, close to good results (α = 0.791).

Average Discrimination Index

The global average discrimination index produced an excellent result (D = 0.601), as did the specific index for each of the four factors (0.515, 0.646, 0.654, and 0.519, respectively). After testing the appropriateness of the internal consistency and the average discrimination index, we concluded that we could carry out the factor structure study.

Bivariate Normality Assumption

By having 40 items, a total of 780 correlations were obtained (40 × 39/2). The results showed that a bivariate normality assumption considering χ² was accepted in 95.3% of the instances (743 correlations) (p = 0.05/780 = 0.00006 using the Bonferroni correction). Additionally, the RMSEA values were lower than 0.1 in 98.5% of occasions (768 correlations). These results support the use of the matrix of polychoric correlations as the basis for the factor analyses.

Standardized Factor Loadings

The standardized factor loadings (lambda) in the CFA (Table 3) were appropriate (over 0.3) for all of the items. The gamma values were high for the four factors (0.89, 0.81, 0.82, and 0.86, respectively).

TABLE 3

Table 3. Standardized factors loadings obtained in the CFA.

Model Fit

Although the χ² test was significant, probably due to the large sample size effect, ${\chi }_{(730)}^2$ = 3807.37, p < 0.001, the other fit indexes showed the adequacy of the second-order-four-factor model: CAIC = 2460.93 (saturated CAIC = 5347.60; independent CAIC = 39115.98); RMSEA = 0.079, 90% CI [0.075, 0.084]; GFI = 0.97; AGFI = 0.97; CFI = 0.97; NFI = 0.95; and NNFI = 0.97.

DIF

Thirty-two items (80% of the total) did not present any DIF; the 8 items that presented some DIF were items 2, 3, 5, 6, 18, 23, 27, and 29. For the variable gender, the items with DIF were items 3 (χ² = 4.38, p = 0.03), 6 (χ² = 8.3, p < 0.001), 23 (χ² = 6.44, p = 0.01), 27 (χ² = 4.73, p = 0.02), and 29 (χ² = 6.95, p < 0.001). In items 3, 6 and 23, women scored higher than men, while in items 27 and 29, men scored higher than women. In seniority at work, the workers with more years in the company scored higher in items 2 (χ² = 4.28, p = 0.03) and 5 (χ² = 6.82, p < 0.001), while workers with fewer years of experience working scored higher in item 18 (χ² = 7.27, p < 0.001). For the variable age, older workers scored higher in item 5 (χ² = 6.07, p = 0.01) and item 6 (χ² = 5.74, p = 0.01), while younger people scored higher in item 18 (χ² = 4.47, p = 0.03). Finally, for type of employment relationship, workers with permanent contracts scored higher in items 5 (χ² = 4.94, p = 0.02) and 6 (χ² = 6. 78, p < .001), while workers with eventual contracts scored higher in item 18 (χ² = 6.19, p = 0.01).

Testing Assumptions before the Criterion Validity Correlation

The normality assumption was accepted on the global scale (Z = 0.607, p = 0.855), in F1 (Z = 1.329; p = 0.058) and in F2 (Z = 0.693, p = 0.722), while it was rejected in F3 (Z = 1.729, p = 0.005), F4 (Z = 1.409, p = 0.038), and the criterion (Z = 3.047, p < 0.001). The linearity assumption was accepted in all instances when relating with Y, X [F_{(1, 105)} = 187.569, p ≤ 0.001]; F1 [F_{(1, 168)} = 88.717, p ≤ 0.001]; F2 [F_{(1, 138)} = 146.957, p < 0.001]; F3 [F_{(1, 177)} = 141,197, p ≤ 0.001]; and F4 [F_{(1, 177)} = 41,335, p < 0.001]. Finally, the independence of errors was accepted in all occasions for the relationship of Y with X (d = 1.608), F1 (d = 1.654), F2 (d = 1.634), F3 (d = 1.792), and F4 (d = 1.761). As a conclusion, although most assumptions were accepted, because there was no normal distribution in Y, the non-parametric test (ρ) was used to study the relationship between Y and X, and Y and the sum of the different factors.

Validity Evidence Based on a Test-Criterion Relationship

Validity evidence based on a test-criterion relationship for the whole scale presented an adequate result, ρ_XY = 0.68, p < 0.001. The relationships between the sum of scores in the different factors and Y also yielded adequate results with F1 (ρ = 0.560, p < 0.001), F2 (ρ = 0.632, p < 0.001), F3 (ρ = 0.658, p < 0.001), and F4 (ρ = 0.430, p < 0.001).

Discussion

Based on the non-consensus of the main components that form the construct work climate and the lack of an instrument to measure it, by being applied to EHS specifically and with adequate psychometric properties, we elaborated a scale consisting of four dimensions (work satisfaction, productivity/achievement of aims, interpersonal relationships, and performance at work) and 40 items, which presented evidence of reliability as well as validity based on test content, internal structure and relations to other variables. Supplementary Data 4, 5 present the final version of the scale ready to be used by those who are interested in the English and Spanish versions, respectively.

Thus, as a result, Study 1 (a mixed-method approach) produced 56 items grouped into the four previously mentioned components. Study 2 (validity evidence based on test content through experts' opinion) permitted refinement of the scale in accordance with the representativeness, utility, and feasibility of its items (16 were removed as a result), and six were reorganized into different factors. Finally, Study 3 (psychometric properties) presented a second-order factor model with the four previously mentioned components with adequate values for reliability and validity evidence based on internal structure and test-criterion relationships.

The four factors obtained have certain similarities to those obtained in previous studies: (a) Work satisfaction (F1) is one of the factors that form the construct work climate in models by García et al. (2010), Rojas et al. (2011), and Zacher and Yang (2016); (b) Productivity/achievement of aims (F2) appears in Payne and Mansfield (1978) with the label of efficiency and in Brown and Calnan (2016); (c) Interpersonal relationships (F3) is also a factor in the proposals given by Fernández-Argüelles et al. (2015) and Litwin and Stringer (1968); finally, (d) Performance at work (F4) is a factor also found in Likert (1967) and Brown and Calnan (2016). On the other hand, some elements that are not in previous studies have been found to be relevant to measuring the work climate in EHS, such as items to evaluate personal and group shortcomings and possible quarrels between different professions in terms of delimitation of functions according to their specialty and recognition.

We obtained four factors when compared to the four-factor model by Payne and Mansfield (1978)—authority, efficiency, innovation, and adaptation—and we find one in common: productivity/achievement of aims, which is understood as efficiency. Although the other three factors are different, some of the contents of the items of our proposal are related, e.g., item 29, We participate in the decisions of our work group, refers to authority; item 13, We have the necessary infrastructure to carry out our work, and item 14, We receive the necessary training to carry out our tasks are related to innovation; and finally, item 3, We readily adapt to new circumstances, is related to adaptation.

Additionally, given that the partial starting point of this work was 18 items proposed by Perry et al. (2005) and that 15 have been included in the last version of the proposed scale presented in Supplementary Data 4 (specifically, items 1–4, 8–12, 17, 27–30, and 36), we now compare the psychometric properties of both instruments. The reliability values were adequate in both cases: α = 0.96 in the final version of the scale proposed and α = 0.87 in Perry et al. (2005). Both instruments obtained evidence based on the internal structure, although the factors found differ substantively, where three (clarity, support, and challenge) were found in Perry et al. (2005). Both instruments obtained evidence based on the relation test—criterion: ρ_XY = 0.68, p < 0.001 when correlating the scale obtained and an omnibus item and r_XY = 0.93, p < 0.001 in Perry et al. (2005) after correlating their instrument with the one proposed by Stringer (2002). In this sense, because both proposals presented adequate psychometric properties, the most important contribution of the scale proposed in this work is its specification in setting; Perry et al. (2005) based their study on public health organizations, and the present work is specifically for EHS, a context with an extremely specific idiosyncrasy (Hunt et al., 2006; Carret et al., 2007; Arora et al., 2013; Johnston et al., 2016; Vedaa et al., 2016; Gill et al., 2017).

Taking into account that any process to obtain evidence is affected by the characteristics of the intervention contexts in this particular case, given the sample characteristics, the instrument was designed with the aim of measuring the global construct quality of the work climate (as opposed to proposals centered on a specific aspect within work climate, e.g., Fletcher and Nusbaum, 2010; Mazzetti et al., 2016) for all professionals (as opposed to works based on specific professions, e.g., Rama-Maceiras et al., 2012; Popoola, 2016; Olsen et al., 2017) who work in EHS (as opposed to other specific contexts, e.g., Ehrhart et al., 2014).

To check empirically that the items worked in an unbiased way across different groups, we analyzed their DIF. Thirty-two items (80%) did not present any DIF. The fact that 8 items (20%) presented DIF does not necessarily indicate the weakness of the instrument given that substantive reasons explain the differences (American Educational Research Association et al., 2014). Item 2, We seek to understand the needs of our clients, was scored higher by workers with more experience because there is evidence that confirms a direct relationship between experience and implication with the patients (Ballester-Arnal et al., 2016) and the experience and perception of patients' needs in a holistic way instead of only targeting the isolated symptom (Zamanzadeh et al., 2015). Item 3, We readily adapt to new circumstances, and 23, Our expectations when we entered the working group have been fulfilled, were scored higher by women because females have a greater ability to adapt to new situations (Catalyst, 2007) and are more conformist (Aspiazu, 2016) in work. Item 5, We have the necessary experience to do our work well, was scored higher by workers with more experience, those who were older and those who had permanent contracts due to a real difference in the level of the aspect measured in this item (experience). Item 6, Our workday is adequate to develop our work, was scored higher by women, older workers, and those with permanent contracts. Differences in gender can be explained by the fact that women tend to assign greater value and spend time most productively in the workplace since they are also involved in other types of family activities (Artazcoz et al., 2004; Eagly and Carli, 2007); and workers with more experience and permanent contracts spend less time carrying out their functions than those who are less experienced and with eventual contracts, so it is logical that the first group believes more strongly than the second one that the time they have is sufficient to perform their work. Item 18, We feel motivated when doing our work, was more valued by younger workers with eventual contracts because studies conclude that there is an inverse relationship between age and job stability and motivation at work (Fernández et al., 2015; Akkermans et al., 2016; Dawson et al., 2017). Finally, items 27, We are recognized for our individual contributions, and 29, We participate in the decisions of our work group, were scored higher by men because they tend to develop dominant behaviors such as autonomy, independence and decision-making and seeking individual social recognition (Eagly et al., 2004; Godoy and Mladinic, 2009).

The proposed tool can be used in EHS to measure the work climate. Apart from a global value for each worker obtained by summing the values for the 40 items, the tool can be used to detect the global average work climate in EHS, factor by factor, or by studying each item to detect weaknesses so as to implement actions (with the origin in the work colleagues or the boss) to improve them; e.g., if a worker scored item 5 low (We have the necessary experience to do our work well), a workmate with more experience could start acting as a temporary mentor, or if the average of a work group in item 20 (Our colleagues value our profession) is low, then the boss of the group could implement strategies to improve the multidisciplinary work group conditions. In addition, by measuring the values with the scale before and after the actions implemented to improve the work climate, we can see the effectiveness of such actions (the use of inferential statistics for repeated measures would give evidence about the significance of the change).

In summary, the proposed scale can be used as a tool to diagnose the climate given in the work place and, based on this information, implement action to attempt to improve the situation (Perry et al., 2005). This can involve improvements at different levels since increasing the quality of the work climate will probably influence workers' satisfaction (Hooper et al., 2010), productivity (Brown and Leigh, 1996), interpersonal relationships (Lozano-Lozano et al., 2013), and performance (Engelen et al., 2016). Furthermore, it may act as a protective factor against alcoholism (Carreño et al., 2006), stress (Laposa et al., 2003), fatigue (Adriaenssens et al., 2011), or absenteeism (Mendoza-Llanos, 2015) in workers. Therefore, it will probably increase patients' satisfaction with the service (Hooper et al., 2010; Ajeigbe et al., 2013).

One limitation to highlight is the length of the resultant scale. Although 40 items to measure four factors is apparently not excessive, it is important to note that, currently, workers in EHS receive more patients than they can manage, so it would be difficult for them to find enough time to complete such a long instrument. Another possible limitation is the use of one item as a criterion. In this sense, it is necessary to carry out research that further develops the instrument obtained in this study.

To cover the first limitation (excessive length of the scale), we will shorten the scale without excluding relevant information to measure the work climate in EHS. First, based on the Spearman-Brown prediction (or prophecy) formula that relates reliability and length of the test (Brown, 1910; Spearman, 1910), we will estimate the number of items that we can remove in each factor without implying a decrease in the reliability coefficient lower than 0.8, a cut-off point to be considered a good result following the criteria established by George and Mallery (2003). Second, we will develop a Delphi method (Dalkey and Helmer, 1963) to determine, with a sample of at least 20 judges (50% experts in psychometrics and 50% EHS workers), the most redundant items to be removed. In further applications of the resulting short scale, we will check whether, as expected, the reliability coefficient maintains good reliability for coefficients in all the factors. To cover the second limitation (the use of one item as a criterion), we will use another scale as a criterion, specifically the scale proposed by Biggs et al. (2016), which presents partial validity evidence (based on relationships with other variables).

Additionally, we will delve into validity based on relations to other variables that present convergent and discriminant evidence of the resultant scale following the multitrait-multimethod matrix proposed by Campbell and Fiske (1959). We will request completion of three different instruments from ~100 EHS workers: the scale we proposed; another scale with partial validity evidence to measure the work climate in EHS (the one proposed by Biggs et al., 2016); and the Safety Attitudes Questionnaire (Sexton et al., 2016), an instrument that measures safety climate, a construct similar to work climate although not exactly the same (work climate is considered wider). We expect to find high reliability coefficients in the three instruments, a high correlation between the two instruments that measure work climate in EHS (convergent evidence) and a low correlation (at least lower than that obtained in the convergent evidence study) between our instrument and the one that measures a different construct (discriminant evidence).

Finally, we will test its factorial invariance to check whether the use of the scale obtained can be generalized to different hospitals, countries, genders, roles, and professions. First, we will gather a massive number of answers to the scale from participants of different hospitals in one country, from different countries (initially, Spain and Chile), different genders, different roles (coordinator/boss or without being in charge of other people), and different professions (nurses, nursing assistants, doctors, orderlies, and administrative officers). Second, using the software LISREL, we will test the invariance of the structural model and the measurement model across groups (Byrne, 1998).

Ethics Statement

This study was carried out in accordance with the recommendations of the “Declaration on bioethics and human rights, UNESCO, 2005” with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the “Ethics Committee, Universidad Autónoma de Chile.”

Author Contributions

The initial idea was generated by SC-M and was later supplemented and developed by SS-C, JL-L, and SC-M. JL-L gathered all the information and partially analyzed the data in Studies 1, 2, and 3. FH-T carried out most of the data analyses in Study 3. The manuscript was written by SS-C; JL-L and SC-M made a substantial contribution to the design of the paper, improving both its writing and structure. All gave consent to this final version for publication, and all agreed to be responsible for all aspects of the work, such as the accuracy of the data and the integrity of the research.

Funding

This research was funded by the Spanish Ministry of Science and Innovation (Reference PSI2011-29587), Chilean National Fund of Scientific and Technological Development, FONDECYT (1150096), and Spanish Ministry of Economy and Competitiveness (PSI2015-71947-REDT).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to dedicate this article to our late fellow researcher José A. Pérez-Gil, who participated actively in the initial stage of this work. This work would not have been possible without the collaboration of the professionals at the Punta Europa (Algeciras, Spain) and La Línea de la Concepción (Spain) Hospitals, led by Dr. Juan Rodríguez Medina, who always collaborated kindly.

The authors greatly appreciate all the comments received from the reviewers and the English language editor. We believe that the quality of this paper has been substantially enhanced as a result.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00010/full#supplementary-material

References

Adriaenssens, J., De Gucht, V., Van Der Doef, M., and Maes, S. (2011). Exploring the burden of emergency care: predictors of stress-health outcomes in emergency nurses. J. Adv. Nurs. 67, 1317–1328. doi: 10.1111/j.1365-2648.2010.05599.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Ajeigbe, D. O., McNeese-Smith, D., Leach, L. S., and Phillips, L. R. (2013). Nurse-physician teamwork in the emergency department: impact on perceptions of job environment, autonomy, and control over practice. J. Nurs. Adm. 43, 142–148. doi: 10.1097/NNA.0b013e318283dc23

PubMed Abstract | CrossRef Full Text | Google Scholar

Akkermans, J., de Lange, A. H., van der Heijden, B. I. J. M., Kooij, D. T. A. M., Jansen, P. G. W., and Dikkers, J. S. E. (2016). What about time? Examining chronological and subjective age and their relation to work motivation. Career Dev. Int. 21, 419–439. doi: 10.1108/CDI-04-2016-0063

CrossRef Full Text | Google Scholar

American Educational Research Association American Psychological Association, and National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.

Arora, M., Asha, S., Chinnappa, J., and Diwan, A. D. (2013). Burnout in emergency medicine physicians. Emerg. Med. Australas. 25, 491–495. doi: 10.1111/1742-6723.12135

PubMed Abstract | CrossRef Full Text | Google Scholar

Artazcoz, L., Benach, J., Borrell, C., and Cortès, I. (2004). Unemployment and mental health: Understanding the interactions among gender, family roles, and social class. Am. J. Public Health 94, 82–88. doi: 10.2105/AJPH.94.1.82

PubMed Abstract | CrossRef Full Text | Google Scholar

Aspiazu, E. L. (2016). Heterogeneidad y desigualdades de género en el sector salud: entre las estadísticas y las percepciones sobre las condiciones de trabajo [Heterogeneity and gender inequalities in the health sector: between statistics and perceptions about working conditions]. Revista Pilquen 19, 55–66. Available online at: http://www.scielo.org.ar/pdf/spilquen/v19n1/v19n1a05.pdf

Google Scholar

Bagozzi, R. P., and Yi, Y. (2012). Specification, evaluation, and interpretation of structural equation models. J. Acad. Market. Sci. 40, 8–34. doi: 10.1007/s11747-011-0278-x

CrossRef Full Text | Google Scholar

Ballester-Arnal, R., Gómez-Martínez, S., Gil-Juliá, B., Ferrándiz-Sellés, M. D., and Collado-Boira, E. J. (2016). Burnout y factores estresantes en profesionales sanitarios de las unidades de cuidados intensivos [Burnout and stressors in health workers in intensive care units]. Revista Psicopatol. Psicol. Clín. 21, 129–136. doi: 10.5944/rppc.vol.21.num.2.2016.16146

CrossRef Full Text

Bandalos, D. L. (1993). Factors influencing cross-validation of confirmatory factor analysis models. Multivariate Behav. Res. 28, 351–374. doi: 10.1207/s15327906mbr2803_3

PubMed Abstract | CrossRef Full Text | Google Scholar

Barbero-García, M. I. (1993). Psicometría II. Métodos de Elaboración de Escalas [Psychometrics II. Methods to Develop Scales]. Madrid: UNED.

Barbero-García, M. I., Vila-Abad, E., and Holgado-Tello, F. P. (2008). Tests Adaptation in cross-cultural comparative studies. Acción Psicol. 5, 7–16. doi: 10.5944/ap.5.2.454

CrossRef Full Text

Bendayan, R., Blanca, M. J., Fernández-Baena, J. F., Escobar, M., and Trianes, M. V. (2013). New empirical evidence on the validity of the Satisfaction with Life Scale in early adolescents. Eur. J. Psychol. Assess. 29, 36–43. doi: 10.1027/1015-5759/a000118

CrossRef Full Text | Google Scholar

Biggs, D. M., Swailes, S., and Baker, S. (2016). The measurement of worker relations: the development of a three-component scale. Leadersh. Organ. Dev. J. 37, 2–12. doi: 10.1108/LODJ-08-2012-0098

CrossRef Full Text | Google Scholar

Billings, J., and Raven, M. C. (2013). Dispelling an urban legend: frequent emergency department users have substantial burden of disease. Health Aff. 32, 2099–2108. doi: 10.1377/hlthaff.2012.1276

PubMed Abstract | CrossRef Full Text

Borg Xuereb, C., Shaw, R. L., and Lane, D. A. (2016). Patients' and physicians' experiences of atrial fibrillation consultations and anticoagulation decision-making: a multi-perspective IPA design. Psychol. Health 31, 436–455. doi: 10.1080/08870446.2015.1116534

PubMed Abstract | CrossRef Full Text | Google Scholar

Bragard, I., Dupuis, G., and Fleet, R. (2015). Quality of work life, burnout, and stress in emergency department physicians: a qualitative review. Eur. J. Emerg. Med. 22, 227–234. doi: 10.1097/MEJ.0000000000000194

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, P. R., and Calnan, M. W. (2016). Chains of (dis)trust: exploring the underpinnings of knowledge sharing and quality care across mental health services. Sociol. Health Illn. 38, 286–305. doi: 10.1111/1467-9566.12369

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, S. P., and Leigh, T. W. (1996). A new look at psychological climate and its relationship to job involvement, effort, and performance. J. Appl. Psychol. 81:358. doi: 10.1037/0021-9010.81.4.358

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, W. (1910). Some experimental results in the correlation of mental abilities. Br. J. Psychol. 3, 296–322.

Google Scholar

Browne, M. W., Cudeck, R., Bollen, K. A., and Long, J. S. (1993). Alternative ways of assessing model fit. Sage Focus Editions 154, 136–136.

Google Scholar

Byrne, B. M. (1998). Structural Equation Modeling with LISREL, PRELIS, and SIMPLIS: Basic Concepts, Applications, and Programming. Mahwah, NJ: Lawrence Erlbaum Associates.

Google Scholar

Campbell, D. T., and Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol. Bull. 56, 81–105. doi: 10.1037/h0046016

PubMed Abstract | CrossRef Full Text | Google Scholar

Carreño, S., Medina-Mora, M. A., Martínez, N., Juárez, F., and Vázquez, L. (2006). Características organizacionales, estrés y consumo de alcohol en trabajadores de una empresa textil mexicana [Organizational characteristics, stress and alcohol consumption in workers from a Mexican textile company]. Salud Mental 29, 63–70. Available online at: http://www.medigraphic.com/pdfs/salmen/sam-2006/sam064i.pdf

Google Scholar

Carret, M. L., Fassa, A. G., and Kawachi, I. (2007). Demand for emergency health service: factors associated with inappropriate use. BMC Health Serv. Res. 7:131. doi: 10.1186/1472-6963-7-131

PubMed Abstract | CrossRef Full Text | Google Scholar

Castaneda, G. A., and Scanlan, J. M. (2015). Job satisfaction in nursing: A concept analysis. Nurs. Forum 9, 130–138. doi: 10.1111/nuf.12056

CrossRef Full Text | Google Scholar

Catalyst (2007). The Double-Bind Dilemma for Women in Leaderships: Damned If You Do, Doomed If You Don't. New York, NY: Author.

Chacón-Moscoso, S., Sanduvete-Chaves, S., and Sánchez-Martín, M. (2016). The development of a checklist to enhance methodological quality in intervention programs. Front. Psychol. 7:1811. doi: 10.3389/fpsyg.2016.01811

PubMed Abstract | CrossRef Full Text | Google Scholar

Chakravarti, M., Laha, R. G., and Roy, J. (1967). Handbook of Methods of Applied Statistics. New York, NY: John Wiley & Sons.

Google Scholar

Chiang, H. Y., Hsiao, Y. C., and Lee, H. F. (2017). Predictors of hospital nurses' safety practices: work environment, workload, job satisfaction, and error reporting. J. Nurs. Care Qual. 32, 359–368. doi: 10.1097/NCQ.0000000000000240

PubMed Abstract | CrossRef Full Text | Google Scholar

Dalkey, N., and Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Manage. Sci. 9, 458–467.

Google Scholar

Davenport, D. L., Henderson, W. G., Mosca, C. L., Khuri, S. F., and Mentzer, R. M. (2007). Risk-adjusted morbidity in teaching hospitals correlates with reported levels of communication and collaboration on surgical teams but not with scale measures of teamwork climate, safety climate, or working conditions. J. Am. Coll. Surg. 205, 778–784. doi: 10.1016/j.jamcollsurg.2007.07.039

PubMed Abstract | CrossRef Full Text | Google Scholar

Dawson, C., Veliziotis, M., and Hopkins, B. (2017). Temporary employment, job satisfaction and subjective well-being. Econ. Ind. Democracy 38, 69–98. doi: 10.1177/0143831X14559781

CrossRef Full Text | Google Scholar

Delgado-Sánchez, A., Bellón, J. A., Martinez, M. T., Luna, J. D., López, L. A., and Lardelli, P. (2006). Las dimensiones del clima organizacional percibidas por los médicos de familia [Dimensions of organizational climate perceived by family physicians]. Aten. Primaria 37, 489–497. doi: 10.1157/13089089

CrossRef Full Text | Google Scholar

Eagly, A. H., and Carli, L. L. (2007). Women and the labyrinth of leadership. Harv. Bus. Rev. 62–71. Available online at: https://www.researchgate.net/profile/Linda_Carli/publication/5957753_Women_and_the_labyrinth_of_leadership/links/0046351eeab13a8f9b000000.pdf

Eagly, A. H., Wood, W., and Johannesen-Schmidt, M. C. (2004). “Social roles theory of sex differences and similarities: implications for the partner preferences of women and men,” in The Psychology of Gender, eds A. H. Eagly, A. Beall, and R. J. Sternberg (New York, NY: Guilford Press), 269–295.

Ehrhart, M. G., Aarons, G. A., and Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implement. Sci. 9:157. doi: 10.1186/s13012-014-0157-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Eliseo, L. J., Murray, K. A., White, L. F., Dyer, S., Mitchell, P. A., and Fernandez, W. G. (2012). EMS providers' perceptions of safety climate and adherence to safe work practices. Prehospital Emerg. Care 16, 53–58. doi: 10.3109/10903127.2011.621043

PubMed Abstract | CrossRef Full Text | Google Scholar

Elmi, S., Hassankhani, H., Abdollahzadeh, F., Abadi, M. A., Scott, J., and Nahamin, M. (2017). Validity and reliability of the Persian practice environment scale of nursing work index. Iran. J. Nurs. Midwifery Res. 22, 106–111. doi: 10.4103/1735-9066.205953

PubMed Abstract | CrossRef Full Text | Google Scholar

Engelen, L., Dhillon, H. M., Chau, J. Y., Hespe, D., and Bauman, A. E. (2016). Do active design buildings change health behaviour and workplace perceptions? Occup. Med. 66, 408–411. doi: 10.1093/occmed/kqv213

PubMed Abstract | CrossRef Full Text | Google Scholar

Estryn-Behar, M., Doppia, M. A., Guetarni, K., Fry, C., Machet, G., Pelloux, P., et al. (2011). Emergency physicians accumulate more stress factors than other physicians–results from the French SESMAT study. Emerg. Med. J. 28, 397–410. doi: 10.1136/emj.2009.082594

PubMed Abstract | CrossRef Full Text | Google Scholar

Fernández, A. L., Rodríguez, D. S., Aranda, M. J. G., Paiz, A. L., and Saavedra, L. H. (2015). Estudio sobre las relaciones del síndrome de burnout con algunos factores psicosociales [Study on the relationships of burnout syndrome with some psychosocial factors]. ReiDoCrea 4, 94–99. Available online at: http://digibug.ugr.es/bitstream/10481/37009/1/ReiDoCrea-Vol.4-Art.13.pdf

Google Scholar

Fernández-Argüelles, R., Cobos, P. A., and Figueroa, M. R. (2015). Evaluation of the organizational climate in a center for rehabilitation and special education. Revista Cubana de Salud Pública 41, 593–602.

Field, A. P. (2000). Discovering Statistics Using SPSS for Windows: Advances Techniques for the Beginner. London: Sage.

Fletcher, T. D., and Nusbaum, D. N. (2010). Development of the competitive work environment scale: a multidimensional climate construct. Educ. Psychol. Meas. 70, 105–124. doi: 10.1177/0013164409344492

CrossRef Full Text | Google Scholar

García, A., Moro-Tejedor, M. N., and Medina-Torres, M. (2010). Evaluation and dimensions that define the labor environment and job satisfaction in nursing staff. Revista de Calidad Asistencial 25, 207–214. doi: 10.1016/j.cali.2010.02.003

CrossRef Full Text | Google Scholar

George, D., and Mallery, P. (2003). SPSS forWindows Step by Step: A Simple Guide and Reference. Boston, MA: Allyn & Bacon.

Google Scholar

Gill, P. J., Saunders, N., Gandhi, S., Gonzalez, A., Kurdyak, P., Vigod, S., et al. (2017). Emergency department as a first contact for mental health problems in children and youth. J. Am. Acad. Child Adolesc. Psychiatry 56, 475–482. doi: 10.1016/j.jaac.2017.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Glück, J., König, S., Naschenweng, K., Redzanowski, U., Dorner, L., Straßer, I., et al. (2013). How to measure wisdom: Content, reliability, and validity of five measures. Front. Psychol. 4:405. doi: 10.3389/fpsyg.2013.00405

PubMed Abstract | CrossRef Full Text | Google Scholar

Godoi, V. C. G., Ganassin, G. S., Inoue, K. C., and de Moraes, N. L. (2016). Reception with risk classification: Characteristics of the demand in an emergency unit. Cogitare Enferm 21, 1–8. Available online at: http://www.saude.ufpr.br/portal/revistacogitare/wp-content/uploads/sites/28/2016/12/44664-186755-1-PB.pdf

Google Scholar

Godoy, L., and Mladinic, A. (2009). Estereotipos y roles de género en la evaluación laboral y personal de hombres y mujeres en cargos de dirección [Gender stereotypes and roles in the work and personal evaluation of men and women in management positions]. Psykhe 18, 51–64. doi: 10.4067/S0718-22282009000200004

CrossRef Full Text | Google Scholar

Gómez, C. A. (2004). Dise-o, construcción y validación de un instrumento que evalúa clima organizacional en empresas colombianas, desde la teoría de respuesta a los ítems [Design, construction and validation of an instrument that evaluates organizational climate in Colombian companies, from the item response theory]. Acta Colomb. Psicol. 11, 97–113. Available online at: http://editorial.ucatolica.edu.co/ojsucatolica/revistas_ucatolica/index.php/acta-colombiana-psicologia/article/view/482/479

González, A., Padilla, J. L., Hidalgo, M. D., Gómez-Benito, J., and Benítez, I. (2011). EASY-DIF software for analyzing differential item functioning using the Mantel-Haenszel and standardization procedures. Appl. Psychol. Meas. 35, 483–484. doi: 10.1177/0146621610381489

CrossRef Full Text | Google Scholar

Hoe, S. L. (2008). Issues and procedures in adopting structural equation modeling technique. J. Appl. Quant. Methods 3, 76–83. Available online at: http://www.jaqm.ro/issues/volume-3,issue-1/pdfs/jaqm_vol3_issue1.pdf#page=81

Google Scholar

Holgado-Tello, F. P., Chacón-Moscoso, S., Barbero-García, M. I., and Vila-Abad, E. (2010). Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Qual. Quant. 44, 153–166. doi: 10.1007/s11135-008-9190-y

CrossRef Full Text | Google Scholar

Holgado-Tello, F. P., Chacón-Moscoso, S., Vila-Abad, E., Delgado, B., Sanduvete-Chaves, S., and Barbero-García, M. I. (2015). Evaluación de programas de formación continua en contextos no estandarizados: complementariedad entre Análisis Factorial y Multinivel para la obtención de evidencias de validez de constructo [Training program evaluation in non-standardized context: complementarity across factorial and multilevel analysis to obtain construct validity evidence]. Anal. Psicol. 31, 725–732. doi: 10.6018/analesps.31.2.172501

CrossRef Full Text

Hooper, C., Craig, J., Janvrin, D. R., Wetsel, M. A., and Reimels, E. (2010). Compassion satisfaction, burnout, and compassion fatigue among emergency nurses compared with nurses in other selected inpatient specialties. J. Emerg. Nurs. 36, 420–427. doi: 10.1016/j.jen.2009.11.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Hooper, D., Coughlan, J., and Mullen, M. (2008). Structural equation modelling: guidelines for determining model fit. Electron. J. Bus. Res. Methods 6, 56–60. Available online at: http://www.sciencedirect.com/science/article/pii/S0099176709005534?via%3Dihub

Google Scholar

Hunt, K. A., Weber, E. J., Showstack, J. A., Colby, D. C., and Callaham, M. L. (2006). Characteristics of frequent users of emergency departments. Ann. Emerg. Med. 48, 1–8. doi: 10.1016/j.annemergmed.2005.12.030

PubMed Abstract | CrossRef Full Text | Google Scholar

Insel, P., and Moos, R. H. (1974). Psychological environments: Expanding the scope of human ecology. Am. Psychol. 29, 179–188. doi: 10.1037/h0035994

CrossRef Full Text | Google Scholar

International Test Commission (2005). ITC Guidelines for Translating and Adapting Tests. Available online at: https://www.intestcom.org/files/guideline_test_adaptation.pdf

Johnston, A., Abraham, L., Greenslade, J., Thom, O., Carlstrom, E., Wallis, M., et al. (2016). Review article: staff perception of the emergency department working environment: integrative review of the literature. Emerg. Med. Australas. 28, 7–26. doi: 10.1111/1742-6723.12522

PubMed Abstract | CrossRef Full Text | Google Scholar

Jöreskog, K. G. (2003). Factor Analysis by MINRES. To the Memory of Harry Harman and Henry Kaiser. Available online at: http://www.ssicentral.com/lisrel/techdocs/minres.pdf

Kutner, M. H., Nachtsheim, C. J., Neter, J., Li, W., and Butler, J. L. (2004). Applied Linear Statistical Models. Boston, MA: McGraw-Hill/Irwin.

Google Scholar

Laposa, J. M., Alden, L. E., and Fullerton, L. M. (2003). Work stress and posttraumatic stress disorder in ED nurses/personnel (CE). J. Emerg. Nurs. 29, 23–28. doi: 10.1067/men.2003.7

CrossRef Full Text | Google Scholar

Lee, S., Davis, K. D., Neuendorf, C., Grandey, A., Lam, C. B., and Almeida, D. M. (2016). Individual-and organization-level work-to-family spillover are uniquely associated with hotel managers' work exhaustion and satisfaction. Front. Psychol. 7:1180. doi: 10.3389/fpsyg.2016.01180

PubMed Abstract | CrossRef Full Text | Google Scholar

Leung, C. C., and Gomersall, C. D. (2016). “Intensive care unit triage,” in Quality Management in Intensive Care: A practical Guide, eds H. Flaatten, B. Guidet, and A. Valentin (Cambridge, UK: Cambridge University Press), 77–84.

Google Scholar

Li, C., Wu, Y., and Kee, Y. H. (2016). Validation of the volunteer motivation scale and its relations with work climate and intention among chinese volunteers. Asian J. Soc. Psychol. 19, 124–133. doi: 10.1111/ajsp.12127

CrossRef Full Text | Google Scholar

Likert, R. (1967). The Human Organization: Its Management and Value. New York, NY: McGraw-Hill.

Google Scholar

Litwin, G., and Stringer, R. (1968). Motivation and Organizational Climate. Boston, MA: Harvard Business School Press.

Google Scholar

Lozano-Lozano, J. A., Chacón-Moscoso, S., Sanduvete-Chaves, S., and Pérez-Gil, J. A. (2013). Main components of organizational climate in an emergency medical service: a qualitative approach. Acción Psicol. 10, 101–114. doi: 10.5944/ap.10.2.12213

CrossRef Full Text

Mazzetti, G., Schaufeli, W. B., Guglielmi, D., and Depolo, M. (2016). Overwork climate scale: psychometric properties and relationships with working hard. J. Manage. Psychol. 31, 880–896. doi: 10.1108/JMP-03-2014-0100

CrossRef Full Text | Google Scholar

Mendoza-Llanos, R. (2015). Job satisfaction and organizational culture as predictors of absenteeism. Rev. Méd. Chil. 143, 1028–1033. doi: 10.4067/S0034-98872015000800010

PubMed Abstract | CrossRef Full Text | Google Scholar

Meneghel, I., Borgogni, L., Miraglia, M., Salanova, M., and Martínez, I. M. (2016). From social context and resilience to performance through job satisfaction: a multilevel study over time. Hum. Relations 69, 2047–2067. doi: 10.1177/0018726716631808

CrossRef Full Text | Google Scholar

Moos, R. H., and Insel, P. M. (1974). The Work Environment Scale. Palo Alto, CA: Consulting Psychologists Press.

Morata-Ramírez, M. A., and Holgado-Tello, F. P. (2013). Construct validity of Likert scales through confirmatory factor analysis: a simulation study comparing different methods of estimation based on Pearson and polychoric correlations. Int. J. Soc. Sci. Stud. 1, 54–61. doi: 10.11114/ijsss.v1i1.27

CrossRef Full Text

Morata-Ramírez, M. A., Holgado-Tello, F. P., Barbero-García, M. I., and Méndez, G. (2015). Análisis factorial confirmatorio: recomendaciones sobre mínimos cuadrados no ponderados en función del error Tipo I de Ji-Cuadrado y RMSEA [Confirmatory factor analysis: unweighted least squares recommendations as a function of Type I Chi-Squared error and RMSEA]. Acción Psicol. 12, 79–90. doi: 10.5944/ap.12.1.14362

CrossRef Full Text

Olsen, E., Bjaalid, G., and Mikkelsen, A. (2017). Work climate and the mediating role of workplace bullying related to job performance, job satisfaction, and work ability: a study among hospital nurses. J. Adv. Nurs. 31, 880–896. doi: 10.1111/jan.13337

CrossRef Full Text | Google Scholar

Osterlind, S. J. (1998). Constructing Test Items. Boston, MA: Kluwer Academic Publishers.

Google Scholar

Patterson, P. D., Huang, D. T., Fairbanks, R. J., and Wang, H. E. (2010). The emergency medical services safety attitudes questionnaire. Am. J. Med. Qual. 25, 109–115. doi: 10.1177/1062860609352106

PubMed Abstract | CrossRef Full Text | Google Scholar

Payne, R. L., and Mansfield, R. (1978). Correlates of individual perceptions of organizational climate. J. Occup. Psychol. 51, 209–218. doi: 10.1111/j.2044-8325.1978.tb00417.x

CrossRef Full Text | Google Scholar

Pérez-Gil, J. A., Chacón-Moscoso, S., and Moreno, R. (2000). Validez de constructo: el uso del análisis factorial exploratorio - confirmatorio para obtener evidencias de validez [Construct validity: The use of exploratory - confirmatory factor analysis to obtain validity evidence]. Psicothema 2, 442–446. Available online at: http://www.psicothema.es/pdf/601.pdf

Perry, C., LeMay, N., Rodway, G., Tracy, A., and Galer, J. (2005). Validating a work group climate assessment tool for improving the performance of public health organizations. Hum. Resour. Health 3, 1–8. doi: 10.1186/1478-4491-3-10

PubMed Abstract | CrossRef Full Text | Google Scholar

Popoola, S. O. (2016). Personal factors, psychological work climate, and role related factors as precursors of organizational commitment of records management employees in state civil services of Northwest, Nigeria. Afr. J. Bus. Manage. 10, 318–327. Available online at: http://www.academicjournals.org/journal/AJBM/article-full-text/D5F099159857

Google Scholar

Prieto, G., and Muñiz, J. (2000). Un modelo para evaluar la calidad de los test utilizados en Espa-a [A model to evaluate the quality of the tests used in Spain]. Papeles Psicól. 77, 65–75. Available online at: http://www.redalyc.org/pdf/778/77807709.pdf

Rama-Maceiras, P., Parente, S., and Kranke, P. (2012). Job satisfaction, stress and burnout in anaesthesia: relevant topics for anaesthesiologists and health care managers? Eur. J. Anaesthesiol. 29, 311–319. doi: 10.1097/EJA.0b013e328352816d

CrossRef Full Text | Google Scholar

Rojas, M., Tirado, L. L., Pacheco, R. L., Escamilla, R., and López, M. (2011). Work climate in Mexican hemodialysis units. A cross sectional study. Neurología 21, 76–83. doi: 10.3265/Nefrologia.pre2010

CrossRef Full Text

Sabri, S. (2013). Item analysis of student comprehensive test for research in teaching beginner string ensemble using model based teaching among music students in public universities. Int. J. Educ. Res. 1, 1–14. Available online at: http://www.ijern.com/journal/December-2013/28.pdf

Google Scholar

Safi, M. H., Torkaman-Nejad, S., and Arshi, S. (2016). Job satisfaction and its influencing factors among Shomal health center of Tehran personnel in 2014. Commun. Health 2, 64–71. Available online at: http://journals.sbmu.ac.ir/en-ch/article/view/9126

Google Scholar

Sanduvete-Chaves, S., Chacón-Moscoso, S., Sánchez-Martín, M., and Pérez-Gil, J. A. (2013). The revised Osterlind index. A comparative analysis in content validity studies. Acción Psicol. 10, 3–10. doi: 10.5944/ap.10.2.11821

CrossRef Full Text | Google Scholar

Schneider, B., Ehrhart, M. G., and Macey, W. H. (2013). Organizational climate and culture. Annu. Rev. Psychol. 64, 361–388. doi: 10.1146/annurev-psych-113011-143809

PubMed Abstract | CrossRef Full Text | Google Scholar

Schulz, H., Zacher, H., and Lippke, S. (2017). The importance of team health climate for health-related outcomes of white-collar workers. Front. Psychol. 8:74. doi: 10.3389/fpsyg.2017.00074

PubMed Abstract | CrossRef Full Text | Google Scholar

Sexton, J. B., Schwartz, S. P., Chadwick, W. A., Rehder, K. J., Bae, J., Bokovoy, J., et al. (2016). The associations between work–life balance behaviours, teamwork climate and safety climate: cross-sectional survey introducing the work–life climate scale, psychometric properties, benchmarking data and future directions. BMJ Qual. Saf. 6:44. doi: 10.1136/bmjqs-2016-006032

PubMed Abstract | CrossRef Full Text | Google Scholar

Spearman, C. C. (1910). Correlation calculated from faulty data. Br. J. Psychol. 3, 271–295. doi: 10.1111/j.2044-8295.1910.tb00206.x

CrossRef Full Text | Google Scholar

Strauss, A., and Corbin, J. M. (1998). Basic of Qualitative Research: Techniques and Procedures for Developing Grounded. Thousand Oaks, CA: Sage.

Stringer, R. (2002). Leadership and Organizational Climate. Upper Saddle River, NJ: Prentice Hall.

Google Scholar

Swiger, P. A., Raju, D., Breckenridge-Sproat, S., and Patrician, P. A. (2017). Adaptation of the practice environment scale for military nurses: a psychometric analysis. J. Adv. Nurs. 73, 2219–2236. doi: 10.1111/jan.13276

PubMed Abstract | CrossRef Full Text | Google Scholar

Tavakol, M., and Dennick, R. (2011). Making sense of Cronbach's alpha. Int. J. Med. Educ. 2, 53–55. doi: 10.5116/ijme.4dfb.8dfd

PubMed Abstract | CrossRef Full Text | Google Scholar

Taylor, J. A., Dominici, F., Agnew, J., Gerwin, D., Morlock, L., and Miller, M. R. (2011). Do nurse and patient injuries share common antecedents? An analysis of associations with safety climate and working conditions. BMJ Qual. Saf. 21, 101–111. doi: 10.1136/bmjqs-2011-000082

PubMed Abstract | CrossRef Full Text | Google Scholar

Torres, E., and Zegarra, S. J. (2015). Organizational climate and work performance in the Bolivarian educational institutions of the city Puno - 2014. Comuni@cción 6, 5–14. Available online at: http://www.redalyc.org/articulo.oa?id=449844870001

Google Scholar

Uribe-Prado, J.-F., Patlan, J., and García, A. (2015). Psychosomatic manifestations, commitment and burnout as consequents from climate and organization culture: a path analysis. Consultoría Administración 60, 447–467. Available online at: http://www.cya.unam.mx/index.php/cya/article/view/720

Vedaa, Ø., Harris, A., Bjorvatn, B., Waage, S., Sivertsen, B., Tucker, P., et al. (2016). Systematic review of the relationship between quick returns in rotating shift work and health-related outcomes. Ergonomics 59, 1–14. doi: 10.1080/00140139.2015.1052020

PubMed Abstract | CrossRef Full Text | Google Scholar

Wankhade, P., and Mackway-Jones, K. (2015). Ambulance Services: Leadership and Management Perspectives. Cham: Springer.

Google Scholar

Yela, M. (1957). La Técnica del Análisis Factorial: Un Método de Investigación en Psicología y Pedagogía [The Factor Analysis Technique: A Research Method in Psychology and Pedagogy]. Madrid: Biblioteca Nueva.

Zacher, H., and Yang, J. (2016). Organizational climate for successful aging. Front. Psychol. 7:1007. doi: 10.3389/fpsyg.2016.01007

PubMed Abstract | CrossRef Full Text | Google Scholar

Zadow, A. J., Dollard, M. F., McLinton, S. S., Lawrence, P., and Tuckey, M. R. (2017). Psychosocial safety climate, emotional exhaustion, and work injuries in healthcare workplaces. Stress Health 33, 558–569. doi: 10.1002/smi.2740

PubMed Abstract | CrossRef Full Text | Google Scholar

Zamanzadeh, V., Jasemi, M., Valizadeh, L., Keogh, B., and Taleghani, F. (2015). Effective factors in providing holistic care: a qualitative study. Indian J. Palliat. Care 21, 214–224. doi: 10.4103/0973-1075.156506

PubMed Abstract | CrossRef Full Text | Google Scholar

Zenteno, A. C., and Durán, C. A. (2016). Factores y prácticas de alto desempe-o que influyen en el clima laboral: análisis de un caso [High performance factors and practices that influence the work climate: a case analysis]. Innovar 26, 119–136. doi: 10.15446/innovar.v26n59.54367

CrossRef Full Text | Google Scholar