Methodological procedure based on quantitizing/liquefying: a case study to assess work climate in an emergency department

Introduction: In the assessment of health organizations, results-based indicators are mainly used, with no consideration of internal work dynamics. This type of assessment forfeits much of the rich, useful information needed to make decisions on improving the organization. In order to address this, a rigorous procedure based on mixed methods is laid out here on gathering, analyzing, and interpreting data associated with the implementation process.

Methods: A 55-year-old doctor was selected at random from among the staff who volunteered to be interviewed at the emergency department at a public hospital located in southern Spain for an interview. Qualitative data obtained from the in-depth interview (indirect observation) were progressively systematized (liquefied and quantitized) based on a theoretical framework until a code matrix was obtained, without losing or distorting any information. Afterwards, data quality was controlled using Cohen’s kappa (κ) coefficient. A quantitative polar coordinate analysis was then carried out using the free software HOISAN (v. 1.6.3.3) to obtain robust results, vectorizing the relationships between codes and specifying whenever such relationships were statistically significant (and if they resulted in behavior activation or inhibition). Finally, a supplementary quantitative and qualitative assessment was carried out.

Results and discussion: The proposed method was applied to the needs assessment of teams in order to evaluate that work climate in the hospital’s emergency department Health Services of a hospital. Data quality control yielded an adequate result (κ = 0.82). Significant activation and inhibition of behaviors occurred, both prospectively and retrospectively. For instance, We seek to understand the needs of our clients and We readily adapt to new circumstances showed a significant activation (vector length = 3.43, p < 0.01) both prospectively (Z_sum = 0.48) and retrospectively (Z_sum = 3.4).

An adequate method to obtain detailed information about group dynamics in a work environment is presented, based on an in-depth interview. Practical applications for implementations to improve the functioning of organizations are presented.

1 Introduction

There have been three predominant theoretical approaches to the concept of work climate: (a) those that focus on the objective and structural characteristics of organizations (Li et al., 2022); (b) those that delve into the individual psychological characteristics of workers (Mozgovoy, 2022); and (c) those that adopt a perspective that considers both organizational and individual aspects (Altuntaş et al., 2021). This paper is based on the third approach, which is widely regarded as the most comprehensive given that it allows a multidimensional exploration of work climate.

In the context of healthcare services, the model proposed (Perry et al., 2005) conceptualizes work climate as an integral component of the organization primarily shaped by individual behavior and human interactions. According to this perspective, each working team within a hospital setting possesses a unique work climate, defined as the quality of the internal environment experienced by its members. It is believed to have a substantial influence on staff behavior. Therefore, any alteration in the work climate of a healthcare organization can lead to changes in other aspects of the organization, such as the effectiveness or quality of patient care (Santana and Pérez-Rico, 2023).

Hospital Emergency Departments (ED) exhibit distinctive features compared to other healthcare services. Professionals working in the ED confront any number of unique challenges: (a) they constitute the frontline of medical care for patients, playing a pivotal role in initial care (Trebach et al., 2021); (b) they provide care to a substantial volume of patients, irrespective of the severity of their condition (Holzinger et al., 2021); c) they deal with traumatic situations that require immediate and critical decision-making on a daily basis (Bijani et al., 2021); (d) they experience heightened levels of stress and responsibility owing to the inherent nature of their work (Barth et al., 2022); (e) they face a higher risk of post-traumatic stress disorder and depression due to their workplace experiences (Taylor et al., 2023). In this specific context, an assessment of work climate offers insights into both workgroup characteristics and organizational aspects, providing insight into job satisfaction, productivity, goal achievement, interpersonal relationships, and job performance (Sanduvete-Chaves et al., 2018; Lozano-Lozano et al., 2021).

Results-based indicators are mainly used when evaluating healthcare organizations, with no consideration of internal work dynamics. This type of assessment forfeits much of the rich, useful information needed to make decisions on improving the organization. Therefore, this work presents a rigorous procedure based on mixed methods to gather, analyze, and interpret the data associated with the implementation process. Qualitative data obtained from an in-depth interview (indirect observation) were progressively systematized until a code matrix was obtained, without losing or distorting any information. A quantitative polar coordinate analysis was then carried out to obtain robust results, vectorizing the relationships between codes. Finally, a supplementary quantitative and qualitative assessment was carried out.

Observational methodology was optimal for this task, given that it is both rigorous and flexible (Chacón-Moscoso et al., 2014; Anguera, 2020). Specifically, indirect observation was used (Anguera, 2021). In terms of the information obtained in the in-depth interview, as noted in an earlier work,

…all these materials constitute an extremely rich source of information for studying everyday life, and they are continuously growing with the burgeoning of new technologies for data recording, dissemination, and storage. Narratives are an excellent vehicle for studying everyday life, and quantization is proposed as a means of integrating qualitative and quantitative elements. However, this analysis requires a structured system that enables researchers to analyze varying forms and sources of information objectively (Anguera et al., 2018c).

Indirect observation facilitates the analysis of diverse texts, as has been shown in works on self-registrations (Squires et al., 2011), teacher – student conversations (García-Fariña et al., 2018), therapist – teenager patient dialog (Arias-Pujol and Anguera, 2020), therapist – adult patient dialog (del Giacco et al., 2020), in-depth interviews with elite trainers (Nunes et al., 2022), in-depth interviews with professional coaches and players (Iván-Baragaño et al., 2023), panel discussions with professionals about the foreseeable consequences of COVID-19 (Moya et al., 2020), and in-depth interview with family firms where there is an intrafamilial conflict (Alvarado-Alvarez et al., 2021).

In-depth interviews, unlike other types of interviews, allow researchers to delve in detail into aspects that required the participant to recount them as they were experienced. Therefore, these interviews capture a wide range of opinions and experiences among a group of professionals. They provide more extensive and detailed data on the unique contextual factors of each participant, facilitating a comprehensive representation of professionals’ daily experiences, their decision-making processes, and their interactions with both patients and the medical team (Merriam and Tisdell, 2016; Marshall et al., 2022). Furthermore, in-depth interviews can be adapted, including new questions or modifying existing ones based on the previous responses of participants. This enables a deeper exploration of specific areas of interest (Rubin and Rubin, 2005).

A single-case study can be defined as an intensive study of a single unit with an aim to generalize across a larger set of units (Gerring, 2004; Hunziker and Blankenagel, 2021). The interest in single-case studies has increased in recent years in different health fields (Stetler, 2009; Champagne, 2014; Porcheret, 2014). Single-case studies present advantages, such us they enable methodological advances, especially in the case of observational studies (Kitzmiller, 2010; Arias-Pujol et al., 2022). Additionally, the polar coordinate analysis proposed has been used in recent years in various fields, such as clinical psychology (Rodríguez-Medina et al., 2018; del Giacco et al., 2020; Arias-Pujol et al., 2022), occupational health (Portell et al., 2019), classroom activities (Belza et al., 2019), sports (Maneiro et al., 2020 Iván-Baragaño et al., 2023), and conflict mediation (Alvarado-Alvarez et al., 2021). However, the context in which this single-case study (Sandelowski et al., 2009; Anguera, 2018; Arias-Pujol et al., 2022) was developed, a hospital ED, is totally new.

Though the approach to single-case studies has generally been qualitative, it has also been heterogeneous in the literature. Our objective was to develop a case study from a mixed methods approach, thus integrating qualitative and quantitative elements. Specifically, a needs assessment (Sanduvete-Chaves et al., 2009) of work teams was carried out to evaluate the work climate. Given that the starting point was a single interview, an intensive treatment of the information is needed.

2 Methods

Two checklists guided the methodological process: the Guidelines for Reporting Evaluations based on Observational Methodology (Portell et al., 2015) and the Methodological Quality Checklist for studies based on Observational Methodology (Chacón-Moscoso et al., 2019), derived from the Guidelines for Reporting Evaluations based on Observational Methodology, which presents adequate validity evidence based on its content (Chacón-Moscoso et al., 2018).

2.1 Design

The observational design presented is idiographic, punctual, and multidimensional (Blanco-Villaseñor et al., 2003; Anguera et al., 2021). It is idiographic because it is based on a single case (an interview). The design can be considered punctual because the information was obtained in a single session, although there was intra-session follow-up because the full interview was analyzed. Finally, it is multidimensional because the study was based on a theoretical framework and structured in four dimensions. It then materialized in the observation tool.

2.2 Participant

The person selected at random from among the staff who volunteered to be interviewed was an ED worker at a public hospital in the south of Spain. The patients at this hospital hail from a diverse demographic. The selection criteria for the participant were (a) currently working at the ED when the study was carried out, and (b) having worked in the ED for at least three years.

The worker interviewed was a 55-year-old man who participated voluntarily. He was a doctor in the ED with an indefinite contract, 25 years of experience in his profession, and 23 years working at his current workplace. One of the authors, JALL, interviewed the participant in a hospital room. The interview yielded substantial new and rich information.

2.3 Instruments

2.3.1 Observation instrument

Based on a theoretical framework that laid out the fundamental work climate factors for ED teams (Sanduvete-Chaves et al., 2018) and the unique characteristics of this service, a non-standard tool was designed in keeping with indirect observation (Portell et al., 2015; Anguera et al., 2018c; Anguera, 2021). This tool combined field format and a system of categories.

The dimensions proposed for the theoretical framework included (A) job satisfaction, referring to how workers feel about their job and work conditions; (B) productivity/achievement of aims, which refers to the workers’ sense of whether they have everything they need to do their job properly and achieve their goals; (C) interpersonal relationships, referring to the workers’ feelings about their fellow team members; and (D) work performance, referred to all aspects related to how the workers do their jobs (Table 1). A system of exhaustive and mutually exclusive categories was built for each dimension.

TABLE 1

Table 1. Indirect observation instrument.

2.3.2 Recording device

A digital audio recorder was used to record the in-depth interview. Express Scribe software (version 2.0) was used for the transcription.

2.4 Procedure

2.4.1 Quantitizing/liquefying

According to the methodological process of indirect observation, the text can be liquefied (Anguera, 2020). This is combined with the quantitizing of the text in keeping with (direct and indirect) observational methodology, from the mixed-method perspective. It is possible to operate quantitatively with codes to which the textual units are assigned, either uniquely or in co-occurrence.

The mixed methods (Creswell and Plano Clark, 2011), broadly developed in the last two decades, offers vast potential for the integration of qualitative and quantitative elements. When referring to such integration paths, connecting (Creswell and Plano Clark, 2011) was chosen to build one dataset upon the other. The connecting forms the groundwork for the quantitizing, which consolidates previous proposals that have enabled the observational methodology to be considered a mixed method on its own (Anguera and Hernández-Mendo, 2016 Anguera et al., 2017). Neither methods nor techniques are combined, as is traditional according to conventional mixed methods or in multimethod studies (Anguera et al., 2018b). Data from different sources are neither combined. The information is transformed in a different way than what is described in the literature on mixed methods (Sandelowski et al., 2009).

This work presents a proposal for quantitizing that can be adapted to all situations, enabling the observational methodology to be used as a scientific method (Anguera et al., 2018a Chacón-Moscoso et al., 2021) Additionally, it fits with the connecting (Creswell and Plano Clark, 2011). It materializes when the macro-stages QUAL-QUAN-QUAL are alternated (Figure 1). In the first QUAL stage, the observational design is proposed, and the indirect observation instrument is built, yielding a descriptive record that is systematized until it becomes a code matrix. In the QUAN stage, the systematized record is subjected to data quality control (Chacón-Moscoso et al., 2023), using robust analytical techniques appropriate for categorical data. Finally, in the second QUAL stage, the results are interpreted. This procedure is consistent with the nature of the information obtained and the methodology applied.

FIGURE 1

Figure 1. Steps of indirect observation.

The information captured during the interview is transformed into a record and this constitutes the raw material for analysis. This data collection relies on a text transcription as its first step, taking into account that “transcripts are necessarily selective in the details that are represented” (Hepburn and Bolden, 2013). In this study, only verbal behavior was considered, without taking into account any additional information that could be garnered (gesture, gaze, cadence, etc.) in studies focused on the multimodality of human behavior.

The first decision to establish was the segmentation criterion in units. The behavior flow was set in interactive units (Birdwhistell, 1970) that were then ranked based on their granularity (Schegloff, 2000). Various levels of granularity could be formed (Schegloff and Sacks, 1973). Given the advantages and disadvantages of each unit type, we have combined syntactic and contextual criteria (Anguera, 2020).

Once the transcription was segmented into text units by means of the indirect observation instrument (Table 1), each text unit was coded. One or more codes were obtained for each of them, depending on the corresponding dimension/s involved. If only one dimension was affected, the text unit was assigned to the corresponding category; if several dimensions were affected, the co-occurring categories were recorded, generating a systematized record (Figure 2). The key to this coding is based on the data classification widely used in both direct and indirect observational methodology (Bakeman, 1978; Anguera et al., 2018c), from the beginning to the end of the session, obtaining a codes matrix whose columns were the four dimensions (A, B, C and D) established in the indirect observation instrument. Each row represented one of the textual units obtained when the text of the interview was segmented.

FIGURE 2

Figure 2. A graphic depiction of quantitizing/liquefying.

When completed, the first QUAL stage yielded a code matrix. Since data are already arranged sequentially and available for analysis, once the data went through quality control, quantizing is possible.

2.4.2 Data quality control

To control the data quality, (Holgado-Tello et al., 2016) three independent researchers (authors MTA, SCM, and SSC) coded the transcription of the interview, dividing the text into content units and assigning a code for each different content. Disagreements were resolved through consensus. The Supplementary material includes a segment of the interview with an example of a coding assignation. The kappa (κ) coefficient (Cohen, 1960) was calculated using HOISAN (v. 1.6.3.3), based on the records made by the three coders. Values over 0.7 were considered adequate.

2.4.3 Polar coordinate analysis

Polar coordinate analysis is one of the most suitable categorical data analysis techniques (Blanco-Villaseñor et al., 2003; Anguera et al., 2021), given that it allows a relations map to be drafted. This map shows the statistical relationships of association between a specific behavior of interest, the focal behavior, and others known as conditioned behaviors, i.e., behaviors that may be related to the focal behavior. For those that do relate, it is then necessary to determine the type and degree of the relationship. This quantitative relations analysis also allows these relationships to be charted as vectors, facilitating conclusions about the relationships, types, and intensities of behaviors.

The polar coordinate analysis (Sackett, 1980) constitutes a second stage of the lag sequential analysis (Bakeman, 1978). It considers the adjusted residuals obtained in the lag sequential analysis as data, once these are corrected (Allison and Liker, 1982). The analysis starts with the focal behavior and is both prospective (forward-looking) and retrospective (looking backwards). It provides insight into how the relationship between the focal behavior and the conditioned behaviors varies or evolves over time (in this case, throughout the recording of the interview). Retrospectivity from lag 0 to −1, from lag −1 to −2, and so on, was applied (Anguera, 1997). This analysis algorithm was included when programming the HOISAN software (Hernández-Mendo et al., 2012).

The data obtained were independent because each datum is calculated based on the corresponding lag. This enabled the use of parameter Z_sum or the “flag” parameter (Sackett, 1980), thus facilitating data reduction. The Z_sum parameter is based on the principle that the distribution of the sums of n-independent Z values is normal, with μ = 0 and σ = 1. It is calculated using the formula

$Z_{\mathit{sum}}=\frac{\sum Z}{\sqrt{n}}$

where n is the number of lags. Z_sum is calculated for each conditioned behavior. The prospective Z_sum values are obtained considering the positive lags, and the retrospective Z_sum values are calculated considering the negative lags. The number of positive and negative lags should be the same, and a minimum of five lags is recommended, i.e., from −1 to −5 and from +1 to +5 (Sackett, 1980).

For the prospective and retrospective Z_sum values, the relationships between focal behavior and conditioned behaviors are vectorized (Sackett, 1980). The length or radius of each vector would be

$\mathrm{Length}=\sqrt{{\left({\mathrm{Z}}_{\mathrm{sum}\mathrm{prospective}}\right)}^2+{\left({\mathrm{Z}}_{\mathrm{sum}\mathrm{retrospective}}\right)}^2}$

and the angle

$\phi =\mathrm{Arc}\mathrm{s}\mathrm{e}\mathrm{n}\frac{{\mathrm{Z}}_{\mathrm{sum}\mathrm{retrospective}}}{\mathrm{Length}}$

A vector is obtained for each conditioned behavior. The graphic origin of all vectors will be the focal behavior. Given that prospective and retrospective Z_sum values can be positive or negative, the corresponding vectors can be plotted considering that the prospective Z_sum are represented on the abscissa axis, and the retrospective Z_sum, on the order axis.

The adjusted residuals, Z values, angles, and vector lengths are obtained using HOISAN (v. 1.6.3.3) [www.menpas.com] (Hernández-Mendo et al., 2012), which calculates them from sequential multi-event type II data (Bakeman, 1978). In addition, the vector graphing was optimized using the free software R (Rodríguez-Medina et al., 2022) [https://jairodmed.shinyapps.io/HOISAN_to_R_2022/].

The vectors are interpreted based on whether the prospective and retrospective Z_sum values obtained are positive or negative, as indicated in Table 2. The significant relationships between the focal behavior and the conditioned behaviors (p < 0.05) are characterized by vector lengths >1.96, while highly significant relationships (p < 0.01) are distinguished by vector lengths >2.58.

TABLE 2

Table 2. Interpretive guideline for vectors in polar coordinate analysis.

3 Results

Intercoder reliability was κ = 0.82, which indicated satisfactory concordance. Afterwards, a polar coordinate analysis was carried out considering each code of the indirect observation instrument as a focal behavior and all the codes that comprise the instrument as conditioned behaviors.

The results corresponding to the focal behaviors A2 We seek to understand the needs of our clients (Table 3 and Figure 3), and B2 We have a common purpose (Table 4 and Figure 4) are presented for illustrative purposes and considering all the codes of the indirect observation instrument as conditioned behaviors. Tables 3, 4 present all results, regardless of whether the relationship was significant. Figures 3, 4 include only significant and highly significant vectors.

TABLE 3

Table 3. Parameters corresponding to the polar coordinate analysis, considering A2 as a focal behavior.

FIGURE 3

Figure 3. Significant vectors obtained in the polar coordinate analysis, considering A2 as a focal behavior, and all codes as conditioned behaviors.

TABLE 4

Table 4. Parameters corresponding to the polar coordinate analysis, considering B2 as a focal behavior.

FIGURE 4

Figure 4. Significant vectors obtained in the polar coordinate analysis, considering B2 as a focal behavior, and all the codes as conditioned behaviors.

When A2, We seek to understand the needs of our clients, is considered the focal behavior, it is found to have a significant relationship with A3, We readily adapt to new circumstances. There is reciprocal activation A2 $\leftrightarrow$A3 (quadrant I), showing its functional and organizational benefits for the ED. Additionally, asymmetric relationships with A7, We have good relations with the other services, and A10, We develop our skills and knowledge, were found in quadrant IV. A2 favors both behaviors (prospective Z_sum = 3.02 and 3.18 respectively). Nevertheless, A7 and A10 present an inhibitory relationship with A2 close to zero (retrospective Z_sum = −0.04 and − 0.56 respectively); i.e., they have a good relationship with the other hospital services or developing skills and knowledge does not imply that it is easy to adapt to new circumstances.

Considering B2, We have a common purpose, as a focal behavior, three significant reciprocal activations (located in quadrant I) are found with the following behaviors: B3, We have the necessary infrastructure (B2 $\leftrightarrow$B3); B6, Our service works correctly (B2 $\leftrightarrow$B6); and B20, We know what is expected of us on the job (B2 $\leftrightarrow$B20). This result is especially relevant from the point of view of shared mission and good organization.

4 Discussion

In-depth interviews are not commonly used in the implementation of programs, given that research has not indicated any productive use of them. This paper presents an adequate method to obtain detailed and thorough information based on an in-depth interview.

The mixed method approach used herein started exclusively with qualitative data (the responses to an in-depth interview) which, after being systematized, were subject to a quantitative analysis, and finally reached a qualitative interpretation of the results. Polar coordinate analysis is commonly used in direct observation. This is the first time to our knowledge that its use is presented in indirect observation and in the context of ED.

There are some limitations to the method proposed. First, substantially more time and human resources are needed for an in-depth interview than for other tools, such as questionnaires (Cano-García et al., 2017). Nevertheless, we consider that the level of detail and usefulness of the information obtained proved worthwhile. Additionally, the qualitative interpretation could prove difficult for an external evaluator, given that this person is unfamiliar with the context and details. For this reason, we recommend carrying out an internal evaluation within the organization or, if an external evaluation is needed, to work closely with stakeholders from the organization that is being evaluated.

All workers in the ED were asked to participate. As a result, we interviewed different ED professionals (emergency technicians, nurses, doctors, administrative staff and security personnel). Nonetheless, the aim of the present study was to show how qualitative and quantitative elements can be integrated through indirect observation, enabling a mixed-method approach to be used in an in-depth interview. In order to achieve this, a doctor’s interview was chosen at random from the set of interviews conducted and analyzed as an illustrative but non-representative example (Gerring, 2006). High external validity was not the main objective in this study, which is why we do not discuss the generalization of the results. It is impossible to know whether a single doctor is not representative of the doctor subgroups unless the homogeneity of the subgroup was assessed beforehand. Additionally, other ED workers would probably have a different point of view about their group’s work climate than doctors do. To obtain representativeness of the work climate in this ED, a study with multiple cases would be useful: other workers, such as nurses and orderlies, would be included, and the kappa coefficient would indicate the extent of agreement between their responses.

There are some important practical applications for implementation systems in terms of improving the functioning of organizations. Taking the case study presented herein, and considering that B2, We have a common purpose, and B6, Our service works correctly, presented significant reciprocal activation, workers should receive a clarification about the common purpose of the services their area provides if it is not deemed to be working properly (for example, if patients complain about the treatment received).

In the single-case study presented, the method proposed was illustrated in the context of an ED. Nevertheless, the method is flexible and can be adapted to contexts such as another hospital service and any other organization, as long as an informative in-depth interview can be carried out.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Scientific Ethics Committee of Universidad Autónoma de Chile. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

SC-M and MTA came up with the idea for the study. SC-M, MTA, and SS-C contributed to the design. JL-L acquired the data. MTA and JL-L conducted the analysis. All authors interpreted the results, wrote the initial draft of the manuscript, revised the article and approved the final version. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the grant PID2020-115486GB-I00 funded by the Ministerio de Ciencia e Innovación, MCIN/AEI/10.13039/501100011033, Government of Spain.

Acknowledgments

We would like to thank the doctor interviewed at the Punta Europa (Algeciras, Spain) hospital under Dr. Rodríguez Medina’s supervision, who kindly collaborated; the Generalitat de Catalunya Research Group, GRUP DE RECERCA I INNOVACIÓ EN DISSENYS (GRID). Tecnología i aplicació multimedia i digital als dissenys observacionals [grant number 2021 SGR 00718]; the support of a Chilean government project FONDECYT Regular 1190945 funded by the National Fund for Scientific and Technological Development, ANID; and the support of a Spanish government project Integración entre datos observacionales y datos provenientes de sensores externos: Evolución del software LINCE PLUS y desarrollo de la aplicación móvil para la optimización del deporte y la actividad física beneficiosa para la salud [EXP_74847] (2023), Ministerio de Cultura y Deporte, Consejo Superior de Deporte and European Union.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1247577/full#supplementary-material

References

Allison, P. D., and Liker, J. K. (1982). Analyzing sequential categorical data on dyadic interaction: a comment on Gottman. Psychol. Bull. 91, 393–403. doi: 10.1037/0033-2909.91.2.393

CrossRef Full Text | Google Scholar

Altuntaş, S., Seren İntepeler, Ş., Sökmen, S., Kantek, F., Öztürk, H., and Baykal, Ü. (2021). The effect of ethical work climate on the organizational citizenship behavior of academic nurses. Int. Nurs. Rev. 68, 15–23. doi: 10.1111/inr.12622

CrossRef Full Text | Google Scholar

Alvarado-Alvarez, C., Armadans, I., Parada, M. J., and Anguera, M. T. (2021). Unraveling the role of shared vision and trust in constructive conflict management of family firms. An empirical study from a mixed methods approach. Front. Psychol. :12. doi: 10.3389/fpsyg.2021.629730

CrossRef Full Text | Google Scholar

Anguera, M. T. (1997). From prospective patterns in behavior to joint analysis with a retrospective perspective. In Méthodologie d’analyse des interactions sociales (Paris, France: Université de la Sorbonne).

Google Scholar

Anguera, M. T. (2018). “Del Caso único al Caso múltiple en el estudio del comportamiento humano [from single case to multiple case in the study of human behavior]” in Academia de Psicología de España, Psicología para un mundo sostenible (Pirámide), 31–50.

Google Scholar

Anguera, M. T. (2020). “Is it possible to perform ‘liquefying’ actions in conversational analysis? The detection of structures in indirect observations” in The temporal structure of multimodal communication. Intelligent systems reference library. eds. L. Hunyadi and I. Szekrényes (London: Springer)

Google Scholar

Anguera, M. T. (2021). “De la entrevista en profundidad a la observación sistemática: una pincelada de comunicación multimodal desde los mixed methods [From the in-depth interview to the systematic observation: a brushstroke of multimodal communication from the mixed methods]” in Estudios en homenaje a Emigdia Repetto Jiménez. eds. V. M. Hernández-Suárez, P. Calvo-Hernández, and J. R. Suárez-Robaina (Universidad de Las Palmas de Gran Canaria, Servicio de Publicaciones)

Google Scholar

Anguera, M. T., and Hernández-Mendo, A. (2016). Avances en estudios observacionales en Ciencias del Deporte desde los mixed methods [Advances in mixed methods observational studies in sports sciences]. Cuadernos de Psicología del Deporte, 16, 17–30.

Google Scholar

Anguera, M. T., Blanco-Villaseñor, A., Losada, J. L., and Portell, M. (2018a). Pautas para elaborar trabajos que utilizan la metodología observacional [Guidelines for preparing papers that use observational methodology]. Anuario de Psicología 48, 9–17. doi: 10.1016/j.anpsic.2018.02.001

CrossRef Full Text | Google Scholar

Anguera, M. T., Blanco-Villaseñor, A., Losada, J. L., Sánchez-Algarra, P., and Onwuegbuzie, A. J. (2018b). Revisiting the difference between mixed methods and multimethods: is it all in the name? Qual. Quant. 52, 2757–2770. doi: 10.1007/s11135-018-0700-2

CrossRef Full Text | Google Scholar

Anguera, M. T., Jonsson, G. K., and Sánchez-Algarra, P. (2017). Liquefying text from human communication processes: a methodological proposal based on T-Pattern detection. J. Multimod. Commun. Stud. 1–2, 10–15.

Google Scholar

Anguera, M. T., Portell, M., Chacón-Moscoso, S., and Sanduvete-Chaves, S. (2018c). Indirect observation in everyday contexts: concepts and methodological guidelines within a mixed methods framework. Front. Psychol. 9:e13. doi: 10.3389/fpsyg.2018.00013

PubMed Abstract | CrossRef Full Text | Google Scholar

Anguera, M. T., Portell, P., Hernández-Mendo, A., Sánchez-Algarra, P., and Jonsson, G. K. (2021). “Diachronic analysis of qualitative data” in Reviewer’s guide for mixed methods research analysis. eds. A. J. Onwuegbuzie and B. Johnson, 125–138.

Google Scholar

Arias-Pujol, E., and Anguera, M. T. (2020). A mixed methods framework for psychoanalytic group therapy: from qualitative records to a quantitative approach using t-pattern, lag sequential, and polar coordinate analyses. Front. Psychol. 11:922. doi: 10.3389/fpsyg.2020.01922

PubMed Abstract | CrossRef Full Text | Google Scholar

Arias-Pujol, E., Mestres, M., Miralbell, J., Bachs, N., and Anguera, M. T. (2022). Implementation and evaluation in low intensity intervention programs from the CONNECT perspective of mixed methods: application in a case of an autistic child. Front. Psychol. 13:658. doi: 10.3389/fpsyg.2022.998658

CrossRef Full Text | Google Scholar

Bakeman, R. (1978). “Untangling streams of behavior: sequential analysis of observational data” in Observing behavior, 2 data collection and analysis method. ed. G. P. Sackett (Pennsylvania: University Park Press), 63–78.

Google Scholar

Barth, J., Greene, J. A., Goldstein, J., and Sibley, A. (2022). Adverse health effects related to shift work patterns and work schedule tolerance in emergency medical services personnel: a scoping review. Cureus 14:e23730. doi: 10.7759/cureus.23730

PubMed Abstract | CrossRef Full Text | Google Scholar

Belza, H., Herrán, E., and Anguera, M. T. (2019). Early childhood education and cultural learning: systematic observation of the behaviour of a caregiver at the Emmi Pikler nursery school during breakfast / Educación temprana y aprendizaje cultural: observación sistemática de la conducta de la educadora Pikler durante el desayuno. Infancia y Aprendizaje 42, 128–178. doi: 10.1080/02103702.2018.1553268

CrossRef Full Text | Google Scholar

Bijani, M., Abedi, S., Karimi, S., and Tehranineshat, B. (2021). Major challenges and barriers in clinical decision-making as perceived by emergency medical services personnel: a qualitative content analysis. BMC Emerg. Med. 21:11. doi: 10.1186/s12873-021-00408-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Birdwhistell, R. L. (1970). Kinesics and context: Essays and body motion communication. Pennsylvania: University of Pennsylvania Press.

Google Scholar

Blanco-Villaseñor, A., Losada, J. L., and Anguera, M. T. (2003). Data analysis techniques in observational designs applied to the environment-behaviour relation. Medio Ambiente y Comportamiento Humano 4, 111–126.

Google Scholar

Cano-García, F. J., González-Ortega, M. C., Sanduvete-Chaves, S., Chacón-Moscoso, S., and Moreno-Borrego, R. (2017). Evaluation of a psychological intervention for patients with chronic pain in primary care. Front. Psychol. 8:435. doi: 10.3389/fpsyg.2017.00435

CrossRef Full Text | Google Scholar

Chacón-Moscoso, S., Anguera, M. T., Sanduvete-Chaves, S., Losada, J. L., Lozano-Lozano, J. A., and Portell, M. (2019). Methodological quality checklist for studies based on observational methodology (MQCOM). Psicothema 31, 458–464. doi: 10.7334/psicothema2019.116

PubMed Abstract | CrossRef Full Text | Google Scholar

Chacón-Moscoso, S., Anguera, M. T., Sanduvete-Chaves, S., and Sánchez-Martín, M. (2014). Methodological convergence of program evaluation designs. Psicothema 26, 91–96. doi: 10.7334/psicothema2013.144

PubMed Abstract | CrossRef Full Text | Google Scholar

Chacón-Moscoso, S., Sanduvete-Chaves, S., Anguera, M. T., Losada, J. L., Portell, M., and Lozano-Lozano, J. A. (2018). Preliminary checklist for reporting observational studies in sports areas: content validity. Front. Psychol. 9:291. doi: 10.3389/fpsyg.2018.00291

PubMed Abstract | CrossRef Full Text | Google Scholar

Chacón-Moscoso, S., Sanduvete-Chaves, S., Lozano-Lozano, J. A., Portell, M., and Anguera, M. T. (2021). From randomized control trial to mixed methods: A practical framework for program evaluation based on methodological quality. Anales de Psicología, 37, 599–608. doi: 10.6018/analesps.470021

CrossRef Full Text | Google Scholar

Chacón-Moscoso, S., Sanduvete-Chaves, S., Lozano-Lozano, J. A., and Holgado, F. P. (2023). The Methodological Quality Scale (MQS) for intervention programs: Validity evidence. Frontiers in Psychology, 14:1217661. doi: 10.3389/fpsyg.2023.1217661

CrossRef Full Text | Google Scholar

Champagne, F. (2014). Organizational impact of evidence-informed decision making training initiatives: a case study comparison of two approaches. Implement. Sci. 9:1. doi: 10.1186/1748-5908-9-53

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46. doi: 10.1177/001316446002000104

CrossRef Full Text | Google Scholar

Creswell, J. W., and Plano Clark, V. L. (2011). Designing and conducting mixed methods research. 2nd London: Sage.

Google Scholar

del Giacco, L., Anguera, M. T., and Salcuni, S. (2020). The action of verbal and non-verbal communication in the therapeutic alliance construction: a mixed methods approach to assess the initial interactions with depressed patients. Front. Psychol. 11:234. doi: 10.3389/fpsyg.2020.00234

PubMed Abstract | CrossRef Full Text | Google Scholar

García-Fariña, A., Jiménez-Jiménez, F., and Anguera, M. T. (2018). Observation of communication by physical education teachers: detecting patterns in verbal behavior. Front. Psychol. 9:334. doi: 10.3389/fpsyg.2018.00334

CrossRef Full Text | Google Scholar

Gerring, J. (2004). What is a case study and what is it good for? Am. Polit. Sci. Rev. 98, 341–354. doi: 10.1017/S0003055404001182

CrossRef Full Text | Google Scholar

Gerring, J. (2006). Single-outcome studies: a methodological primer. Int. Sociol. 21, 707–734. doi: 10.1177/0268580906067837

CrossRef Full Text | Google Scholar

Hepburn, A., and Bolden, G. B. (2013). “The conversation analytic approach to transcription” in The handbook of conversation analysis. eds. J. Sidnell and T. Stivers (Hoboken: Wiley-Blackwell)

Google Scholar

Hernández-Mendo, A., López-López, J. A., Castellano-Paulis, J., Morales-Sánchez, V., and Pastrana-Brincones, J. L. (2012). HOISAN 1.2: programa informático Para uso en metodología observacional [Hoisan 1.2: software for observational methodology]. Cuadernos de Psicología del Deporte 12, 55–78. doi: 10.4321/S1578-84232012000100006

CrossRef Full Text | Google Scholar

Holgado-Tello, F. P., Chacón-Moscoso, S., Sanduvete-Chaves, S., and Pérez-Gil, J. A. (2016). A simulation study of threats to validity in quasi-experimental designs: Interrelationship between design, measurement, and analysis. Frontiers in Psychology, 7:897. doi: 10.3389/fpsyg.2016.00897

CrossRef Full Text | Google Scholar

Holzinger, F., Oslislo, S., Resendiz Cantu, R., Möckel, M., and Heintze, C. (2021). Diverting less urgent utilizers of emergency medical services to primary care: is it feasible? Patient and morbidity characteristics from a cross-sectional multicenter study of self-referring respiratory emergency department consulters. BMC. Res. Notes 14:113. doi: 10.1186/s13104-021-05517-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Hunziker, S., and Blankenagel, M. (eds) (2021). “Single case research design” in Research Design in Business and Management (Wiesbaden: Springer Fachmedien Wiesbaden)

Google Scholar

Iván-Baragaño, I., Ardá, A., Anguera, M. T., Losada, J. L., and Maneiro, R. (2023). Future horizons in the analysis of technical-tactical performance in women’s football: a Mixed Methods approach to the analysis of in-depth interviews with professional coaches and players. Frontiers in Psychology, 14:1128549. doi: 10.3389/fpsyg.2023.1128549

CrossRef Full Text | Google Scholar

Kitzmiller, R. (2010). Making sense of health information technology implementation: a qualitative study protocol. Implement. Sci. 5:1. doi: 10.1186/1748-5908-5-95

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, H., Xu, Y., Jing, M., Wei, X., Li, L., and Guo, Y. (2022). The mediating effects of adversity quotient and self-efficacy on ICU nurses’ organizational climate and work engagement. J. Nurs. Manag. 30, 3322–3329. doi: 10.1111/jonm.13773

PubMed Abstract | CrossRef Full Text | Google Scholar

Lozano-Lozano, J. A., Chacón-Moscoso, S., Sanduvete-Chaves, S., and Holgado-Tello, F. P. (2021). Work climate scale in emergency services: abridged version. Int. J. Environ. Res. Public Health 18:6495. doi: 10.3390/ijerph18126495

PubMed Abstract | CrossRef Full Text | Google Scholar

Maneiro, R., Amatria, M., and Anguera, M. (2020). Diachronic analysis application for the detection of soccer performance standards: a case study. Int J Comput Sci Sport 19, 77–109. doi: 10.2478/ijcss-2020-0011

CrossRef Full Text | Google Scholar

Marshall, C., Rossman, G. B., and Blanco, G. L. (2022). Designing qualitative research. London: Sage.

Google Scholar

Merriam, S. B., and Tisdell, E. J. (2016). Qualitative research: a guide to design and implementation. San Francisco, CA: Jossey-Bass.

Google Scholar

Moya, J., Anguera, M. T., Fornells, E., de Armas, M., and Catalá, M. (2020). Análisis prospectivo de los cambios en las relaciones interpersonales en el contexto de la pandemia COVID-19 [prospective analysis of changes in interpersonal relationships in the context of the COVID-19 pandemic]. Available at: https://raed.academy/wp-content/uploads/2021/01/Analisis-prospectivo-del-futuro-de-las-relaciones-interpersonales-en-el-contexto-de-la-Covid-19-v.26-cmpr.pdf (Accessed November 30, 2023).

Google Scholar

Mozgovoy, V. (2022). Does psychological work environment matter? Relationship between psychological climate reflecting individual perception of work environment and stress among public servants. Cogent Bus. Manag. 9:2010483. doi: 10.1080/23311975.2021.2010483

CrossRef Full Text | Google Scholar

Nunes, H., Iglesias, X., del Giacco, L., and Anguera, M. T. (2022). The pick-and-roll in basketball from deep interviews of elite coaches: a mixed method approach from polar coordinate analysis. Front. Psychol. 13:1100. doi: 10.3389/fpsyg.2022.801100

PubMed Abstract | CrossRef Full Text | Google Scholar

Perry, C., LeMay, N., Rodway, G., Tracy, A., and Galer, J. (2005). Validating a work group climate assessment tool for improving the performance of public health organizations. Hum. Resour. Health 3:10. doi: 10.1186/1478-4491-3-10

PubMed Abstract | CrossRef Full Text | Google Scholar

Porcheret, M. (2014). Development of a behaviour change intervention: a case study on the practical application of theory. Implement. Sci. 9:1. doi: 10.1186/1748-5908-9-42

PubMed Abstract | CrossRef Full Text | Google Scholar

Portell, M., Anguera, M. T., Chacón-Moscoso, S., and Sanduvete-Chaves, S. (2015). Guidelines for reporting evaluations based on observational methodology. Psicothema 27, 283–289. doi: 10.7334/psicothema2014.276

CrossRef Full Text | Google Scholar

Portell, M., Sene-Mir, A. M., Anguera, M. T., Jonsson, G. K., and Losada, J. L. (2019). Support system for the assessment and intervention during the manual material handling training at the workplace: contributions from the systematic observation. Front. Psychol. 10:1247. doi: 10.3389/fpsyg.2019.01247

PubMed Abstract | CrossRef Full Text | Google Scholar

Rodríguez-Medina, J., Arias, V., Arias, B., Hernández-Mendo, A., and Anguera, M. T. (2022). Polar coordinate analysis, from HOISAN to R: a tutorial paper. GitHub repository. Available at: https://jairodmed.shinyapps.io/HOISAN_to_R_2022/ (Accessed November 30, 2023).

Google Scholar

Rodríguez-Medina, J., Rodríguez-Navarro, H., Arias, V., Arias, B., and Anguera, M. T. (2018). Non-reciprocal friendships in a school-age boy with autism: the ties that build? J. Autism Dev. Disord. 48, 2980–2994. doi: 10.1007/s10803-018-3575-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Rubin, H., and Rubin, I. (2005). Qualitative interviewing (2nd ed.): The art of hearing data. London: SAGE.

Google Scholar

Sackett, G. P. (1980). “Lag sequential analysis as a data reduction technique in social interaction research” in Exceptional infant. Psychosocial risks in infant-environment transactions. eds. D. B. Sawin, R. C. Hawkins, L. O. Walker, and J. H. Penticuff (Levittown: Brunner/Mazel)

Google Scholar

Sandelowski, M., Voils, C. I., and Knafl, G. (2009). On quantitizing. J. Mix Methods Res. 3, 208–222. doi: 10.1177/1558689809334210

PubMed Abstract | CrossRef Full Text | Google Scholar

Sanduvete-Chaves, S., Barbero, M. I., Chacón-Moscoso, S., Pérez-Gil, J. A., Holgado-Tello, F. P., Sánchez-Martín, M., et al. (2009). Scaling methods applied to set priorities in training programs in organizations. Psicothema 21, 509–514.

PubMed Abstract | Google Scholar

Sanduvete-Chaves, S., Lozano-Lozano, J. A., Chacón-Moscoso, S., and Holgado-Tello, F. P. (2018). Development of a work climate scale in emergency health services. Front. Psychol. 9:e10. doi: 10.3389/fpsyg.2018.00010

PubMed Abstract | CrossRef Full Text | Google Scholar

Santana, S., and Pérez-Rico, C. (2023). Dynamics of organizational climate and job satisfaction in healthcare service practice and research: a protocol for a systematic review. Front. Psychol. 14:1186567. doi: 10.3389/fpsyg.2023.1186567

PubMed Abstract | CrossRef Full Text | Google Scholar

Schegloff, E. A. (2000). On granularity. Annu. Rev. Sociol. 26, 715–720. doi: 10.1146/annurev.soc.26.1.715

CrossRef Full Text | Google Scholar

Schegloff, E. A., and Sacks, H. (1973). Opening up closings. Semiotica 8, 289–327. doi: 10.1515/semi.1973.8.4.289

CrossRef Full Text | Google Scholar

Squires, J. E., Estabrooks, C. A., O’Rourke, H. M., Gustavsson, P., Newburn-Cook, C., and Wallin, L. (2011). A systematic review of the psychometric properties of self-report research utilization measures used in healthcare. Implement. Sci. 6:1. doi: 10.1186/1748-5908-6-83

PubMed Abstract | CrossRef Full Text | Google Scholar

Stetler, C. (2009). Institutionalizing evidence-based practice: an organizational case study using a model of strategic change. Implement. Sci. 4:1. doi: 10.1186/1748-5908-4-78

CrossRef Full Text | Google Scholar

Taylor, A. M., Cocks, J. H., Ashe, M. C., Van Hooff, M., and Lawn, S. (2023). Mental health in young adult emergency services personnel: a rapid review of the evidence. Int. J. Disast. Risk Reduct. 88:103605. doi: 10.1016/j.ijdrr.2023.103605

CrossRef Full Text | Google Scholar

Trebach, J. D., Levy, M., Ali, F., Beauchamp, G., Biary, R., Everett, C., et al. (2021). Establishing consensus-based objectives for the creation of an opioid overdose curriculum for emergency medical services clinicians. AEM Educ. Train. 5:e10499. doi: 10.1002/aet2.10499

PubMed Abstract | CrossRef Full Text | Google Scholar