Educational model transition: Student evaluation of teaching amid the COVID-19 pandemic

Campos, Esmeralda; Daruich, Sandra Dennis Núñez; de la O, Jose Francisco Enríquez; Castaño, Raquel; Escamilla, Jose; Hosseini, Samira

doi:10.3389/feduc.2022.991654

ORIGINAL RESEARCH article

Front. Educ., 31 October 2022

Sec. Higher Education

Volume 7 - 2022 | https://doi.org/10.3389/feduc.2022.991654

This article is part of the Research Topic Education and Society: New Approaches for New Challenges View all 16 articles

Educational model transition: Student evaluation of teaching amid the COVID-19 pandemic

$\r\nEsmeralda Campos&#x;$ Esmeralda Campos^1†

Sandra Dennis Núñez Daruich²

Jose Francisco Enríquez de la O³

Raquel Castaño⁴

Jose Escamilla⁵

Samira Hosseini^1,6*

¹Writing Lab, Institute for the Future of Education, Tecnológico de Monterrey, Monterrey, Mexico
²Department of Teacher Training, ECOA National and Institutional Effectiveness Department, Tecnológico de Monterrey, Monterrey, Mexico
³Design and Experience Faculty, ECOA National and Institutional Effectiveness Department, Tecnológico de Monterrey, Monterrey, Mexico
⁴Vicerrectoría de Facultad, Vicerrectoría de Desarrollo de la Facultad, Tecnológico de Monterrey, Monterrey, Mexico
⁵Institute for the Future of Education, Tecnológico de Monterrey, Monterrey, Mexico
⁶School of Engineering and Sciences, Tecnológico de Monterrey, Monterrey, Mexico

The Education 4.0 Framework calls for Higher Education Institutions (HEIs) to innovate their curriculum for developing the competencies of the future. Tecnologico de Monterrey started a transition from an active-learning educational model to Tec21, a challenge-based learning educational model focused on competency development. After one semester of this transition, the COVID-19 pandemic disrupted education worldwide, causing most universities to adapt to online education. We found the opportunity to analyze the institutional Student Evaluation of Teaching (SET) survey at different stages of the COVID-19 pandemic, prior to the outbreak, in the transition to online learning, and after the transition to fully online course delivery. We performed this analysis separately for the two coexisting educational models and each of the schools at the university. We also compared the SET scores for the spring semester of 2021, when the two educational models had a comparable number of students. We found that SET scores were not negatively impacted by the COVID-19 pandemic hinting toward positive implications about the institutional response to the pandemic. Another finding is that the Tec21 educational model has received higher SET scores, which implies a positive perception by students. There were a few exceptions to these results, which we address explicitly; for instance, the COVID-19 pandemic might have affected SET scores in the School of Medicine and Health Sciences. Further research is necessary to evaluate the implementation of the Tec21 model comprehensively.

Introduction

The Fourth Industrial Revolution emphasizes the need for digital and socio-emotional skills development. However, most education worldwide is still satisfying the needs demanded by the First and Second Industrial Revolutions (World Economic Forum, 2020). Higher Education Institutions (HEIs) must respond to the needs of competency development for preparing future professionals with disciplinary content as well as digital and socio-emotional skills, within the Education 4.0 Framework. Innovative pedagogies, such as experiential learning, are key approaches for driving innovation in educational systems (World Economic Forum, 2020). Universities in different parts of the world have responded to the needs posed by the Education 4.0 Framework by presenting proposals for change and their respective analyses (Buasuwan, 2018; Vu, 2018; Ovinova and Shraiber, 2019; Sudaryono et al., 2019), and by implementing new educational models or pedagogies (Anaraki, 2018; Muawiyah et al., 2018; Ab Rahman et al., 2019; Almeida and Simoes, 2019; Anito and Morales, 2019). The role of HEI is to adapt to the needs of the future, while responding to the present context, highly disrupted by the COVID-19 pandemic.

Higher Education Institutions worldwide responded differently to the COVID-19 pandemic. Developed economies in Europe generally reported campus closure and moving to online teaching, while not all HEIs in the United Stated of America did so. In developing economies, campus closure and switching to online learning depended on the country’s policy, the preparedness of faculty and students’ anxiety levels (Sasere and Makhasane, 2020). For example, in Brazil, upon the onset of the pandemic authorities suspended face-to-face activities in HEIs (Gisela Biberg-Salum et al., 2020). Several HEIs reported their support and evaluation practices for changing from face-to-face to remote learning. Moreover, faculty needed preparation in pedagogical approach and technological support for adapting their educational content and delivery to online platforms (Quezada et al., 2020). The University of Turin evaluated the support provided during the emergency transition from face-to-face to online teaching, exploring influential or unfavorable elements (Floris et al., 2020). A model based on the response of three European HEIs to COVID-19 shows that IT use and practices pass through different phases of exploration and exploitation, finding that these phases present challenges that can be alleviated with well-timed management (Carugati et al., 2020). A study in Europe about anatomy education emphasizes the strengths, weaknesses, opportunities, and threats of adapting to distance learning (Longhurst et al., 2020). They consider the development of new online resources and upskilling in new technologies and resources as organizational strengths; time constraints, lack of practical sessions and issues with assessment as organizational weaknesses; academic collaboration, working remotely and the incorporation of blended learning in future curriculum development as environmental opportunities, and reduced student engagement and diminished professor-student relationship as environmental threats. A study performed in Spain analyzed how students adapted to the situation posed by the COVID-19 pandemic and found a lack of preparedness and facing difficulties with the online implementation which have translated to additional effort required by students and lower academic performance (Faura-Martínez et al., 2022). A study in Saudi Arabia gathered evidence on the positive impact of using social media on students’ academic performance during the COVID-19 pandemic highlighting the perceived usefulness, ease of use and enjoyment (Alismaiel et al., 2022). Additionally, the topic has been of interest for meta-analyses and review studies that analyze and summarize a vast number of articles that have tackled HEIs institutional response to the COVID-19 pandemic (Karakose, 2021; Zhang et al., 2022).

In the university where this study takes place, the educational model is shifting from a traditional active-learning approach to a competency-based model (known as Tec21) as an attempt to contribute to competency development considering the Education 4.0 Framework. The implementation of the Tec21 educational model happened almost simultaneously with the COVID-19 pandemic. Extensive research has studied the implementation of the Tec21 model during the COVID-19 pandemic, specially at its intersection with the Education 4.0 Framework. One of such studies proposes that transforming the concept of sustainability into a transversal competency present in all academic programs enables students to focus their knowledge on actions that lead to sustainable development (Membrillo-Hernández et al., 2021). Another study arguments that only through the design of flexible, interactive and technology-centered courses based on the Fourth Industrial Revolution can academia prepare students for the challenges of the future (Membrillo-Hernández et al., 2019). Another study presented a Capstone project focusing on the intersection between the Tec21 Model and the Education 4.0 Framework, where students proposed solutions to a specific problem of the automotive industry (López et al., 2021). This project showcases the nature of the Tec21 Model because it places students in a challenging experience directly linked with the public and private sectors, in innovative spaces and resulting in the development of disciplinary and transversal competencies. The authors highlighted that some students were connected to high-end companies, such as Tesla, to make internships as a result of their competency development in this project. In a different project, students solved the challenge of improving the energetic efficiency in the artisanal production of an alcoholic beverage in Mexico (Reyna-González et al., 2020) leading to the development of disciplinary and transversal competencies in students, as well as a vision for applying engineering solutions to social problems. A case study about interactive education reported that the use of interactive education improves the way of teaching, reinforces competencies, and enhances students’ creativity and motivation (López Ríos et al., 2020). Along these lines, a case study presented the inclusion of artificial intelligence as an educational strategy implemented in the Tec21 model for the digital transformation required by the Education 4.0 Framework (Cantú-Ortiz et al., 2020).

We aim to add to this research from the perspective of SET. While some of the cited studies take a look at the evolution of Tec21 through the lens of SET (Membrillo-Hernández et al., 2019; Reyna-González et al., 2020) no previous study has comprehensively evaluated the institutional SET including students of all schools and courses, and as a means for comparison with the traditional model. The specific objective of this study is therefore to compare how students evaluate their professors’ performance with respect to the educational innovation that intersects with the Education 4.0 Framework, the Tec 21 model, and the traditional educational model. Since this transition has happened mostly amid the COVID-19 pandemic, we also aim to analyze the potential shift in students’ opinion of their professors’ performance in three different periods of the pandemic: pre-COVID-19, trans-COVID-19 (which stands for the transition to online teaching and learning) and intra-COVID-19 (during the pandemic, since by June 2021 the world was not entirely free of this global challenge). We aim to answer the research questions as follows: (1) Do students in the traditional and the Tec21 educational model evaluate their professors differently in the SET survey at different stages of the pandemic (pre-, trans- and intra-COVID-19)? (2) Do students in the Tec21 educational model evaluate their professors differently than students in the traditional educational model?

Study context

Student evaluation of teaching

Evaluating students’ experience with the transformations that take place in face of the Education 4.0 Framework and the COVID-19 pandemic is an important task. There are multiple ways to evaluate students’ experience, such as satisfaction surveys and conducting in-depth studies of specific implementations. One way of evaluating their experience is through institutional Student Evaluation of Teaching (SET) surveys. The validity of SET use for making academic decisions has been criticized due to different biases, such as student’s and teacher’s gender, student’s age, teacher’s experience (Spooren et al., 2013), and cultural bias (Arnold and Versluis, 2019), among others. Other studies have found that SET surveys can be useful for understanding the students’ experience, rather than satisfaction (Warfvinge et al., 2021). Different studies that use SET scores in the context of experiential learning before the pandemic have found positive evaluations. Problem-based learning with a client based and a non-client-based approach was overall highly rated, except for the question related to students’ motivation to learn in the client-based projects (Appiah-Kubi, 2018). The authors attribute this to the fact that client-based projects may require a lot of time and can be unstructured resulting in students being unable to control changes. The implementation of a Global Experience, an innovative program designed to broaden students’ engagement through international experiential learning, in South Australia resulted in positive evaluations in SET, among other instruments (Feast et al., 2011).

In face of the COVID-19 pandemic, SET scores have been used to study whether the change to online teaching affected the way students evaluated faculty. There are some mixed results; for instance, when shifting from in-person to online teaching, the SET scores lowered for faculty in communication courses (LeBlanc, 2021). Regarding gender bias, several studies have found no significant gender-related differences in students’ evaluation of teaching in different contexts (Arrona-Palacios et al., 2020; Tangalakis et al., 2022). However, in the COVID-19 pandemic, male students make more negative comments about their female professors’ teaching style (Tangalakis et al., 2022). On the positive side, studies have reported that overall student evaluation was not affected by the changes in instruction or life experiences outside of the professor’s control (Boysen, 2020). Moreover, the factors that have affected SET during the pandemic are the professors’ educational skills, personal and ethical characteristics and their views on principles and rules (Sepahi et al., 2021). Some professors found the change to online teaching amid the pandemic as a challenge that would allow them to grow, while others perceived it as a threat. These attitudes are related to burnout levels and student evaluation of teaching (Daumiller et al., 2021). During the pandemic, professors found the feedback received from SETs useful for improving their teaching practices and meeting students’ needs (Silalahi, 2021).

Institutional educational model

We present some of the characteristics that differentiate the competency-based educational model (Tec21) from the traditional active-learning model regarding pedagogical approach. The pedagogical approach in the traditional educational model was lecture-based with active learning components. Students would take 16-week long courses in the semester, presenting their final exams in the last 2 weeks of the semester. In general, undergraduate courses are comprised of 3 lecture hours and 5 coursework hours per week. Some courses include laboratories, when applicable, as part of their coursework hours (with no extra credit for students). The objectives for each course is to gain disciplinary knowledge or develop skills necessary for their future careers. Professors are encouraged to include active learning strategies and collaborative work, in addition to using digital technologies for educational innovation. Professors have access to continuous learning tools focused on the use of a wide range of digital technologies (e.g., virtual reality, interactive videos) and active-learning strategies (e.g., problem-based learning, tutorials) (Vicerrectoría de Facultad, 2022).

In the Tec21 educational model, the pedagogical design and approach changes its focus from content-based knowledge to competency development (Tecnologico de Monterrey, 2018). This change involves a reconfiguration of courses into 5- or 10-week blocks. The semester is restructured into three 5-week periods with immersive weeks after each period (a total of 18 weeks). The blocks can take one or two periods, and, in general, take from 4 to 12 h of synchronous learning per week. The objective of each block is defined through disciplinary and transversal competency development, which is achieved through solving challenges associated with real world problems in their fields of study. Challenge-based learning is at the core of the Tec21 model. Each block requires a solution to a challenge directly related to the disciplinary and transversal competencies that would be developed within the respective block. The block is structured into learning modules that also include the learning content to provide students with the theoretical and practical knowledge needed to solve the challenge.

The shift from the traditional to the Tec21 educational model is ongoing. A few semesters before the rollout of the Tec21 educational model, between August 2016 and June 2019, a few select programs implemented the flexibility of degree choice by defining Career Paths. These implementations were at the early stages of the educational model. In August 2019, the Tec21 educational model officially started for the incoming students in all undergraduate programs across 26 campuses of the university. The students who had initially enrolled to the traditional educational model would continue their degree in the same format but were given the option to enroll in the Tec21 educational model. During the first semester of the academic year 2019-2020 all first-year students were enrolled in the Tec21 model, while second, third- and fourth-year students continued studying in the traditional model. The full shift to the new model will take place when all the students who enrolled to the traditional model before August 2019 complete their studies. During this period (presumably, from 2019 to 2023), the two educational models will coexist; however, the number of students in the traditional model decreases, as that of Tec21 model increases.

Institutional response to the COVID-19 pandemic

Pre-COVID-19

In August 2019, the institution offered the first full implementation of the Tec21 educational model. During this semester, students in the traditional educational model were taking courses from their second, third or fourth year of studies, while students in the Tec21 educational model entered their first year. In this study, we refer to the semester August-December 2019 (AD2019) as pre-COVID-19, as this was the only semester when the two educational models coexisted before the pandemic stroke. All activities and courses during this semester occurred in person. There is a solid body of data about the semesters prior to AD2019, and with respect to the traditional educational model hence minimal premise for comparison between models. The data obtained with students from the traditional educational model has been analyzed and presented in previous studies (Arrona-Palacios et al., 2020; Okoye et al., 2020).

Trans-COVID-19

The COVID-19 pandemic was declared on March 11, 2020, disrupting educational activities worldwide. The following day, Tecnologico de Monterrey suspended all activities for the upcoming week, from March 13 to 20, with the aim to resume activities fully online on March 23 with the virtual model HyFlex + Tec. The academic semester which started in February would end in June 2020 (FJ2020). The educational activities at the beginning of the semester until March 13 were held in-person at every campus of the institution. After the declaration of the pandemic, the involved stakeholders (students, professors, directors, administrators and policymakers) had to transition their entire operation to the HyFlex + Tec in a matter of 1 week. To highlight the transition that took place in the semester FJ2020, we named this stage as the “Trans-COVID-19.” The educational technologies mainly employed in this transition were Zoom for synchronous course delivery and Canvas or Blackboard as the Learning Management System. Both tools were previously available to the professors through their institutional access and approximately 80% of professors were familiar with these tools. Nevertheless, the institution provided with appropriate training during the 1-week pause of activities to ensure an adequate transition for all professors and students. Furthermore, all courses were assigned an academic helper called “Academic Buddy,” who were university staff to provide technical support to professors in their online experience. At this period of the pandemic, the university estimated over 55,000 class sessions to be delivered per week, anticipating the need to adapt and make the necessary adjustments to face the challenges imposed by COVID-19. The university monitored the emotional well-being of the community throughout the pandemic, identifying the most frequently reported feelings of students being anxious, stressed, overwhelmed, tired or even depressed (Camacho-Zuñiga et al., 2021).

Intra-COVID-19

During the fall semester of 2020 (AD2020), the university continued delivering online classes. As the vaccination against COVID-19 progressed, the institution launched a program to consciously return to in-person activities known as the Hybrid in-Person + Remote Simultaneous (HPRS) model. Across 26 campuses, those with reduced number of infection reports were allowed to return to in-person activities, following strict protocols in a controlled hybrid environment. To this end, the university provided adequate infrastructure and training to the professors for having students both in person and online simultaneously. This program began in the spring semester of 2021 (FJ2021) with only a few campuses delivering in-person classes slowly increasing the number of in-person activities. The transition is still ongoing. Since the challenges of the pandemic were still ongoing, we refer to the semesters AD2020 and FJ2021 as “Intra-COVID-19.”

Methodology

At the moment of this study, we are presumably at the midpoint of the transition between the two educational models. The pandemic may have affected the way students evaluate their professors due to several factors including emotions, students’ opinion regarding the institutional response to the COVID-19 pandemic, and professors’ ability to shift to online education, among others. Considering this scenario, we present a study that analyses students’ evaluation of teaching comparing pivotal moments relevant to the transition from in-person to online education and from the traditional to the Tec21 educational model. The transition from in-person to online education is analyzed for the two models independently, while the transition from the traditional to the Tec21 model is analyzed for the semester of FJ2021, when the two populations are comparable in size.

Data collection

The participants of this study were the undergraduate students of large, multi-campus, private Mexican university, Tecnologico de Monterrey. The instrument used in this study is the institutional SET, Student Opinion Survey (ECOA, for its acronym in Spanish). This survey was designed to collect students’ opinion regarding their professors’ academic performance. Students evaluated their professors in an interval scale from 0 to 10. The evaluation included the quality of the course delivery, the professors’ competencies and other academic services offered by directors and departments. The survey was designed and delivered in Spanish, the main language in the institution. This instrument has shown acceptable reliability in previous studies (Arrona-Palacios et al., 2020). The survey included several evaluation criteria to assess different aspects of the professors’ academic performance including course delivery methodology, knowledge and experience, guidance and feedback, course grading and evaluation, and an overall evaluation of professors’ performance.

A total of 35,840 students answered the ECOA survey during four semesters, which represents an average of 68% of the students across the four semesters (58% in AD2019, 78% in FJ2020, 74% in AD2020, and 63% in FJ2021). The survey was applied to students across School of Engineering and Sciences, Business School, School of Social Science and Government, School of Medicine and Health Sciences, School of Humanities and Education, and School of Architecture, Art and Design. Table 1 summarizes the number of participants in each semester, schools and the respective educational model. For reminder, pre-COVID-19 was the semester prior to the COVID-19 pandemic with full in-person learning; trans-COVID-19 was the semester when the COVID-19 pandemic started and the transition to online learning was implemented; and intra-COVID-19 were two semesters after the transition to online learning. The survey was administered in the last week of classes and before the end of the semester. The data collection was completely anonymous, complying with the principles of the Declaration of Helsinki of research with human participants. This research has received the ethical approvals from the Office of the Vice-rectory of Investigation, Tecnologico de Monterrey, Mexico.

TABLE 1

Table 1. Summary of participants in each educational model and school in the pre-, trans, and intra-COVID-19 semesters.

This study focuses on the final question of the survey since it provides students’ overall opinion regarding their professors’ performance. The phrasing of the final question is different for students in the traditional educational model than in the Tec21 educational model. The ECOA for students in the traditional educational model asks “Would you recommend a friend to take a course with this professor?” while the ECOA for students in Tec21 asks “In general, my learning experience with this professor was.” The answer to both questions is a 0 to 10 scale.

Data analysis

To answer the research questions, we analyzed the data using the Statistical Package for the Social Sciences (SPSS) as a comparison between samples at different stages of the pandemic. We used a 2-tailed t-test to compare the results from students at each school in different moments. The comparison pre- v. trans-COVID-19 provides an account of whether students answered the ECOA differently immediately before the pandemic (AD2019), when they had fully in-person education, compared to the semester when the pandemic was declared (FJ2020), when they had to transition to fully online learning in a week. The comparison pre- v. intra-COVID-19 describes the differences in students’ evaluation of teaching between two fall semesters (same course offering), one fully in-person before the pandemic (AD2019) and the other fully online, after the pandemic (AD2020). The comparison trans- v. intra-COVID-19 analyses whether there were significant differences during and after the transition to online learning between two spring semesters, FJ2020 and FJ2021. The comparison between educational models was performed only for the FJ2021 semester, when the two populations were comparable (n_Traditional = 4,800, n_Tec21 = 4,224). For all tests, the null hypothesis is that the means are equal, we consider a significance of α = 0.05.

Results and discussion

We present and discuss the results in two subsections. In the first subsection, we analyze the longitudinal SET scores at different stages of the pandemic. This analysis approaches the first research question: Do students in the traditional and the Tec21 educational model evaluate their professors differently in the SET survey at different stages of the pandemic (pre-, trans- and intra-COVID-19)? We first present the descriptive statistics of the SET scores for the two academic models across the six schools at the different stages of the pandemic. Afterward, we provide the comparison between academic periods, to reveal the possible changes brought by the transition to online learning amid the pandemic. In the second subsection, we analyze the cross-sectional SET scores during the FJ2021 semester, when the population of both educational models was comparable. This analysis approaches the second research question: Do students in the Tec21 educational model evaluate their professors differently than students in the traditional educational model?

Longitudinal analysis of student evaluation of teaching scores at different stages of the pandemic

Descriptive statistics

The average evaluation for all schools in AD2019 was 8.67 (from a maximum of 10.00) in both educational models; in FJ2020, 8.78 in the traditional and 8.90 in the Tec21 educational model; in AD2020, 8.80 in the traditional and 8.99 in the Tec21 educational model, and in FJ2021, 8.77 in the traditional and 9.00 in the Tec21 educational model. We present the results of the descriptive analysis for each school and educational model in Table 2.

TABLE 2

Table 2. Average evaluation by educational model and school along the different COVID-19 periods and standard deviation in parenthesis.

Comparison between academic periods at different stages of the pandemic

From the results presented in the descriptive statistics, we may infer that there was an improvement between the pre- and trans-COVID-19 periods. However, the trans- and intra-COVID-19 seem constant. We present the results of the comparisons between periods in Table 3, highlighting the periods with a significant difference.

TABLE 3

Table 3. Statistical significance (p-value) of two-tailed t-test comparison between COVID-19 periods.

Transition to online learning

We studied differences between the pre-COVID-19 and the trans-COVID-19 periods, to identify whether the COVID-19 pandemic influenced students’ evaluation of teaching. The results show significant differences between these periods. For students in the traditional model, there is an improvement in students’ evaluation of teaching in all schools except for the School of Social Science and Government. For students in the Tec21 model, there is significant improvement in most schools, except the School of Social Science and Government and the School of Humanities and Education. In these cases, there seems to be an improvement, but it is not significant. The improvement on students’ evaluation of teaching during the transition to online learning is coherent with the positive outcomes found in studies performed internationally. The International Association of Universities (IAU) highlights positive impacts on students generated through transitioning to online education. The report specifies that the flexibility and community engagement initiatives which were considerably increased in Latin American education system throughout the COVID-19 pandemic have had a large share in generating positive educational outcomes (Marinoni et al., 2020). Moreover, an increased teacher-student interaction, new opportunities for content development, and resilience to adapt to online learning and adopt new technologies were observed in the time of pandemic (Oliveira et al., 2021). Another study demonstrated that the feedback of students and faculty has overall been positive focusing on satisfaction and effectiveness (Abu Talib et al., 2021), while the academic performance of students has increased in emergency remote teaching (Iglesias-Pradas et al., 2021).

To reduce the effects of the differences between the courses offered in the fall and spring semesters, we compared the two semesters in the intra-COVID-19 period. For the students in the traditional educational model, the results present no significant differences. For the students in the Tec21 educational model, we found significant differences only in the case of the School of Architecture, Art and Design (p-value: 0.022), and the School of Engineering and Sciences (p-value: 0.029). Given this finding, we decided to make the following comparisons: pre-COVID-19 v. intra-COVID-19 comparing the fall semesters of AD2019 and AD2020, and trans-COVID-19 v. intra-COVID-19 comparing the spring semesters of FJ2020 and FJ2021.

When comparing pre- and intra-COVID-19 (AD2019 v. AD2020), the evidence shows significant differences in most schools in the traditional model, except for the School of Social Science and Government and the School of Medicine and Health Science. In both cases, an improvement was observed, however, not significant. In the Tec21 educational model, the improvement was significant in all schools. Previous findings of the institutional monitoring process of student’s feelings amid the pandemic reported that students felt anxious, stressed, overwhelmed, tired and depressed (Camacho-Zuñiga et al., 2021). One of the main findings of this study is that in the longitudinal comparison for each educational model, the results provide evidence that students evaluated their professors more positively during and after the transition to online teaching versus face-to-face interaction consistently within schools. This is coherent with other studies in that the students’ evaluation was not negatively affected by the changes that professors implemented due to a situation beyond their control (Boysen, 2020). Moreover, the institutional response generated a positive impact as the university provided faculty training to face this new challenge during the week of paused activities, and tech guidance for professors who were not familiar with the technological tools through the “Academic Buddies” initiative (Vicerrectoría de Facultad, 2022). The institution was also committed to providing continuous faculty training, hence professors remained motivated to embrace challenges and to receive feedback for continuous improvement of their courses. These are characteristics that affect the way students evaluate their professors (Daumiller et al., 2021; Silalahi, 2021) which also help explain why the students perceived their professors’ performance as higher when facing the COVID-19 pandemic.

Fully online learning

We found contrasting results when comparing the trans- and intra-COVID-19 periods (FJ2020 v. FJ2021). In this case, for the traditional model, only the School of Engineering and Sciences and the School of Medicine and Health Science presented significant differences as the former showed an improvement in their students’ evaluation of teaching, while the latter experienced a decline. This, in general, could be an indicator of the greater toll the COVID-19 pandemic has taken on medical and health science students compared to the rest. While medical students are somewhat aware of the threat posed by the virus, their lack of clinical experience prevents them from being involved in patient care, which may lead to feelings of frustration (Nieto and López, 2020). Moreover, the possibility of high exposure to the virus during clinical practice may cause fear among health science students in addition to feelings of frustration and worry which may add to their learning experience and negatively affecting their perception of teaching. The research-based evidence suggests that the feeling of frustration as a social factor may be even stronger than the fear of exposure to virus-related risks (Shanahan et al., 2020). It is also important to note that most health science faculty are also practicing medical experts who are likely to be in the front line of response to the COVID-19 crisis. This could potentially affect the timely and effective course delivery which in turn may affect both the student’s evaluation of teaching and their own perception of skill development (López-Ruiz, 2020).

Overall, we propose a comprehensive analysis of the impact of the pandemic on students should be performed in conjugation with their respective disciplines in order to understand the challenges each group may face in a more profound manner. We also propose that deeper insights on the impact of the pandemic on medical students should be generated to understand the differences among those undertaking pre-clinical courses as opposed to those in clinical practice courses and whether the latter may mead to enhanced anxiety among students.

Cross-sectional analysis: Comparison between educational models

As described before, we present the comparison between models only for the FJ2021 semester in which the sample size for both educational models were comparable (Table 4). The results yield a significant increase in students’ evaluation of their professors in the Tec21 educational model compared to the traditional model in all schools, except for the School of Engineering and Science where the improvement is not statistically significant.

TABLE 4

Table 4. Statistical significance (p-value) of two-tailed t-test comparison between educational models for the FJ2021 semester.

This positive response of the students to the new educational model across schools reflects the effectiveness of implementing challenge-based and experiential learning to improve student’s experience of higher education. The Engineering and Sciences domain can be heavily affected by the lack of access to experimentation and laboratories during the pandemic, which is not necessarily the case in other schools. Contrasting with previous reports, students’ satisfaction with the design and implementation of the challenge-based learning in this school was generally high (Campos et al., 2021; Zavala et al., 2021). Several studies have reported positive results regarding competency development related to the Education 4.0 Framework in the Tec21 Educational Model in the School of Engineering and Science (Reyna-González et al., 2020; López et al., 2021). Nevertheless, students can be rather harsh in their evaluation of what they perceive as difficult (Rosen, 2018). It is also important to note that the students within the traditional model were in their final years of studies as opposed to those studying Tec21 Educational model who were at the initial phases of their academic journey. These reasons can explain a smaller level of significance in the students’ evaluation of the Tec21 model within specific school.

As mentioned before, we find it crucial to conduct an in-depth study on the disciplinary influence of the pandemic on students to raise awareness among the higher education stakeholders and to tune and refine the educational policies.

Conclusion

This study analyses whether student’s evaluation of teaching was affected by the pandemic and by the ongoing educational model transition in our institution. The analysis longitudinal data of the institutional SET survey during three stages of the pandemic, and cross-sectional data of the same instrument between the two educational models was performed for all the schools that conform the institution. To answer the first research question, the analysis compared the SET scores at different stages of the pandemic, pre-, trans-, and intra-COVID-19 with a t-test for each of the different and independently for each educational model. Both the longitudinal comparisons at different stages of the COVID-19 pandemic, as well as the cross-sectional comparison between educational models, yielded positive results. In most schools, the results indicated statistically significant improvements from the pre-pandemic period to the trans- or intra-COVID-19 periods. This is a positive indicator as it implies that the institutional response to the COVID-19 pandemic was well-perceived by students. From the analysis, it is evident that external negative factors outside institutions’ and professors’ control have not necessarily affected the SET scores in a negative manner. The only instance where the results yielded a significant decrease in SET scores was in the School of Medicine and Health Science, which can be due to the pressure the medical staff, faculty and students have undergone in the critical conditions exposed by the pandemic.

In response to the second research question, the analysis compared the SET scores between the two educational models in the semester of FJ2021. During this semester, the implementation of the Tec21 educational model was in the middle of its transition (fourth semester), and the two populations had a comparable number of students. It is important to note that, at this stage of the transition, all students in Tec21 were in their first 2 years of higher education, while the students in the traditional educational model were in their last 2 years of their programs. We observed that the Tec21 educational model was rated highly in all schools compared to the traditional model. In the School of Engineering and Science the improvement was not statistically significant, which hints to the lack of access to the laboratory, science and engineering facilities which are the necessary components of studying in this specific field.

The current study is limited by multiple factors. Primarily, the analyzed question for comparing the two educational models slightly varied from the traditional model to Tec21. This has limited our analysis to a certain extend. Another limitation is that the students’ populations for the two educational models were not experiencing the same phase of education (freshmen as opposed to nearly graduating samples) hence a certain degree of bias seems to be inevitable. As the time progresses, the study can be advanced to have a closer comparison of the students’ opinion with respect to the educational model whereby they are trained. Moreover, the discipline plays a crucial role in understanding the students’ needs and demands and those of professors. We can further improve our finding by performing a more in-depth discipline-based analysis of the evaluations. The results of this study provide insights on the importance of higher education institutional response to crises such as the COVID-19 pandemic, and of studying the shifts of students’ evaluation of teaching under such circumstances. Moreover, the evidence suggests that institutional changes in educational models with an emphasis on experiential learning and the Education 4.0 Framework is beneficial for students’ perception of higher education experience.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Office of the Vice-rectory of Investigation, Tecnológico de Monterrey, Mexico. The patients/participants provided their written informed consent to participate in this study.

Author contributions

EC: conception, research design, data curation, formal data analysis, writing – final draft, and project administration. SD, JO, and RC: data curation, validation, and writing – review and comments. JE: supervision and review and comments. SH: supervision and writing – review, comments, and editing. All authors contributed to the article and approved the submitted version.

Acknowledgments

We acknowledge the help of Mildred Lopez, Associated Dean of Academic Affairs of the School of Medicine and Health Science at Tecnológico de Monterrey, for her insight on the interpretation of the results regarding this school. We thank Genaro Zavala and Carlos Martínez-Torteya from the School of Engineering and Science at Tecnológico de Monterrey for their insight on the interpretation of the results regarding this school. We also acknowledge the financial and technical support of the Writing Lab, Institute for the Future of Education, Tecnológico de Monterrey, Mexico, in the production of this work.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ab Rahman, R., Ahmad, S., and Hashim, U. R. (2019). “A study on gamification for higher education students’ engagement towards education 4.0,” in Intelligent and interactive computing. Lecture notes in networks and systems, Vol. 67, eds V. Piuri, V. Balas, S. Borah, and S. Syed Ahmad (Singapore: Springer). doi: 10.1007/978-981-13-6031-2_5