Impact of the COVID-19 pandemic on online learning in higher education: a bibliometric analysis

Aristovnik, Aleksander; Karampelas, Konstantinos; Umek, Lan; Ravšelj, Dejan

doi:10.3389/feduc.2023.1225834

ORIGINAL RESEARCH article

Front. Educ., 03 August 2023

Sec. Higher Education

Volume 8 - 2023 | https://doi.org/10.3389/feduc.2023.1225834

This article is part of the Research Topic Increased Quality Education Through Cross-Campus Learning Environments View all 7 articles

Impact of the COVID-19 pandemic on online learning in higher education: a bibliometric analysis

Aleksander Aristovnik¹^*

Konstantinos Karampelas²

Lan Umek¹

Dejan Ravšelj¹^*

¹Faculty of Public Administration, University of Ljubljana, Ljubljana, Slovenia
²Department of Primary Level Education, University of the Aegean, Rhodes, Greece

The outbreak of the COVID-19 pandemic significantly disrupted higher education by forcing the transition to online learning, which became a mandatory teaching process during the lockdowns. Although the epidemiological situation has gradually improved since then, online learning is becoming ever more popular as it provides new learning opportunities. Therefore, the paper aims to present recent research trends concerning online learning in higher education during the COVID-19 pandemic by using selected bibliometric approaches. The bibliometric analysis is based on 8,303 documents from the Scopus database published between January 2020 and March 2022, when repeated lockdowns meant most countries were experiencing constant disruptions to the educational process. The results show that the COVID-19 pandemic increased interest in online learning research, notably in English-speaking and Asian countries, with most research being published in open-access scientific journals. Moreover, the topics most frequently discussed in the online learning research during the COVID-19 pandemic were ICT and pedagogy, technology-enhanced education, mental health and well-being, student experience and curriculum and professional development. Finally, the COVID-19 pandemic encouraged explorations of emergency remote learning approaches like e-learning, distance learning and virtual learning, which are intended to limit physical contact between teachers and students, where the specific requirements of a given field of study often guide which online learning approach is the most suitable. The findings add to the existing body of scientific knowledge and support the evidence-based policymaking needed to ensure sustainable higher education in the future.

1. Introduction

The outbreak of the COVID-19 pandemic significantly disrupted higher education by forcing the transition to online learning, which became a mandatory teaching process during the lockdowns (Aristovnik et al., 2020a). Despite the educational process saw disruptions on all levels of education, i.e., primary, secondary and tertiary (Tang, 2023), as well as in adult education (James and Thériault, 2020), worker education (Dedeilia et al., 2023) and lifelong education (Waller et al., 2020), higher education students proved to be one of the worst affected groups because the social distancing measures, on top of their education, challenged their financial and housing situation (Aristovnik et al., 2020a). Challenges arising from the density of students in educational facilities (e.g., campuses, faculties, dormitories etc.) meant higher education institutions were forced to offer education relying on various information and communication technologies (ICTs) and tried to ensure education comparable in quality to traditional learning, noting that the quality of online learning delivery holds important implications for student satisfaction and student performance (Keržič et al., 2021). Nevertheless, the lockdown periods were devastating for many students also in terms of their emotional functioning (Raccanello et al., 2022). The COVID-19 pandemic eventually grew more predictable and manageable, allowing higher education institutions to gradually shift back to traditional learning approaches. Although the epidemiological situation has improved over time, online learning is becoming increasingly popular as it provides new learning opportunities, especially when combined with traditional learning.

The rapid, yet from the health protection point of view necessary (Aristovnik et al., 2020b), shift from traditional learning to online learning considerably affected teaching and learning. The transition to online learning was made without adequate consideration of whether the study materials and teaching methods were suitable for this mode of higher education delivery. This was an ad hoc shift in a situation of great uncertainty for both teachers and students. The transition to online learning has also brought to the surface gaps in higher education providers’ preparedness and their lack of ICT infrastructure, resulting in unequal access to quality education for all, particularly students from rural areas and regions with lower socio-economic development. It is important to note here that the rapid shift to an online learning environment in emergency circumstances should not be confused with properly planned online education equipped with appropriate infrastructure that enables and supports pedagogical work and study in an online environment (Hodges et al., 2020; Fuchs, 2022; Misiejuk et al., 2023). Apart from the changes in teaching and learning, the social aspect of students’ lives has been affected as well. The most worrying consequence has been social isolation leading to a lack of crucial social interaction for students (Elmer et al., 2020; Bonsaksen et al., 2021; Fried et al., 2021; Van der Graaf et al., 2021) and in some cases also in coronavirus-related post-traumatic stress syndrome (PTSD) (Ochnik et al., 2021). According to Gavriluţă et al. (2022), three dimensions affected students during the COVID-19 pandemic: educational, social, and emotional. The transition from traditional to online learning entailed a significant transformation in education, requiring changes in teaching practices and new learning approaches. Further, the social aspect of the COVID-19 pandemic and associated lockdowns is evident in the absence of relational, economic and professional problems (in)directly affecting the transition to adulthood. The new reality changed attitudes to various aspects of life and, in turn, also affected emotional responsiveness. Briefly, substantial changes to everyday student lives were made during the COVID-19 pandemic that may hold far-reaching effects of currently unknown scope in the near and distant future (Campos et al., 2022; Gao et al., 2022; Keržič et al., 2022; Rasli et al., 2022).

Therefore, the educational community requires greater insights into different aspects of the COVID-19 pandemic’s impact on online learning, e.g., students, teachers, pedagogy, ICT technology, online learning approaches and implications for various fields of study. In the context of higher education, some bibliometric studies (e.g., Gurcan et al., 2022; Saqr et al., 2023) have already sought to address issues involving online learning during the pandemic. Yet, they relied on a limited and narrow bibliographic dataset of peer-reviewed literature or lacked a qualitative synthesis of the results beyond the metrics, thereby neglecting some general comprehensive outlines of the global research into the topic (Saqr et al., 2023). Moreover, despite some bibliometric studies focusing on technical aspects (e.g., Navarro-Espinosa et al., 2021; Bozkurt, 2022; Tlili et al., 2022), the identification of the most effective ICT tools for specific online learning approaches remains unclear. Finally, there are also some bibliometric studies that attempt to determine the effectiveness of online learning in providing higher education (Brika et al., 2021; Baber et al., 2022; Bilal et al., 2022; Bozkurt, 2022; Fauzi, 2022; Küçük-Avci et al., 2022; Yan et al., 2022), however, they often overlook the specific requirements of individual fields of study, thereby neglecting the crucial aspect of tailoring online learning provision to different disciplines.

The bibliometric study presented in the paper accordingly aims to fill the presented gaps in the literature. Specifically, it aims to present a global overview of the recent research trends in online learning in higher education using a comprehensive dataset of literature encompassing different varieties of online learning approaches that can facilitate online learning during the COVID-19 pandemic, provide some relevant qualitative synthesis of the results beyond the metrics and examine the relationships between ICT tools, online learning approaches and fields of study. Thus, the present bibliometric study, focusing on higher education, tries to answer the following three research questions:

• RQ1: What is the current state of the online learning research by conducting a descriptive overview and identifying top-cited documents?

• RQ2: What is the scientific production of online learning research across countries and sources?

• RQ3: Which are the main research hotspots and concepts in online learning research?

The remainder of the paper is structured as follows. The next section provides a literature review of recent bibliometric studies. The following section outlines the materials and methods applied in the study before the results of the present bibliometric analysis are described in the next section. At the end, the final section provides a discussion and conclusion while summarizing the main findings and implications.

2. Literature review

The outbreak of the COVID-19 pandemic led many governments to expand the use of online learning approaches as a solution to the global health challenge. Researchers thus showed rising interest in investigating the field of online learning, its dimensions, and its trends on all levels of education, particularly higher education. Such research relied heavily on bibliometric approaches to analyzing scientific research in the higher education context. Pham et al. (2022) concluded based on the 414 articles that although in the decades prior, there was an increase in the number of articles touching on the components of e-learning, such as the learning management system, this rise was accelerated during the pandemic in both developed and developing countries. This may be attributed to the attention of governmental policies that considered the topic of e-learning to be critical and worthy of priority. Similarly, Fauzi (2022) investigated 1,496 articles and concluded that the research focused on a few specific topics. The first is the delivery factor, which refers to selecting the appropriate learning practices. The second is the health and safety factor that relates to minimizing any risk that e-learning could bring to the mental and physical health of learners or teachers, such as stress, anxiety or even depression. The third topic refers to the field of study and the impact of e-learning. In areas like medical education, where clinical activities and labs have to be attended in person, some online learning approaches might be less appropriate than when used in other areas, such as social studies, where the requirements are less complex or different. Zhang et al. (2022) confirmed this finding after performing bibliometric research on 1,061 articles published between January 2020 and August 2021. They explained that theorists and researchers showed a growing interest in ways to respond to crises, such as the pandemic, and how to develop the best practices to ensure the quality and efficiency of e-learning. Examples of such practices might be inquiry-oriented learning and hands-on activities. This could derive from the already existing tendency of education researchers to respond to unprecedented global challenges or changes. The authors explain that this conclusion addresses interest in e-learning practices holistically.

In the same context, Yan et al. (2022) employed a bibliometric approach and identified that various digital tools are used in e-learning in the field of health studies. After investigating 132 studies, they concluded that selecting appropriate tools depends on many factors, including the field of a given course, the aims, and their effectiveness. They add that these findings can be significant for groups of people such as experts or trainee teachers. Okoro et al. (2022) researched 1,722 articles published between 2012 and 2021 and detected a surge in interest in the mental health of postgraduate students, as revealed by the research trends discussed in these articles. Still, they describe this surge as having been greater between 2020 and 2021, which may be attributed to the COVID-19 restrictions and their implications. Moreover, they believe that this research focus will likely continue soon.

After looking at 2,307 articles published between 2017 and 2021, Baber et al. (2022) detected an increasing trend in researching digital literacy. While this was underway before the pandemic, the latter caused a statistically significant further surge. Digital literacy is approached in the studied articles through parameters like instruction, teachers, learners, ICT and its applications, content knowledge, competencies, skills, perceptions, and higher education. It is also associated with acquiring the qualities required to deal with topics such as misinformation, fake news, technological content knowledge, health literacy, COVID-19, and distance education. The authors state that their study identified dynamics hidden in these research trends, which will likely continue in the next few years.

In higher education specifically, based on 602 articles, Brika et al. (2021) corroborated the growing trend of publishing articles on e-learning during the pandemic and outlined certain sub-topics of it, namely: motivation and students’ attitudes; blended and virtual learning comparison; types of online assessment; stress, anxiety and mental health; strategies to improve learners’ skills; quality; performance of the education delivered; challenges; and the potential of technology to lead to change and reform of higher education syllabi or curricula. The scope of those articles was to paint a bigger picture of how higher education communities and institutions use and treat online learning. This is expected to help with efficient decision-making in the future in order to have better results and functions in higher education and appropriate response to crises.

The bibliometric studies carried out during the pandemic identified a trend among researchers in higher education institutions to investigate more the technology factor and how the progress of the Internet, along with information and communication technologies generally, can further assist new modes of learning, such as online learning and distance learning. This might be attributed to a vision for a better means for new types of learning, as Küçük-Avci et al. (2022) claimed after carrying out a bibliometric analysis of 1,547 articles published between 2020 and 2021. The authors detected certain trends regarding distance learning in higher education. A main finding of their study, along with the increase in studies on distance education and e-learning in higher education, is that before the pandemic, the fact that these approaches were not so mandatory meant there was greater efficiency, probably due to the learners’ motivation. The authors further claim that researchers show a stronger interest in the technological means that can assist these types of learning. In addition, while researching 1,986 articles, Bozkurt (2022) established an increase in the implementation of blended learning by researchers who also aim to investigate the relationship between technological applications and learning institutions. Within these tendencies, researchers consider four thematic fields: a comparison of online and onsite learning with regard to effectiveness and efficiency; the experience, impressions and attitudes of stakeholders and learning community members with respect to blended learning; teacher training and curriculum development that will assure the appropriate and challenge-free implementation of blended learning; and the use of mostly a quantitative approach to research of blended learning.

Bilal et al. (2022) also examined research trends concerned with e-learning in higher education during the COVID-19 period by researching 1,595 studies published between 2020 and 2021. The four main trends they identified were supplementary to those mentioned by other authors: the first is about the challenges regarding online learning or blended learning along with the appropriate strategies in response; the second is student-centered collaborative learning and appropriate curriculum design; the third concerns home-based learning through a type of laboratory and the general conditions surrounding it; and the fourth addresses teachers’ background, training, professional competencies and interdisciplinary learning.

Tlili et al. (2022) focused on mapping COVID-19’s impact on Massive Open Online Courses (MOOCs). The overall finding from the 108 articles they considered is that there has been growing interest in these courses generally, and more specifically in research around their function and quality. This interest encompasses the main features of such courses, which provide easy accessibility and flexibility. However, they noted that this interest followed another trend among researchers in the context. In other words, the countries that published on MOOCs before the pandemic are the same countries that published during the period under study. Moreover, they stated that there is interest in the technical characteristics and requirements of such courses. Finally, the authors concluded that although most MOOCs were ICT courses, research has escalated into courses that refer to business, personal development or the humanities.

Several conclusions can be drawn from the above bibliometric studies. First, the series of bibliometric studies conducted during the pandemic demonstrates the rise of interest in online learning in higher education during the pandemic. Of course, there was a tendency toward e-learning before the pandemic, but between 2020 and 2022, this seems to have accelerated. The phenomenon is more intense in countries such as the USA, Canada, Australia, the UK, India and China. Concerning the area of study, the focus of researchers appears to be greater in fields such as Engineering, Sciences, and Health Sciences, albeit all fields seem to be investigated (Djeki et al., 2022; Pham et al., 2022; Vaicondam et al., 2022; Zhang et al., 2022). Various studies have focused on determining the effectiveness of e-learning classes and courses or pointing out parameters that influence their effectiveness. These could be the appropriate conditions or subtopics like motivation, blended learning, learning tools, teacher training, cooperation between different institutions or efficient practices (Brika et al., 2021; Baber et al., 2022; Bilal et al., 2022; Bozkurt, 2022; Fauzi, 2022; Küçük-Avci et al., 2022; Yan et al., 2022). A specific trend of authors is to examine virtual classes and laboratories (Kartimi et al., 2022; Rojas-Sánchez et al., 2022; Zhang et al., 2022). Finally, there is a focus on the technology factor. Namely, researchers have concentrated on technical issues and conditions related to e-learning courses and their proper functioning (Navarro-Espinosa et al., 2021; Bozkurt, 2022; Tlili et al., 2022).

3. Materials and methods

Comprehensive bibliometric data on online learning research during the COVID-19 pandemic were retrieved on 1 March 2022 from Scopus, a world-leading bibliographic database of peer-reviewed literature. The Scopus database was preferred because it has a broader coverage of scientific research than other databases such as Web of Science (Falagas et al., 2008). This was confirmed by an initial search using the same search query in each database, revealing that Scopus provided more relevant documents than Web of Science. Moreover, compared to the Scopus database, the Web of Science has been found to be a database that significantly underrepresents the scientific disciplines of the Social Sciences and the Arts and Humanities (Mongeon and Paul-Hus, 2016). Although English dominates in both Scopus and Web of Science, Scopus generally offers wider coverage of non-English documents, given that the titles, abstracts, and keywords are in English (Vera-Baceta et al., 2019). According to the basic statistical theory, which can also be applied in the context of bibliometric analysis, larger samples lead to analytical outcomes that are likely to be more accurate (Rogers et al., 2020). Therefore, Scopus appears to be a more relevant bibliographic database meeting the specifics of online learning research during the COVID-19 pandemic.

The search strategy was based on title, abstract, and keywords search using the advanced search engine and the search query covered keywords related to different online learning types (using the Boolean operator ‘OR’) and the COVID-19 pandemic (using the Boolean operator ‘AND’). The search was further limited to the period 2020–2022 (using the Boolean operator ‘AND’) to capture documents published between January 2020 and March 2022, when most countries were experiencing constant disruptions in the educational process imposed by repeated lockdowns. As the search query had no language restrictions, the full text of the obtained documents can be in any language, provided that the titles, abstracts, and keywords are in English. Therefore, the language has no impact on the results, as the bibliometric analysis is conducted solely based on the titles, abstracts, and keywords of the documents. According to the presented search query, 9,921 documents were obtained. After further revising the obtained documents, it was identified that some of them are not explicitly related to the context of higher education. By machine screening of documents by title, abstract, and keywords, those related to lower levels of education (i.e., primary and secondary education), as well as adult and worker education (i.e., lifelong education), were excluded from the database. There were 1,618 or 16% of such documents. The remaining 8,303 documents were identified as eligible for further bibliometric examination of online learning research during the COVID-19 pandemic. The bibliometric analysis utilized several bibliometric approaches (Figure 1).

FIGURE 1

Figure 1. Bibliometric approaches used in the bibliometric analysis. Own elaboration.

First, a descriptive overview was conducted to examine particular general bibliometric items, including timespan, number of (all, cited, single-authored) documents, authors, sources and author keywords and authors, references, and citations per document as well as to identify the most relevant documents. Scientific production was also examined to determine the most relevant countries and sources. Finally, network analysis was performed to identify the research hotspots according to the keyword co-occurrence network and examine the relationship between the main concepts based on a three-field plot analysis. The presented bibliometric approaches required the use of several different software tools. The descriptive overview was conducted using the Python Data Analysis Library Pandas (McKinney, 2012), scientific production was visualized by the Python Visualization Library Matplotlib (Hunter, 2007), while network analysis was performed using VOSviewer (keyword co-occurrence) (Van Eck and Waltman, 2010) and the Python Visualization Library Plotly (a three-field plot) (Pandey and Panchal, 2020). Specifically, the calculation for the three-field plot analysis included the following steps. Suppose that $C_{1}, C_{2}, \dots, C_{m}$ are analysed concepts where each concept $C_{i}$ is defined by a set of keywords and represented by binary indicators $W_{i 1}, W_{i 2}, \dots, W_{i k_{i}},$ expressed as $C_{i} = \max_{j = 1, \dots, k_{i}} W_{i j}$ for $i = 1, \dots, m$ (matrix column). Using this notation, the relationship between $C_{i}$ and $C_{j}$ can be defined as $C_{1}^{T} * C_{j}$ (matrix multiplication) where $i$ and $j$ are from three different sets (ICT tools, online learning approaches, fields of study).

4. Results

The descriptive overview presented in Table 1 shows the main characteristics of online learning and COVID-19 research in the higher education context. This research area covers a total of 8,303 documents (of which 7,922 (95%) have the full text in English) published in 2,447 sources between January 2020 and March 2022. Slightly less than half (46%) of these documents have at least one citation, while a relatively small number (15%) were written by a single author. The average number of references per document in this research area is 31.39, which is below the general scientific area of Educational Research (44.00) (Patience et al., 2017), suggesting that online learning research during the COVID-19 pandemic is grounded on fewer existing studies than general research. Finally, 3.50 citations per document can be observed for this research area. Due to the potential benefits of online learning, especially when combined with the traditional learning approaches and hence the development of the blended learning environment, this research is expected to further develop and be extended in the ensuing years (Fauzi, 2022). Further, upon analyzing the documents, it is evident that the average year of references is 2014.03, with an h-index of 60 (indicating at least 60 papers with 60 or more citations each) and a g-index of 94 (denoting that the top 94 publications have accumulated citations equal to or greater than the square of 94). Finally, it was found that within the examined dataset, a total of 1,334 documents (16%) have achieved a minimum of 5 citations (C5), while 691 documents (8%) have attained at least 10 citations (C10), 302 documents (4%) have obtained a minimum of 20 citations (C20), 79 documents (1%) have acquired at least 50 citations (C50), and 31 documents (0.4%) have obtained more than 100 citations (C100).

TABLE 1

Table 1. Descriptive overview of online learning and COVID-19 research (2020–2022).

The most relevant (top-10) highly cited documents in online learning and COVID-19 research in the context of higher education are shown in Table 2. The overview of the most relevant documents reveals several important topics that were intensively discussed. The first most relevant topic concerns ICT. The COVID-19 pandemic has created significant challenges for higher education, especially for medical and surgical education, which requires personal attendance in clinical activities and labs. Accordingly, several innovative ICT tools (i.e., videoconferencing, social media, and telemedicine) and online learning approaches (i.e., flipped classroom or blended learning and virtual learning) were proposed to address this challenge. It is also stressed that by using appropriately established ICT solutions, online learning can lead to more sustainable education (Adedoyin and Soykan, 2020; Chick et al., 2020; Dedeilia et al., 2020).

TABLE 2

Table 2. Most relevant documents in online learning and COVID-19 research (2020–2022).

The next top-cited topic relates to pedagogy. The disruption of education around the world due to the COVID-19 pandemic required teachers to possess specific pedagogical content knowledge related to designing and organizing better learning experiences with digital technologies. At the same time, challenges for online assessment and post-pandemic pedagogy are also highlighted (García Peñalvo et al., 2020; Iyer et al., 2020; Murphy, 2020; Rapanta et al., 2020). Finally, life and work is another of the most cited topics. Namely, the COVID-19 pandemic has considerably reshaped education and other aspects of life and work, often also through the perspective of mental health or emotional well-being (Dwivedi et al., 2020; Kapasia et al., 2020; Aristovnik et al., 2020a).

Furthermore, it is noteworthy that all of the highly cited documents were published in 2020. However, it is also evident that there are notable and highly relevant publications that emerged in the second year of the COVID-19 pandemic. Accordingly, there are two documents with a minimum of 100 citations published in 2021. In the COVID-19 pandemic context, Watermeyer et al. (2021), with 148 citations, examined the implications of digital disruption in universities within the United Kingdom, highlighting the challenges and opportunities arising from the emergency shift to online learning. Meanwhile, Pokhrel and Chhetri (2021) conducted a literature review to assess the impact of the COVID-19 pandemic on teaching and learning.

The scientific production across countries and sources is presented in terms of the number of documents and citations, whereby additional information is provided by a circle’s size, revealing the h-index as a measure of the scientific impact (Harzing and Van Der Wal, 2009) and by its color, presenting the time dimension in scientific production. The most relevant (top-10) highly cited countries in online learning and COVID-19 research are shown in Figure 2. While the United States of America stands out among all countries, the United Kingdom, China and India have a relatively large number of documents and citations. The findings are similar to those of other bibliometric studies on this topic (Saqr et al., 2023).

FIGURE 2

Figure 2. Most relevant countries in online learning and COVID-19 research (2020–2022). Own elaboration based on the Scopus database.

The most relevant (top-10) highly cited sources in online learning and COVID-19 research in the context of higher education are presented in Figure 3. Despite conference proceedings being prominent in terms of the relatively high number of documents, the most prominent journals, considering the number of citations, are Journal of Chemical Education, with the highest number of citations as well as documents, followed by Sustainability, International Journal of Environmental Research and Public Health, and Education Sciences. More specifically, the most relevant journals address different topics. First, Journal of Chemical Education covers the attempts, successes and failures of distance learning during the COVID-19 pandemic in chemistry education. It covers various topics, including the development of at-home practical activities (Schultz et al., 2020), student engagement and learning (Perets et al., 2020), online assessments (Nguyen et al., 2020) and virtual reality labs (Williams et al., 2021). Further, Sustainability is focused on student and teacher perceptions of e-learning and related challenges (Khan et al., 2020; Aristovnik et al., 2020a) and sustainability in education during the COVID-19 pandemic (Sobaih et al., 2020) to improve online learning and sustain higher education during uncertain times. Further, the International Journal of Environmental Research and Public Health covers various topics like the health and psychological implications of the COVID-19 pandemic (Sundarasen et al., 2020), including well-being and changes in behavior and habits. Finally, Education Sciences publishes some general research on the challenges and opportunities for online learning (Almazova et al., 2020), including student and teacher experiences (García-Alberti et al., 2021; Müller et al., 2021).

FIGURE 3

Figure 3. Most relevant sources in online learning and COVID-19 research (2020–2022). Own elaboration based on the Scopus database.

The keyword co-occurrence network is presented in Figure 4. Note that the nodes indicate keywords and the links the relations of co-occurrence between them. The node size is proportional to the number of keyword occurrences, showing the research intensity (node degree), while the link width is proportional to the co-occurrences between keywords (edge weight). In addition, the node color indicates the cluster to which a particular keyword belongs (Wang et al., 2020; Ravšelj et al., 2022). The keyword co-occurrence analysis reveals five research hotspots in online learning in higher education research during the COVID-19 pandemic. These are ICT and pedagogy (red cluster), technology-enhanced education (green cluster), mental health and well-being (blue cluster), student experience (yellow cluster) and curriculum and professional development (purple cluster).

FIGURE 4

Figure 4. Keyword co-occurrence network in online learning and COVID-19 research (2020–2022). Own elaboration based on the Scopus database.

A detailed synopsis of the research hotspots, including representative (the most frequent) keywords and documents (with several representative keywords), is presented in Table 3. The first research hotspot highlights the relevance of ICT and pedagogy in higher education during the COVID-19 pandemic. The most representative documents looked at the quality of online learning mechanisms (Gritsova and Tissen, 2021), active learning activities (Yan et al., 2021) and the role of e-learning departments in controlling the quality of academic processes (Hamdan et al., 2021). The second research hotspot refers to technology-enhanced education from different perspectives, such as opportunities to incorporate technological and curricular innovations (Shapiro and Reza, 2021), the adoption of different virtual experiences such as telehealth and virtual learning (Kahwash et al., 2021), and the utilization of social media to reach higher education students (Leighton et al., 2021). The third research hotspot emphasizes the problem of mental health and well-being issues that became a prevalent topic of discussion during the COVID-19 pandemic. Namely, several studies showed an increase in depression, anxiety and stress levels among higher education students in response to the COVID-19 pandemic (Abu Kwaik et al., 2021; Keskin, 2021; Yaghi, 2022). The fourth cluster is about student experience during the COVID-19 pandemic with specific focus on the between interaction and online learning satisfaction (Bawa'aneh, 2021; Bismala and Manurung, 2021; She et al., 2021). The fifth research hotspot underscores the relevance of curriculum and professional development. Several studies described the ways in which courses were adapted to online learning during the COVID-19 pandemic as well as the related challenges and strategies (Chen et al., 2020; Gonzalez and Knecht, 2020; Rhile, 2020).

TABLE 3

Table 3. Research hotspots based on the author keyword co-occurrence network in online learning and COVID-19 research (2020–2022).

Finally, the three-field plot analysis of the relationship between the main concepts (i.e., ICT tools, online learning approaches, fields of study) is presented in a Sankey diagram shown in Figure 5. The size of a rectangle corresponds to the number of documents for each theme, while the edge width reflects the inclusion index for connected themes (Wang et al., 2020; Ravšelj et al., 2022). These three concepts have been proven to be relevant in the context of online learning. Namely, ICT tools are a precondition for delivering course content through different online learning approaches, while the choice of online learning approaches may depend on the field of study (Ferri et al., 2020). During the COVID-19 pandemic, most attention was devoted to exploring e-learning (a combination of asynchronous and synchronous learning), distance learning (pre-recorded online lectures), followed by virtual learning (real-time online lectures). Since all these online learning approaches limit physical contact between teachers and students, they have been referred to as emergency remote learning approaches (Hodges et al., 2020; Fauzi, 2022; Fuchs, 2022), while other online learning approaches (computer-based learning, blended learning, m-learning) do not necessarily take place in an online learning environment. The emergency remote learning approaches were primarily supported by several ICT tools, particularly by social media (e.g., Facebook), gamification/simulation and virtual reality (integration of game-like elements into online learning platforms, mobile applications, or virtual reality simulations), Zoom and other videoconferencing platforms, as well as telehealth (for educating health professionals). Regarding the fields of study, e-learning, distance learning and virtual learning were mostly addressed in the context of medical/health education, while computer-based learning (i.e., specific engineering software programs etc.) was examined in the context of engineering education. This implies that the specific requirements of a given field of study often guide the selection of the most suitable online learning approaches (Fauzi, 2022).

FIGURE 5

Figure 5. Three-field plot showing the network between ICT tools (left), online approaches (middle), and fields of study (right) (2020–2022). Own elaboration based on the Scopus database.

5. Conclusion

The presented bibliometric study provides several important insights arising from research into online learning during the COVID-19 pandemic. In this period, a large volume of scientific knowledge was produced in the context of education that considered a range of aspects (Saqr et al., 2023). Therefore, a combination of selected bibliometric approaches was utilized to extract some general comprehensive outlines of the global research. The bibliometric analysis revealed the following.

As suggested by the descriptive overview of the state of Educational Research (Patience et al., 2017), the research into online learning during the COVID-19 pandemic is characterized by greater cooperation between authors, which coincides with the general observation that (international) scientific collaboration grew significantly during the pandemic (Duan and Xia, 2021). Further, online learning research during the COVID-19 pandemic is grounded on fewer studies than Educational Research (Patience et al., 2017), which may be explained by the absence of COVID-19-related literature at the time these documents were published. Nevertheless, noting the potential benefits of online learning approaches also when the epidemiological conditions are favorable, this line of research is expected to further develop and be extended in the ensuing years (Fauzi, 2022). The potential benefits refer especially to the development of a blended learning environment, which combines online and traditional learning approaches (Rasheed et al., 2020). The overview of the most relevant documents revealed three topics that were intensively discussed in the academic community, i.e., ICT, pedagogy, and life and work. The COVID-19 pandemic highlighted the importance and role of reliable ICT infrastructure for ensuring effective pedagogy in the online environment, as was needed to prevent the spread of the virus and to protect public health. Apart from the devastating health consequences for those directly affected by the virus and the disrupted educational process, the COVID-19 pandemic also dramatically affected students’ social life and work (Aristovnik et al., 2020a). The educational community is increasingly interested in finding ways to respond to crises like the COVID-19 pandemic and develop effective pedagogical practices that assure high-quality and efficient education in the online learning environment (Zhang et al., 2022).

The scientific production of online learning during the COVID-19 pandemic was geographically uneven. The greatest scientific production in terms of citations and number of documents can be observed in the United States, followed by the United Kingdom, China and India. Besides developed English-speaking countries, emerging Asian economies also seem to have played a crucial role in online learning research. Similar findings also emerged from other bibliometric studies on this topic (Saqr et al., 2023). Moreover, despite conference proceedings being prominent in terms of the relatively high number of documents, the most prominent journals, considering the number of citations, are Journal of Chemical Education, Sustainability, International Journal of Environmental Research and Public Health and Education Sciences, indicating that online learning research at the time of the COVID-19 pandemic was primarily published in open-access journals, as already observed in other research (Zhang et al., 2022).

The network analysis revealed five research hotspots in online learning research during the COVID-19 pandemic in the context of higher education: (1) ICT and pedagogy, focused on the quality of online learning mechanisms (Gritsova and Tissen, 2021), active learning activities (Yan et al., 2021) and the role of e-learning departments in controlling the quality of academic processes (Hamdan et al., 2021); technology-enhanced education concentrated on opportunities to incorporate technological and curricular innovations (Shapiro and Reza, 2021), the adoption of different virtual experiences such as telehealth and virtual learning (Kahwash et al., 2021), and the utilization of social media to reach higher education students (Leighton et al., 2021); (2) mental health and well-being issues facing higher education students, including depression, anxiety, and stress levels (Abu Kwaik et al., 2021; Keskin, 2021; Yaghi, 2022); student experience with specific focus on the between interaction and online learning satisfaction (Bawa'aneh, 2021; Bismala and Manurung, 2021; She et al., 2021) and (3) curriculum and professional development, focused on the ways in which courses were adapted to online learning during the COVID-19 pandemic as well as the related challenges and strategies (Chen et al., 2020; Gonzalez and Knecht, 2020; Rhile, 2020).

Further, the COVID-19 pandemic led to the exploration of emergency remote learning approaches such as e-learning, distance learning and virtual learning, which are intended to limit physical contact between teachers and students. These approaches were chiefly supported by several ICT tools, including social media, gamification/simulation, virtual reality, videoconferencing platforms, and telehealth. While computer-based learning, blended learning and m-learning do not necessarily occur in an online learning environment, they may still be suitable for certain fields of study, especially in the post-COVID-19 pandemic period. This implies that the determination of which online learning approach is the most suitable is often guided by the specific requirements of a given field of study (Fauzi, 2022).

Before generalizing these conclusions, it is important to note the limitations of the paper. First, the bibliometric analysis relied on documents indexed in the Scopus database, which might not cover the entire collection of research. Namely, documents that are published in journals indexed in other databases such as Web of Science, Education Research Index, Educational Resources Information Centre, etc. are not included in the analysis. However, to achieve the comparability of bibliometric metrics across documents, the bibliometric metrics are obtained from the single and, in general, broader Scopus database. Given the substantial overlap of documents across different databases of peer-reviewed literature, this limitation might not significantly affect the general observations on global research trends. Nevertheless, to check the robustness of the findings, it is still valuable to consider other bibliometric databases for future research. Second, the bibliometric analysis is conducted the bibliometric is based on a short time period (January 2020 – March 2022), which may also impact the metrics of documents published in closed-access (subscription-based) journals, placing them at a disadvantage compared to documents published in open-access journals. While it is not possible to overcome this limitation at present, conducting a bibliometric study with a longer time span would provide further time-dimensional insights. This would also be beneficial in terms of achieving better comparability between documents published in closed-access and open-access journals. Finally, despite the detailed search queries, some other relevant keywords may have been overlooked in the document search. Finally, the bibliometric method, as a method based on big data analysis, may miss certain highlights from the scientific literature that a systematic literature review would otherwise capture. Therefore it would be beneficial for future bibliometric studies also to incorporate a systematic literature review methodology, as the combined approach can provide a more comprehensive and nuanced understanding of the implications of the COVID-19 pandemic on online learning in higher education.

The bibliometric study provides some possible avenues for future research. First, in future bibliometric studies, it would be beneficial to conduct in-depth analyses of the relevant contexts that have emerged as highly significant in online learning during the pandemic. These include ICT and innovation, mental health and well-being, online learning and engagement, and curriculum and professional development. Examining these contexts more comprehensively can provide valuable insights into the specific dynamics and trends within each area, contributing to a deeper understanding of the implications of online learning during the pandemic. Second, it would be beneficial to conduct separate bibliometric analyses and comparisons to examine the differences between developed and developing countries. This approach can shed light on the unique research trends, contributions, and challenges faced by each group of countries in the context of online learning during the pandemic. This can provide a more nuanced understanding of the global landscape and identify potential areas for collaboration and knowledge sharing between developed and developing countries. Finally, it would be valuable to investigate the long-term impact of rapid publishing in open-access journals on the recognition and dissemination of scholarly findings in the field of online learning in higher education during the pandemic.

From the practical perspective, the COVID-19 pandemic has significantly disrupted higher education, but at the same time, it also accelerated the use of online learning tools in the educational process. Although the COVID-19 pandemic has gradually subsided over time, online learning approaches developed during this period continue to hold relevance and value for future education. Therefore, higher education institutions should prioritize leveraging ICT tools and innovative solutions in their educational delivery, which proved effective during the pandemic. Moreover, higher education institutions should also prioritize adapting appropriate online learning approaches and curricula to align with modern realities and the corresponding fields of study. This adaptation is crucial for enhancing student engagement and ensuring that educational programs remain relevant and responsive to the evolving needs of students in various disciplines.

The findings may help not only the scientific community in detecting research gaps in online learning research during the COVID-19 pandemic but also evidence-based policymaking by assisting in identifying appropriate educational practices in emergency circumstances. Specifically, the findings may help higher education policymakers to address the underlying shortcomings of the existing educational framework exposed by the COVID-19 pandemic and to design proactive mechanisms to deal effectively with such disruptions, thereby enabling them to create a more resilient and adaptable education system that can successfully navigate unforeseen challenges and ensure the continuity of quality higher education in the future.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.

Author contributions

AA contributed to the design of the study. DR and LU assisted with the data identification, cleaning, and analysis. DR and KK wrote the manuscript in consultation with AA. All authors contributed to the manuscript’s revision and read and approved the submitted version.

Funding

This research and the APC were funded by the Slovenian Research Agency under grant numbers P5-0093 and Z5-4569.

Acknowledgments

The authors acknowledge the financial support from the Slovenian Research Agency (research core funding no. P5-0093 and project no. Z5-4569). A preliminary version of the paper was presented at the International Conference on Information, Communication Technologies in Education (ICICTE) in July 2022. The authors are grateful to colleagues who attended the presentation and provided interesting comments and suggestions. Further, they wish to thank the reviewers for their valuable suggestions and comments.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Abu Kwaik, A., Saleh, R., Danadneh, M., and Kateeb, E. (2021). Stress, anxiety and depression among dental students in times of covid-19 lockdown. Int. J. Dentist. Oral Sci. 8, 1560–1564. doi: 10.19070/2377-8075-21000310