Educational Technology and Student Performance: A Systematic Review

Valverde-Berrocoso, Jesús; Acevedo-Borrega, Jesús; Cerezo-Pizarro, Mario

doi:10.3389/feduc.2022.916502

SYSTEMATIC REVIEW article

Front. Educ., 28 June 2022

Sec. Digital Education

Volume 7 - 2022 | https://doi.org/10.3389/feduc.2022.916502

Educational Technology and Student Performance: A Systematic Review

$\nJesús Valverde-Berrocoso$ Jesús Valverde-Berrocoso^*

Jesús Acevedo-Borrega

Mario Cerezo-Pizarro

Grupo de Investigación ≪Nodo Educativo≫, Departamento de Ciencias de la Educación, Instituto Universitario de Investigación y Prospección Educativa (INPEX), Universidad de Extremadura, Cáceres, Spain

The digital transformation of educational systems requires an evaluation of the effects of the integration of technologies in teaching-learning processes. From a pedagogical approach, Information and Communication Technologies (ICT) are defined, on the one hand, as the set of technologies that contain, store and disseminate information (e.g., e-books, videos, or databases) and, on the other hand, those technologies designed for short-term communication (e.g., social networks and smartphones). Academic achievement is one of the most widely used variables to try to understand how information and communication technologies affect student learning outcomes. Several international studies have shown little improvement in performance attributed to the use of ICT, although other reviews have shown positive results in relation to certain curricular areas. However, in general, the research is inconclusive and more studies are needed on this complex relationship. A systematic review was carried out using the Education Resources Information Center (ERIC) educational database as a documentary source, and research articles on academic performance and ICT use were selected (n = 100). As a result, there was evidence of improved performance in educational practices enriched with ICT. Mathematics and science are the areas of greatest interest to researchers, and it was observed that the educational systems most oriented toward competitiveness and educational selectivity are the most productive in this field. The discrepancies between the “macro-studies” of international organizations and the “micro-studies” analyzed in this review are discussed.

Introduction

There has been persistent controversy concerning the effectiveness of information and communication technologies (ICT) in improving student learning in recent decades. One of the most widely cited debates was the one between Clark (1983) and Kozma (1994), in which Clark argued that educational technology had no impact on student learning under any circumstances and that the “media are mere vehicles that deliver instruction but do not influence student achievement any more than the truck that delivers our groceries causes changes in our nutrition” (Clark, 1983, p. 445). The impact of technologies on learning is mainly due to the effect of innovation or teaching strategies, but not to the technology itself. For Kozma (1994), the analogy of the “truck” creates an unnecessary division between medium and method. This author considers that learning is not a receptive response to the distribution of educational content but an active, constructive, social, and cognitive process through which the student strategically manages physical, cognitive, and social resources to generate new knowledge, interacting with information available to them and integrating it with previous knowledge. Kozma (1994) argues that technological media have a physical or technical structure and mechanisms that can interact with the cognitive and social processes of students. These mechanisms are the symbol system (oral language, written text, numbers and formulas, images and sounds) and processing skills (reception, visualization, storage, retrieval, organization, translation, transformation, or evaluation, among others). Each medium has specific distinguishing characteristics depending on the mechanisms it incorporates (attributes) and the uses appropriate to these characteristics (variables), which is how ICT attributes should be used to have an effect on learning in a given context. Consequently, research on technological media must address their technical skills, the teaching methodologies in which they are integrated, and the complexity of the social situations in which they are used. Thus, we will aim to highlight the difficulties of the current educational research to demonstrate the effects of ICT on learning.

At the beginning of the twenty-first century, Cuban (2001) identified several unintended effects of the massive introduction of computers into education systems: more than half of teachers never used computers in their teaching practice and only 5% integrated the technology into their teaching routines; the use of digital technologies in classrooms was not linked to academic activities relevant to assessment, but to superficial and unconnected tasks. There was no clear evidence of an improvement in students' academic performance as a result of the use of ICT. Most teachers used technology to maintain existing teaching patterns.

In the last decade, the debate has been oriented toward analyzing the role of digital technologies in real classroom practices. As a result, it has become clear that the use of technologies does not promote a radical change in relation to pre-existing practices. The promised innovation is mainly of a symbolic and often ideological nature. It is therefore necessary to approach the phenomenon from the social, cultural, political, economic and historical aspects of education and technology. The effects of digital technologies on learning depend on the “goodness of fit” between the approach to learning, which involves values and ideology, and the educational technology. There is a gap between the rhetoric and the reality of technology-based learning. Many of the values currently espoused (interactive, learner-centered, social, communal, authentic, etc.) are often at odds with the nature of educational systems (Selwyn, 2011).

International Reports

In recent years, several international reports have been published on the effects of ICT on teaching-learning processes (PIRLS, TIMSS, PISA). These studies offer a global and comparative vision that allows us to recognize certain variables and processes that either favor or hinder the full integration of ICT in educational systems.

Progress in International Reading Study (PIRLS) initially observed that there is no correlation between the frequency of ICT use in schools and reading test scores. In both the 2001 study (Mullis et al., 2003) and the 2011 study (Martin and Mullis, 2013), a positive correlation was found with computer use at home, and a negative correlation with its use in schools. In the 2016 study, the “ePIRLS” version was used for the first time to assess students' online information reading via computer and through a simulated internet environment. Students who scored as “good readers” showed little difficulty in online reading and, moreover, come from family environments with more digital devices at home. The results reveal that students in schools without ICT performed lower than students in schools with ICT, although this correlation is difficult to interpret, since socioeconomic levels and teaching methodologies are highly interrelated. The average computer use for reading activities is once per week, especially when used to search for information, research a problem, or read digital texts (Mullis et al., 2017a).

On the other hand, in 1995, Trends in International Mathematics and Science Study (TIMSS) showed a negative relationship between the frequency of ICT use and lower academic performance than that of students with less frequent use or non-use of ICT (Martin et al., 2000). In the same vein, a negative relationship was found in the 1999 and 2004 studies, while the TIMSS 2011 report revealed the absence of a relationship between ICT use in the classroom and academic performance in mathematics (Mullis et al., 2012). In the TIMSS 2015 and 2019 reports (Mullis et al., 2017b,c, 2020), the academic performance in mathematics and science was observed to be higher among students who had access to ICT in the classroom. The most frequent use for mathematics was to conduct procedural exercises; and for science, it was the search for information. In both cases, computer use in the classroom varied considerably, as some countries exceeded 80% of usage and others barely reached 6%; with greater use in science (46–48%) than in mathematics (37–39%).

Very similar results were found in Program for International Student Assessment (PISA). In PISA 2000, no significant correlations were found between computer use at home or at school and academic performance. In PISA 2003, no significant correlations were found between computer use and test scores in mathematics, reading, and science. The findings showing positive effects of computer use at home and no effects or even negative effects of computer use at school have been replicated in different PISAs; in different countries, and by controlling different variables. PISA 2012 confirmed the previous results (Petko et al., 2017).

Skryabin et al. (2015) analyzed data from PIRLS 2011, TIMSS 2011, and PISA 2012 in order to test the effects of the development of ICT integration, as well as the subject's frequency of ICT use at home and at school on mathematics, reading, and science scores. The results showed a positive correlation between test scores and the level of development of ICT integration and ICT use at home, while they found negative correlations between ICT use at school and academic performance.

The Organization for Economic Co-operation and Development (OECD, 2015), in its report “Students, Computers, and Learning”, concluded that there is no evidence of the improvement of academic performance in schools that have invested in ICT, nor that this investment bridges the gap between higher- and lower-achieving students. The study considers that the potential of ICT has not been harnessed in schools and points out that the possible causes may lie in the poor quality of educational software, as well as in the ineffective methodologies used in teaching practices with ICT. Hu et al. (2018) found that students in countries with higher overall levels of digital competences were more likely to achieve better academic results. However, national ICT access and use did not correlate with students' mathematics, reading, and science literacy after mastering digital competences. Therefore, the integration of ICT into the curriculum and the reduction of the digital gap could be determined by the basic skills required for performance in a digital context, rather than by the availability of ICT. The findings show that the educational use of ICT in the classroom has an influence on the pedagogical adoption of the tools, as opposed to individual use without educational support. However, given that teachers tend to use ICT for a limited amount of pedagogical practices that do not substantially alter traditional ways of teaching, an increase in the frequency of ICT use does not seem to offer tangible benefits for learning and could even be considered a detrimental factor. The negative relationship between students' academic use of ICT, both in and out of school, and their learning outcomes could indicate that ICT are not being used appropriately to improve learning. On the other hand, students' interest, competence, and autonomy in the use of ICT revealed different degrees of positive correlation with their academic performance in mathematics, reading, and science, while the use of ICT for the purpose of social interaction showed negative correlations with academic performance in all three subjects. In conclusion, “the quantity of ICT use can advance student learning only when the quality of ICT use is ensured” (Hu et al., 2018, p. 11). Kunina-Habenicht and Goldhammer (2020) found negative correlations between ICT use at school and at home and performance in PISA tests. These results could be explained by the fact that more frequent use of ICT at school is likely to be associated with a purpose of retaking exams for students with lower academic performance.

Previous Reviews and Meta-Analyses on Academic Performance and ICT

It is very difficult to find conclusive and consistent evidence to support the hypothesis of a positive impact of ICT use on student performance, as measured by standardized tests (Biagi and Loi, 2013). Some studies have found that the use of computers does not have a positive effect on academic results based on the assessment by standardized tests (Angrist and Lavy, 2002), nor does increased internet connectivity in schools provide evidence of improved academic results (Goolsbee and Guryan, 2006). However, other research studies have observed positive impacts of the use of ICT in specific disciplines, such as the improvement of reading and language competences (Rouse and Krueger, 2004) and mathematic performance (Banerjee et al., 2007; Barrow et al., 2009).

Three educational uses of ICT for learning mathematics were identified (Drijvers et al., 2010): (a) “to do mathematics”, i.e., the use of devices or apps for performing mathematical calculations, which increase efficiency and accuracy, in addition to allowing teachers to perform more creative and applied learning activities; (b) instrumental understanding, referring to the ability to perform mathematical rules and procedures through repetition and immediate feedback, by means of exercises, tutorials or simulations that are used as a complement to the teaching provided by the teacher; and (c) conceptual understanding, which refers to “knowing what to do and why”, and involves the use of specific models in flexible environments that facilitate exploration and create multiple mathematical representations (e.g., the open source software GeoGebra).

Thousands of studies have been conducted over the past 30 years on the effects of mathematics teaching practices with ICT on academic performance. To gain insight into the extent of these results, Young (2017) conducted a “meta-analysis of meta-analyses” (or second-order meta-analysis) with studies published between 1986 and 2015. In relation to the use of ICT for the improvement of computational thinking skills, it is apparent that the results on the effects of calculator use on mathematic performance modified teachers' perceptions of its use in the classroom, although it is important to consider the assessment model and educational stage as moderating variables. Regarding the use of ICT in the improvement of mathematics teaching, through computer-based instruction (CBI) and computer-assisted instruction (CAI), and their relationship with academic performance, small to moderate effects were observed, considering the relevance of the time spent using ICT and the teaching methodology as influential factors. Lastly, concerning the use of ICT as a tool for exploration and modeling, meta-analyses are still limited because research on the use of mathematics-specific software is still in its early stages. Several effect size moderators were detected, such as age, duration of the educational intervention, or the mathematical content taught (e.g., algebra or geometry). The results suggest that the mean effect sizes were 0.47, 0.42, and 0.36 for computational enhancement technologies, instrumental understanding enhancement technologies, and conceptual understanding enhancement technologies. In conclusion, it can be stated that (a) the effect of ICT use in mathematics teaching on academic performance, regardless of the type of educational use, is moderate; and (b) it can be considered an effective means for the improvement of learning outcomes. It is estimated that students with ICT support would perform better than 62% of students who are not offered this resource.

Furthermore, the results of research on the impact of educational technology on reading coincide in showing positive but moderate effects compared to traditional methods (Becker, 1992; Fletcher-Flinn and Gravatt, 1995; Soe et al., 2000). The use of reading instruction programs that use ICT resources as an educational supplement, which were the most frequently used programs in past decades, did not have a significant effect on reading performance (Dynarski et al., 2007; Campuzano et al., 2009). Other more comprehensive models that use methodologies that combine practices with the presence/absence of ICT, together with specific teacher training, seem to reveal a greater impact on reading (Cheung and Slavin, 2012). These findings show that integrating technological and non-technological components for the teaching of reading is the most relevant issue.

Technological development has contributed to a change in reading habits, and the reading of digital texts has become the predominant activity as an alternative and complement to reading printed material. Therefore, equity in education is important in both printed and digital reading. Rasmusson (2016) studied the influence of digital reading on reading performance. The results of the study show no influence of cultural capital and economic factors on students' digital reading performance. These results could indicate that digital reading is less valued than print reading in light of cultural capital standards. That is, digital reading does not yet belong to the activities and artifacts that represent desirable cultural capital in contemporary society.

Xiao and Hu (2019) found that ICT use improves the reading performance of students, especially those from a disadvantaged socioeconomic background. They observed that the impact of ICT use on students' reading performance gap caused by their socioeconomic status changed from negative to positive over a 3-year period (2012–2015). These researchers consider that a more interactive and attractive digital environment, as well as the use of creative activities in teaching practices with ICT, could explain this positive effect of ICT on reading performance among students from disadvantaged socioeconomic backgrounds.

Verhoeven et al. (2020) carried out a meta-analysis on the effects of computer use on early literacy in Early Childhood Education over the past 25 years. A small average effect size (0.28) was evident, with a very high variability, which was similar for each of the ICT-supported interventions: phonological awareness, alphabetic principle, a combination of both, and story reading. It can be affirmed that ICT can be beneficial in the field of early literacy, as long as they are integrated into the school curriculum and provide continuous instructional scaffolding. In any case, teachers outperform digital devices in facilitating phonological awareness and understanding of the alphabetic principle. Early Childhood Education teacher training should be promoted in order to increase the benefits of ICT.

Delgado et al. (2018) conducted a meta-analysis, with the aim of comparing the use of print and digital media on linear reading, with strong similarities between digital texts and printed texts. The results of the study clearly showed lower reading comprehension performances for digital texts compared to printed texts. These results were consistent across methodologies and for all theoretical frameworks. However, digital devices for reading must be considered, as the computer appears to have a greater negative impact on comprehension than other digital media. It was observed that the advantage of reading printed informational texts was significantly greater when a reading time limit was imposed, compared to self-paced reading, regardless of the length of the text. This evidence is important to consider when using digital texts in assessment tests. Current evidence supports the claim that experience with ICT alone does not improve students' comprehension skills, but may even have detrimental effects. Digital media behavior, based on quick interactions and motivated by immediate rewards, makes it difficult to perform more cognitively demanding tasks. This would explain the negative correlations between frequency of ICT use and reading comprehension among adolescents. Increased exposure to digital reading resources tends to enhance speed and multitasking, to the detriment of behaviors more conducive to deeper comprehension, which requires more time and concentration. As a result, digital environments may not always be the most appropriate choice to facilitate deeper learning. The competences linked to information search and selection or critical reading are essential for comprehension, but require a high level of executive processes, which are not yet sufficiently developed in students engaged in digital reading. The results indicate that the screen inferiority increased over the last 18 years and that there were no differences in the effects of the mediums between age groups. The preference for printed over digital reading has persisted despite technological advances. However, accepting the fact that the inclusion of digital devices in our educational systems is unavoidable, educational methods that encourage an effective digital reading competence must be developed.

In relation to the use of ICT at home and/or in the classroom for educational purposes, Bulut and Cutumisu (2018) examined the extent to which the use and availability of ICT at home and at school had a differential impact on academic performance in mathematics and science. It was observed that ICT use was found to be detrimental to mathematics and science performance, irrespective of learning outcomes in both subjects. ICT availability, especially at home, revealed a positive correlation with mathematics and science performance for students where ICT access was more scarce, but had no effect on students for whom ICT were widely available. On the other hand, the use of ICT at home for school work had no effect on academic performance in mathematics and science. Most recently, Gubbels et al. (2020) found that ICT access at home has a negative correlation with digitally assessed reading performance. Students with access to a variety of ICT resources at home scored lower on digitally assessed reading, compared to students with moderate levels of ICT resources at home. Moderate ICT use was associated with higher digitally assessed reading performance. Higher ICT use at home, explicitly in relation to school-related tasks, was negatively associated with students' test scores. It was also observed that high levels of perceived ICT autonomy is related to high performance in digitally assessed reading. Both a lack of interest in ICT and excessive interest in ICT are related to low digitally assessed reading performance. In conclusion, these results suggest that investing money and time to provide students with ICT resources at home or school and increasing the use of these resources does not necessarily improve digitally assessed reading performance.Similarly, Agasisti et al. (2020) observed the existence of a negative correlation between students' use of ICT at home for academic tasks and academic scores in mathematics, reading, and science assessment tests (PISA), regardless of students' level of academic performance. Researchers argue that “more frequent use of ICT at home, even when explicitly connected to school-related tasks, is detrimental to academic achievement” (Agasisti et al., 2020, p. 16–17). Promoting the widespread use of ICT for academic tasks at home, without precise guidance on how ICT should be used, can have detrimental effects on learning. Evaluating the quality of digital educational materials should be a central element of educational planning and part of the school culture. Among the reasons that could explain this result could be, to begin with, that the devices and software used are not suitable for the required academic functions because they are obsolete. Secondly, students have not been educated to develop the necessary skills that allow them to efficiently make use of ICT to improve their learning. This training should be modeled in the classroom through a variety of ICT-enriched teaching practices. The effective use of digital devices at home would depend on how students are previously trained in the classroom. Third, students could be underutilizing ICT for learning at home due to the lack of skills to avoid the distracting effects of ICT derived from their multiple functions and multitasking. Lastly, it could be that students who use ICT at home more frequently for school purposes do indeed improve in other variables relevant to learning that have not been measured in the assessment tests.

Park and Weng (2020) found (a) that students' use of ICT for entertainment purposes negatively affected their academic performance; (b) that students with a positive attitude toward ICT use have a high probability of attaining better learning outcomes; (c) that students' perceived ICT self-efficacy has a significant positive effect on their academic performance, and its effect size is larger than that of attitude or digital competence; (d) that a country's economic development levels are associated with student performance, as it affects ICT resources and competence in the schools, such as infrastructure, ICT support staff, and educational software. Therefore, it is essential to promote equitable access to ICT through related policies, including discounted internet access and the expansion of ICT infrastructure in public areas, such as schools or libraries, for low-income families; in order to resolve this educational inequality, based on addressing income inequality and the digital gap.

Finally, regarding the 1:1 Model (one computer per student), Zheng et al. (2016), as a result of their meta-analysis, observe that its use modifies many aspects of education at the Primary and Secondary levels. The most common changes identified in the studies reviewed include significantly higher academic performance in science, writing, mathematics, and language subjects; increased use of technology for various learning purposes; more student-centered, individualized, and project-based learning; increased engagement and motivation among students; as well as improved relationships between teachers, students, and families. However, most published research consists of case studies with little representation of experimental and quasi-experimental research, which makes it difficult to conduct a meta-analysis.

The aim of this systematic review is to first discover the research results of the last decade concerning the relationship between ICT use and academic performance in mathematics, science, reading, and writing. This systematic review, unlike previous ones, integrates studies on academic performance and ICT in the most researched curricular skills. Second, this review aims to identify the teaching modalities supported by ICT that have been implemented and the educational contexts where these learning processes have taken place. In contrast to reviews based on “macro” studies (e.g., PISA), this study investigates “micro” research based on classroom practices. This systematic review aims to identify the teaching methodologies and learning contexts present in research on academic performance and ICT. It is very important to understand how technologies are used in the teaching process and what are the learning environments where these technologies are introduced. Third, this review aims to analyze the documentary characteristics of the articles that explain the results and to describe the methodological approaches used in the research.

Methodology

A systematic review consists of compiling a body of research according to previous inclusion criteria, with the objective of answering specific research questions. This systematic review applies the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 standards to identify inclusion criteria, information sources, search strategy, study selection process, data collection process, and presentation and synthesis of data. The systematic review process applied in this study consists of different phases (Zawacki-Richter et al., 2020):

Phase 1: Research questions (RQ). The questions are organized into three areas, as shown in Table 1: (1) documentation dimension (RQ1–RQ3), to identify the areas of knowledge on the topic, the geographical location of the researchers, and the impact of the journals where the results are published; (2) methodological dimension (RQ4–RQ5), to address the approaches and methods applied in the studies, as well as the sample sizes and time frame of the research studies; and (3) pedagogical dimension (RQ6–RQ10), to identify the different educational contexts involved in the educational use of ICT, the educational levels and curricular areas under study, the identification of teaching methods in the teaching practices analyzed, the effects observed on academic performance and, finally, to identify other variables of a pedagogical, psychological, sociological or technological nature that were used in the studies.

Phase 2: Inclusion criteria and information sources. The documentary sources are from articles published in scientific journals during the period 2013–2021, which include the following Education Resources Information Center (ERIC) Thesaurus terms as descriptors: “Technology uses in Education,” “Mathematics Achievement,” “Reading Achievement,” “Writing Achievement,” and “Science Achievement”. Empirical studies with quantitative, qualitative and mixed methods were also included. Exclusion criteria were applied to articles whose main subject of study was not academic performance in relation to ICT.

Phase 3: Search strategies. The ERIC database was used for the study selection process. ERIC is recognized as the largest database specialized in education, with references provided since 1966. In all search queries, the concept of the thesaurus “Technology uses in Education” was used together, alternatively, with the concepts “Mathematics Achievement,” “Reading Achievement,” “Writing Achievement,” and “Science Achievement”, also from the ERIC thesaurus.

Phase 4: Study selection process. The initial search resulted in 316 articles, of which 28 were duplicates. We analyzed the 288 articles on the basis of the title and abstract, according to the inclusion-exclusion criteria. Once we agreed on the results, 186 articles were excluded. We independently analyzed the remaining 102 articles in full, which resulted in the exclusion, upon agreement, of 2 articles. The final sample of documents for the systematic review consisted of 100 articles (Data SRL), as shown in Figure 1.

Phase 5: Data coding and synthesis. The Zotero reference manager was used for data collection. Data synthesis was performed using a coding sheet with 29 fields (LibreOffice Calc). VOSViewer was used for the conceptual network analysis. The three researchers, first independently and then jointly, were involved in the different phases of selection according to prior inclusion criteria and definitive inclusion in the review.

TABLE 1

Table 1. Areas, research questions, and initial coding criteria.

FIGURE 1

Figure 1. Flow chart of the study selection process.

Results

RQ1. What Is the Conceptual Network Extracted From the Literature, and What Are the Topics of the Articles According to the Category of the Journal in the Databases?

A series of clusters generated by the co-occurrence of the keywords of the articles were obtained to analyze the conceptual network, as shown in Figure 2. The first cluster (14 items), in green, shows a network formed by the educational uses of ICT that includes students' attitudes, gender, or the effectiveness of educational programs. The second cluster (14 items), in red, identifies a conceptual network on mathematics performance and the associated teaching methods to attain it. The third cluster (10 items) represents a conceptual network that includes the research methods and techniques used in the studies analyzed. As a result, the conceptual network of keywords in the articles included in this systematic review is composed of three main nodes that identify Pedagogy (Educational Technology), Curriculum (Mathematics as the most noteworthy area), and Educational Research (approaches, methods, and techniques) as pillars on which the interests of the researchers are structured.

FIGURE 2

Figure 2. Map of co-occurrence by keywords of the articles reviewed. Source: prepared with VosViewer.

Of the articles analyzed, according to Scopus, 73% were published in journals with associated indexed categories; namely, in the category of “Education” (67%), “e-Learning” (4%), and “Communication Education” (2%). Eight percent belong to journals categorized in the “Mathematics (miscellaneous)” category. Seven percent belong to the “Multidisciplinary” category. The rest are comprised of various topics related to Natural Sciences (4%), Health (2%), Social Psychology (2%), Language and Linguistics (1%), and Computer Technology (3%).

RQ2. What Is the Geographical Distribution of the Publications?

The geographic location of the research in this review was established based on the country associated with the first author of the articles. As a result, a quarter of the studies were conducted in the United States. This was followed by Turkey (19%) and, in third place, Taiwan (8%). With a representation of 4%, studies from Malaysia and Indonesia were also identified. On the other hand, China, Finland, and Spain represented 3% of the studies analyzed.

RQ3 What Is the Distribution of Articles According to Their Position in the Databases?

The journals containing the articles selected for this review were classified according to the quartile they were assigned in Scopus, according to the year of publication of the study. Thus, it is evident that 28% of the articles were published in journals in quartile 1 (Q1), 22% of the articles belong to the second quartile (Q2), 13% of the studies are located in the third quartile (Q3), and lastly, 8% of the articles are located in the fourth quartile (Q4). The remaining 29% of the articles belong to journals not indexed in Scopus.

RQ4. What Methodological Approaches and Research Methods Are Used in the Selected Studies?

The studies were classified according to their approach as quantitative (58%), qualitative (4%), and mixed (38%). Regarding the methodologies used in the selected research studies, the quasi-experimental method is the most frequently used (54%), followed by experimental studies (26%). Exploratory studies represent 7% of the total, and case studies were used in 6% of the research studies. Studies based on questionnaires were used less frequently (2%), as well as those using instructional design, descriptive and observational studies (each representing 1%).

RQ5. What Are the Sample Sizes of the Studies, and What Is the Duration of Each One?

Half of the studies in this systematic review used samples ranging from 25 to 100 subjects. Eighteen percent were in the range above 100 and below 200. Approximately 22% used samples larger than 200 subjects. Three percent used samples smaller than 25, and the remaining 7% did not report sample size. The majority of the research studies reported a study duration of between 1 and 6 months (33%). Studies with a duration of <1 month represented 26%. Twelve percent reported a duration of between 6 months and 1 year, and 11% reported a duration of over 12 months. Four studies were carried out in a period of fewer than 7 days. Fourteen percent of the articles did not report the duration of their studies.

RQ6. In What Contexts Does the Teaching-Learning Process With ICT Take Place?

Almost three-quarters of the teaching practices with ICT in the studies analyzed were carried out in the classroom (73%). Blended learning accounted for 23% of the studies. Lastly, teaching practice outside the classroom with ICT accounted for 3% of the studies reviewed.

RQ7. Which Educational Levels Are Included in the Research Studies, and Which Components of the Core Curriculum Are Involved?

The highest percentage of the studies reviewed conducted their research in the educational field of secondary education (34%) and primary education (31%). In third place were studies on the academic performance of university students (27%) and, lastly, early childhood education represented 7% of the research. One of the studies analyzed was conducted at various educational levels. The curricular area with the most studies on the relationship between academic performance and ICT use was mathematics (48%), followed by science (35%). Research on reading and writing together accounted for 11%.

RQ8. What Are the ICT Teaching Modalities Applied in the Studies Analyzed?

As shown in Figure 3, almost half of the studies (47%) identified expository methods (lecture). Secondly, the most frequent teaching methodology was Project-Based Learning (11%). Game-Based Learning and the 1:1 Model (one computer per student) each represented 10%. Flipped Classroom represented 8% of the studies, followed by Gamification (7%). The Bring Your Own Device (BYOD) model represented 4%. The Problem-Based Learning model was used in one study. Two studies did not report the teaching modality.

FIGURE 3

Figure 3. Distribution of teaching modalities according to curricular area.

RQ9. What Are the Effects of ICT Use on Academic Performance?

In general, 83% of the research reviewed revealed positive results in the relationship between academic performance and the educational use of ICT. Nevertheless, 15% reported negative results in the relationship between the use of ICT in educational practice and students' academic results, and 2% did not provide any conclusive results in this regard.

RQ10. What Other Variables Have Been Studied in the Selected Research Studies?

In addition to academic performance, the research in this review has analyzed other variables, identified below, organized into different areas: (a) pedagogical variables (lecture length, collaboration, communication, comprehension, experience, gamification, inquiry skills, interactive, key competences, learning environments, mathematical skills, peer learning, student's point of view, student's background, teacher practice, teacher's point of view); (b) psychological variables (attitude, autonomy, cognitive skills, cognitive learning, perceptions, retention, self-efficacy, self-regulatory, spatial abilities); (c) sociological variables (gender); and (d) technological variables (augmented reality, virtual reality, use of e-books, use of smartphones, BYOD effects).

Conclusion and Discussion

The aim of this article was to discover the results of educational research on the relationships between academic performance and the educational use of ICT. To this end, a systematic review was carried out, which has allowed us to answer 10 research questions about the documentary characteristics of the articles analyzed: the thematic scope of the journals, the geographical location of the studies, and an estimate of the quality of the publications based on their positioning in Scopus; the methodological characteristics of the studies by identifying their approach and method applied, as well as the sample used and the time frame of the research; and, finally, in relation to the pedagogical dimension, we explored the educational contexts of use of these technologies (physical classroom, outside the physical classroom, and blended learning), the teaching modalities used, the educational levels and curricular areas involved.

The most relevant findings of this systematic review are, on the one hand, the identification of the curricular areas that focus the attention of educational research on the relationship between ICT use and academic performance. It was observed that interest in this subject of study has been expressed in educational journals, but it is no less relevant that they were also published in journals specialized in mathematics and science. In fact, more than 80% of the studies in this review address the effects of ICT use on mathematics and science learning, as these are the curricular areas in which researchers are most interested to discover how the educational use of ICT affects students' learning outcomes (Liao and Chen, 2021; Çavuş and Deniz, 2022).

On the other hand, it was observed that educational research on the effects of ICT on academic performance seems to be conditioned by the characteristics of the educational systems where these studies are carried out. Considering that the use of ERIC implies a limitation of this review with respect to potential bias (language, sources) in the study selection process from which our sample was obtained, the geographical distribution of the research studies reveals that certain educational systems have a greater interest in ICT accountability. In addition to the expected prevalence of research conducted in the United States, other countries such as Turkey and Taiwan, both with highly competitive educational models focused on knowledge assessment, were also highlighted (Chou, 2019; Gümüş et al., 2021).

With respect to the methodological approach, it has become evident that, as is usually the case, the study of academic performance uses quantitative approaches, although mixed approaches are beginning to play an important role. Quasi-experimental or experimental methods are clearly the dominant methods. Other methodological approaches were not used, characterized by the design of educational environments or resources as an integral part of the research, with the aim of contributing to the resolution of an educational problem or the improvement of the teaching-learning process, such as Design-Based Research (Huang et al., 2019).

Half of the studies analyzed were conducted with small sample sizes of students, which should be considered a possible limitation in the generalization of the results. The same could be said in relation to the time frame of the studies to examine the results. Around 60% had a maximum time frame of 6 months, which could be considered a short time to be able to thoroughly observe the effects of ICT on academic performance. In this regard, we can describe these studies analyzed as “micro-studies” (Moss, 2012). However, the fact that half of the studies analyzed were published in quartile 1 or 2 (Q1/Q2) journals attests to the quality of the research and its relevance for scientific progress.

It was observed that the research studies analyzed in this systematic review report that 3 out of 4 teaching practices with ICT have been carried out in the classroom and, on the other hand, the teaching methodology involved in half of these studies consisted of expository or traditional methods. Consequently, there is a need to increase the number of studies that analyze other blended or virtual educational contexts and their effects on academic performance. Although blended learning is present in almost a quarter of the studies analyzed, online teaching is underrepresented. There is also evidence of the need for further research using active methodologies supported by ICT for digital citizenship training (Sancho-Gil et al., 2020).

The results of this systematic review allow us to conclude that there is a clear difference in the results obtained on academic performance and the use of ICT in the “macro-studies”, with large samples and over long periods of time, carried out by international organizations, especially the OECD, and in the “micro-studies” carried out within university research groups using small samples and over short periods of time, which we have analyzed in this research study.

International studies repeatedly show that frequent use of ICT in the classroom does not establish positive correlations with academic performance. Several explanations for these results can be considered. First, educational technology is a concept that includes the use of devices, applications, and methodological approaches that are applied in specific contexts. Therefore, educational technology can be effective to a certain degree, depending on how it is used. The questionnaires on ICT use in these studies are more concerned with quantity rather than the educational quality of the use of ICT in the classroom. Secondly, the concept of performance linked to ICT differs in the sense that the contributions of the use of ICT are mainly reflected in the development of new skills and not directly assessed, such as digital competence or self-regulated learning. Lastly, it is necessary to redefine the educational research methodologies that have been used in educational technology and to apply longitudinal and mixed studies that study the phenomenon in all its complexity (Petko et al., 2017).

Evidence on the impact of educational ICT plans, programs and projects has been disappointing in most cases (Luckin et al., 2012; Vrasidas, 2015). The great expectations of educational reform or “revolution” as predicted by the most influential technologists, multinational companies in the sector and political discourse have not been fulfilled. On the contrary, the radical changes produced in other social and human activity systems promoted by ICT, the transformation of everyday life due to the use of digital devices connected to the internet, and the increased use of ICT in any context outside the academic environment reveal that the barriers to the integration of ICT are not primarily rooted in the vision of ICT as potential tools for the transformation of the teaching-learning process, but in the obsolescence of organizational structures and dominant cultures in educational systems (Somekh, 2007).

The closer approach to particular educational contexts enriched with ICT, offered by the “micro-studies” of this systematic review, reveals that academic performance improved with the use of technologies. Among the reasons behind this discrepancy with respect to the “macro-studies” is a potential publication bias that favors the dissemination of those studies showing “positive” results on academic performance. On the other hand, the specificity of the research objectives and greater precision in the selection and application of the methodology could lead to more in-depth and contextualized studies that better explain this phenomenon. Finally, consideration should be given to the conceptual and cultural differences that exist concerning the role of technologies in education, as well as the definition of academic performance. In subsequent reviews, it would be necessary to analyze how school culture and research perspectives in educational technology can influence the results of this complex relationship between academic performance and ICT (Espíndola et al., 2020).

The main limitation of this review was the use of ERIC as the only database for the study selection process, which, although it is the most relevant database in the field of education, is limited to the English language. Subsequent reviews could consider the inclusion of other databases that increase the number of source documents (e.g., Scopus and WoS) or incorporate other languages such as Spanish (e.g., Dialnet). On the other hand, we chose to use terms strictly from the ERIC Thesaurus in the search queries for articles. The identification of other frequent keywords in the scientific community could have enriched the final selection of articles.

Data Availability Statement

Data is available in the Supplementary Material and https://doi.org/10.5281/zenodo.6426878.

Author Contributions

JV-B: idea, methodology, and writing (original draft). JV-B and JA-B: literature review (state of the art). JV-B, JA-B, and MC-P: data analysis, result, discussion, conclusions, project design, and sponsorships. MC-P and JA-B: final revisions.

Funding

This publication has been made possible through funding granted by the Consejería de Economía, Ciencia y Agenda Digital from the Junta de Extremadura (Spain), and by the European Regional Development Fund of the European Union, through the Reference Grant IB18088. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2022.916502/full#supplementary-material

References

Agasisti, T., Gil-Izquierdo, M., and Han, S. W. (2020). ICT Use at home for school-related tasks: what is the effect on a student's achievement? Empirical evidence from OECD PISA data. Educ. Econ. 28, 601–620. doi: 10.1080/09645292.2020.1822787