AI in higher education: a systematic literature review

Introduction: The increasing reliance on technology within higher education necessitates a thorough examination of artificial intelligence’s (AI) application in academic research. This analysis aims to elucidate both the advantages and challenges associated with AI utilization, thereby paving the way for future inquiries. Such studies will be instrumental in delineating strategies for the effective integration of AI tools in scholarly research, ensuring their optimal use in advancing the field.

Objective: The purpose of this research is to identify the benefits and challenges of the use of AI in the field of scientific research by analyzing experiences that have implemented AI in scientific research carried out at the university level through a systematic literature review.

Method: The research questions that guided the systematic literature review were as follows: (1) What are the benefits of using AI in research? (2) What are the challenges of using AI in research? (3) What are the use and benefits of AI in scientific writing including limitations? (4) What are the main lines of research identified in studies that address scientific practice with artificial intelligence in the university context? The articles analyzed were published in 2023. After applying the inclusion and exclusion criteria, 85 articles were analyzed.

Results: The analysis allowed findings such as the usefulness of ChatGPT in different disciplinary areas, challenges such as being able to identify artificial intelligence resources limitations and benefits such as being able to make processes of different kinds more efficient.

Discussion: It was possible to establish that although the studies analyzed identified advantages in the application of AI in scientific research, it was also detected that it is necessary to have a critical and creative look to make use of AI resources, such as ChatGPT, in order to use them only as support tools and thus be able to take care of the rigor and quality in the elaboration of scientific texts.

1 Introduction

While the origin of artificial intelligence as a scientific discipline date back to 1956 (Hamet and Tremblay, 2017), the launch of ChatGPT in November 2022, as an advanced model of generative artificial intelligence, represented a significant milestone in the field. Although the full potential of generative AI remains partially unexplored in pedagogical applications (Zawacki-Richter et al., 2019), its influence is increasingly recognized in improving adaptive learning, scientific research, and knowledge management (Razia et al., 2023; Rind et al., 2024).

This paper explores the impact of AI on scientific research in higher education through a systematic literature review of databases such as Scopus and Web of Science. These platforms were selected for their rigorous peer review processes, ensuring the quality of the information analyzed. The study not only outlines emerging research trajectories, but also discusses the benefits and challenges associated with AI, with a particular focus on ethical considerations. In addition, the paper reviews the identified advantages of using this technology at different stages of the research process.

The rapid adoption of artificial intelligence technologies, such as ChatGPT, has revolutionized the field of natural language processing and enabled more efficient data analysis, hypothesis generation and research manuscript writing (Dergaa et al., 2023; Gao et al., 2023; Vaishya et al., 2023). In the academic domain, the application of AI has catalyzed a paradigm shift, markedly altering methodologies, improving data processing capabilities, and expanding the horizons of research and scientific writing, across disciplines (Macdonald et al., 2023; Salvagno et al., 2023). For example, in the field of finance, AI supports the generation of research ideas and facilitates data analysis, as highlighted by Dowling and Lucey (2023). In other disciplinary areas such as health sciences, it is also used for data processing, predictive analytics, and experimental design, presenting transformative opportunities as well as ethical challenges, as demonstrated by the work of Crawford et al. (2023) and Lund et al. (2023).

As the academic community continues to grapple with these opportunities and challenges, the need for continued research becomes apparent. Unanswered questions about the scalability of AI applications, the maintenance of ethical standards, and the balance between AI-driven efficiency and human creativity in scientific research underscore the need for a nuanced understanding of AI’s potential to transform university research. In this regard, this review critically assesses the positive and negative impacts of AI, such as those provided by Kooli (2023) and Májovský et al. (2023), who underline the imperative need to monitor the processes where AI is employed, to avoid a fraud factory.

To this end, this study begins with a comprehensive literature review, with the aim of identifying nascent research trajectories or emerging clusters around this topic, cataloging key findings, and elucidating the advantages of integrating AI within research methodologies. In addition, it scrutinizes the challenges previously identified in this area. By conducting this comprehensive analysis, it lays the groundwork for uncovering knowledge gaps and research problems that demand attention in future research efforts. The importance of this study lies in its contribution to the ongoing dialogue on the role of AI in the scientific research landscape.

The structure of this study is developed as follows:

Initially, the introduction outlines the context and objectives of the research, establishing a basis for understanding its significance within the educational field of scientific research in the AI era. Subsequently, the methodology section details the process of generating the search equation and describes the inclusion and exclusion criteria, adhering to PRISMA standards, thus ensuring methodological rigor. The subsequent segment presents the findings in response to the research questions: What are the main research trajectories identified in studies investigating the application of artificial intelligence in the context of scientific practice at the university level; what discoveries have been made; what advantages does the use of AI offer within the field of academic research; and what challenges are encountered when employing AI in academic research? Furthermore, what challenges are encountered when employing AI in this field? The discussion integrates these findings, leading to the articulation of conclusions and the original contributions of the study on the role of AI in scientific research in higher education settings. This approach not only contextualizes the study within the broader academic dialogue, but also highlights its potential to inform future research.

2 Methods

The research method followed was a systematic literature review (SLR), based on the method proposed by Kitchenham and Charters (2007). This method involves specifying the research question in a protocol before starting the review.

An SLR is a methodical and comprehensive approach to identifying, selecting, and critically evaluating all relevant research studies on a specific topic to answer a well-defined research question (Manterola et al., 2013; Gonçalves and David, 2022). SLRs are conducted objectively, rigorously, and meticulously from both a qualitative and quantitative standpoint, and they aim to synthesize the available evidence from primary studies to summarize the existing information on a particular topic. SLRs are considered “studies of studies,” meaning that they compile the information generated by investigations about a specific topic, which is sometimes assessed mathematically with a meta-analysis. SLR utilizes a clear, established, and repeatable methodology to reduce bias and enhance trustworthiness (Manterola et al., 2009, 2013). In contrast to typical literature reviews, systematic reviews adhere to rigorous standards, encompassing the formulation of search strategies, criteria for inclusion, data gathering, and analytical methods. They frequently include several researchers to guarantee impartiality and reliability. Systematic reviews are appreciated for offering dependable evidence to aid in decision-making across different sectors, including healthcare, policy formulation, and research strategy.

SLRs should be assessed critically before deciding whether the conclusions are based on appropriate evidence. This study began with the articles that emerged from a previous systematic literature mapping. Subsequently, quality criteria were defined to refine the selection of articles for the SLR, inclusion and exclusion criteria were also determined, and six research questions were established for the analysis of the articles. There are several guidelines for writing systematic reviews correctly, such as the PRISMA declaration. The PRISMA declaration, or Preferred Reporting Items for Systematic Reviews and Meta-Analyses, is a set of guidelines for researchers to follow when writing reports of systematic reviews and meta-analyses. It aims to improve the transparency, completeness, and comprehensiveness of these reports, which ultimately helps readers better understand the review’s methods, results, and conclusions (Moher et al., 2009; Page et al., 2021). The PRISMA statement includes a checklist of 27 items and a flow diagram divided into four phases. This checklist outlines the essential elements to be reported in systematic reviews and meta-analyses, such as the strategy for searching, criteria for selecting studies, methods for extracting data, and ways of combining findings. The flow diagram visually represents the progression of information through the various stages of a systematic review, starting from the initial search to the ultimate inclusion of studies. The purpose of the PRISMA guidelines is to promote clear and thorough documentation of reviews, thereby improving the trustworthiness and replicability of research outcomes (Moher et al., 2009).

Four specific questions were established to conduct the systematic literature review (Figure 1). The process outlined by the PRISMA statement for conducting a systematic literature review entails the following phases:

Figure 1

Figure 1. PRISMA declaration for the process of searching and filtering.

2.1 Research questions

The general research question was: What are the experiences based on artificial intelligence used in the practice of scientific research carried out in the university context?

Four specific questions were established to conduct the systematic literature review: RQ1-What are the benefits of using AI in research?, RQ2-What are the challenges of using AI in research?, RQ3-What are the use and benefits of AI in scientific writing including limitations?, and RQ4-What are the main lines of research identified in studies that address scientific practice with artificial intelligence in the university context?

2.2 Search strategy

From the 256 articles that emerged from the search process conducted for the previous LMS (CITA) in the Web of Science (97) and Scopus (159) databases, it was decided to use the following filters for the selection of the articles to be handled in the systematic literature review in both databases: artificial intelligence, AI, ChatGPT, research, academic research, scientific research.

2.3 Inclusion, exclusion, and quality criteria

The inclusion and exclusion criteria must capture and incorporate the questions that the SLR seeks to answer, and the criteria must also be practical to apply. The following inclusion criteria were considered: (a) Studies on the use of AI or ChatGPT in tasks related to scientific research in higher education in the WoS and Scopus databases,

(b) Articles published in the period 2020 to 2024, (c) Is the research carried out in educational institutions (HEIs). The following exclusion criteria were considered: (a) Studies that are not related to the use of AI in tasks related to scientific research (b) Documents that come from book chapters or in the press and that consist of reports, series, books, (c) Articles where artificial intelligence is treated as an object of study and not as a resource for scientific research.

In conducting a systematic literature review, crucial elements of quality standards involve evaluating the pertinence, dependability, validity, and utility of the studies considered (Coeytaux et al., 2014). Tools for assessing quality are employed to appraise the methodological integrity of various research designs. These instruments facilitate the rating of each paper according to particular benchmarks, including the design of the study, evaluation of bias, techniques for data gathering, and amalgamation of findings. Selecting an appropriate quality assessment tool that aligns with the anticipated study types in the review is vital for an exhaustive assessment. Finally, after applying the quality criteria, there were 85 articles left to be analyzed in the SLR.

The criteria used in the quality assessment of Systematic Literature Review was the following:

1. Is a focused research question clearly stated?

2. Are the search methods used to identify relevant studies clearly described?

3. Was a comprehensive literature search performed?

4. Was selection bias avoided?

5. Was there duplicate study selection and data extraction?

6. Were the characteristics of the included studies provided?

7. Was the scientific quality of the included studies assessed and documented?

8. Were the methods used to combine the findings of studies appropriate?

9. Was the scientific quality of the included studies used appropriately in formulating conclusions?

10. Was publication bias assessed?

11. Was the conflict of interest stated?

12. Are the stated conclusions supported by the data presented?

3 Results

In this study we focused on four research questions, namely, RQ1-What are the benefits of using AI in research?; RQ2-What are the challenges of using AI in research?; RQ3-What are the use and benefits of AI in scientific writing including limitations?; and finally, RQ4-What are the main lines of research identified in studies that address scientific practice with artificial intelligence in the university context?

In total 35 scientific articles from Scopus database and 50 empirical articles from Web of Science (WoS) have been taken for the purpose of analysis in this study. The articles represent a broad variety of disciplines, such as health, electrical engineering, social sciences, tourism, and alike. In all these articles the common factor was AI with different uses, benefits and challenges.

Regarding RQ1-What are the benefits of using AI in research? The analysis (see Figure 2) shows that 26 articles refer to time saving as the most common advantage of AI in the disciplines consulted, followed by 16 articles that link the benefits to strategies for teaching and learning. Likewise, 12 articles mentioned academic achievement as the main gain, while 7 indicated larger productivity as an important benefit.

Figure 2

Figure 2. Overview findings of the benefits of using AI in research.

Potential for research on new technologies, to streamline tourism processes, benefits in financial research, transformation, to foster academic productivity, content development and automatic monitoring are the least mentioned benefits when it comes to using AI in research. In terms of analyzing the challenges of using AI in research (RQ2) (see Figure 3), the most common challenge (N = 21) is the need to address ethics, the risks for plagiarism, and the lack of transparency when using AI. Furthermore, awareness of possible risks and limitations of the use of these technologies is mentioned N = 18 times as it is relevant to draw attention to AI possibilities as a not fully known tool in research. In addition, providing valuable information has been found 11 times in the articles consulted. Valuable information in the context of these articles is mainly addressed in the context of health care to help make decisions as an instance. Advantages in idea generation and possibilities for further research have been both mentioned 9 times. Advantages in idea generation are mainly included in articles related to education, teaching and learning, research in writing and alike; while possibilities for further research refers to a broader range of disciplines. Finally, efficiency, decrease workload and facilitates research, and objectivity and repeatability in research processes, are the least referred to in the articles.

Figure 3

Figure 3. Overview of the challenges of using AI in research.

With regards to the RQ3-What are the use and benefits of AI in scientific writing and limitations? The study shows (see Figure 4) that 22 articles refer to the limitations address the challenges and the need for human supervision, the same number as Impact and challenges of Generative AI in Scientific Communication; followed by the Use and benefits of AI in scientific writing (N = 20), are the most frequent topics mentioned, together with Ethical considerations (N = 16). In a lower number of mentions in the scientific articles investigated, a few limitations are found, namely, AI content detection; and impact of stress and mental health on students.

Figure 4

Figure 4. Overview of uses and benefits of AI in scientific writing and limitations.

Finally, regarding RQ4-What are the findings of the categories of research articles? In this study it is also attention paid to the different research lines (see Figure 5). Some of the disciplines and research lines are Specific AI applications in disciplines and thematic areas (N = 18); education and personalized support through AI (N = 9); Ethics, impact and normative considerations of AI (N = 7). In small numbers the following themes are found. Research and scientific publications with AI, Evaluation and validation of content generated by AI; and, Tools and applications in research and scientific education. There are also N = 40 articles that do not mention a research line.

Figure 5

Figure 5. Overview of the categories of research lines.

4 Discussion

Being able to make efficient use of time to perform different tasks corresponding to different disciplinary areas has been one of the notable benefits of using AI such as ChatGPT. Figure 1 shows that the benefit of using AI with the highest number of mentions is time savings. The review showed that, especially in the medical area, considerable time savings can be achieved with respect to aspects such as diagnoses or follow-up of patient treatment. For example, Macdonald et al. (2023) state that AI can support accelerating research processes through simulated data to create predictive models, aiding in decision-making processes, such as vaccination strategies. In this way, physicians can have more time to perform other tasks, as they rely on the use of AI to simplify certain tasks. It is worth mentioning that health and medical care are the most frequent themes addressed when it comes to AI in state-of-the-art literature. The advantages to use AI in health that may contribute to the well-being of the society may be among the reasons for this overwhelming literature in the articles consulted. The use of artificial AI has brought benefits, but it has also raised the need to study possible strategies to avoid risks related to ethical misconduct. Regarding the risks of using AI, Figure 2 shows that the most frequently mentioned are: the need to address ethics, the risks for plagiarism, and the lack of transparency when using AI. Because of these risks, research such as Crawford et al. (2023) and Lund et al. (2023) have emerged that examine the ethical considerations and transformative potential of AI, not only in enhancing academic productivity but also in addressing the challenge of plagiarism in academic assignments. The use of artificial intelligence must bring with it a responsible attitude to be clear that it must be used as a support and not to replace human work.

The writing of scientific articles has benefited from AI, but strategies must be put in place to avoid fraud. In relation to RQ3-What are the use and benefits of AI in scientific writing and limitations? What is most mentioned is the need for human supervision. The study by Ariyaratne et al. (2023) establishes a comparison between articles produced by humans and articles produced by AI. The authors reflect on the need to validate or oversee the processes where AI is employed, because AI can be very persuasive in its presentation, but can convey false information that affects the content and accuracy of citations. Human supervision is still necessary to be able to establish with a critical eye and a thorough analysis when items have been produced with artificial intelligence in their entirety and therefore, constitute fraud.

AI has come to revolutionize in several areas, bringing benefits such as process efficiencies, but also leading to reflection on how to maintain integrity. In this study the lines of research that stand out are: Specific applications of AI in disciplines and subject areas (N = 18); Education and personalized support through AI (N = 9); Ethics, impact and regulatory considerations of AI (N = 7) (Figure 4). Areas such as tourism, scientific writing, financial research have benefited from the use of AI, however, several studies have established the importance of ensuring ethics and integrity, such as the one by Kooli (2023) and Májovský et al. (2023), that have delved into the challenges of ensuring data integrity in AI-driven research. Adapting to technological evolution is important, as it can be a great support to help us reduce time and thus make the performance of various activities more efficient, but the responsible use of technology must be present to avoid committing ethical faults.

5 Limitations

Another limitation of this study is that the methods used to investigate the use and challenges of AI are mainly qualitative methods. This may respond to the fact that AI is a relatively new tool which is still used to conduct experiments. Therefore, the data contained in the articles included in this study are in an incipient phase of experimentation. Studying AI with quantitative methods to prove effectiveness takes longer periods of time.

6 Conclusion

This study, through a systematic literature review, reports the benefits of the use of AI in various disciplinary areas, ranging from education, finance, technology, science, medicine and tourism. The study reveals interesting insights of recent experiments with AI that shed light on the use and challenges in different fields. It also identifies challenges such as avoiding plagiarism, lack of transparency or lack of ethics, as well as the great support it can be to streamline scientific writing, but also the great responsibility to continue with careful monitoring to prevent fraud. The research is, therefore, a basis for further research on the integration of AI in the scientific and academic ecosystem and in multiple areas, which will surely be expanded. This study acknowledges the possibility of using artificial intelligence in academic writing and other research areas, but unanimously points out the need for human supervision in these tasks.

Despite the interesting results provided in this study, there is a need to conduct experimental quantitative research to understand deeper the use and benefits of AI, as well as its effectiveness.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

IC-M: Funding acquisition, Investigation, Writing – original draft, Writing – review & editing. DF-B: Investigation, Methodology, Writing – original draft, Writing – review & editing. SG-P: Investigation, Writing – original draft, Writing – review & editing. VV-L: Investigation, Writing – original draft.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. The authors wish to acknowledge the financial and technical support of WritingLab, Institute for the Future of Education, Tecnologico de Monterrey, Mexico, in the production of this work.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ariyaratne, S., Iyengar, K., Nischal, N., Chitti Babu, N., and Botchu, R. (2023). A comparison of ChatGPT-generated articles with human-written articles. Skeletal Radiol. 52, 1755–1758. doi: 10.1007/s00256-023-04340-5

PubMed Abstract | Crossref Full Text | Google Scholar

Coeytaux, R., McDuffie, J., Goode, A., Cassel, S., Porter, W. D., Sharma, P., et al. (2014). Evidence map of yoga for high-impact conditions affecting veterans : Department of Veterans Affairs (US) Available at: https://pubmed.ncbi.nlm.nih.gov/25254284/.

Google Scholar

Crawford, J., Cowling, M., and Allen, K.-A. (2023). Leadership is needed for ethical ChatGPT: character, assessment, and learning using artificial intelligence (AI). J. Univ. Teach. Learn. Pract. 20, 1–10. doi: 10.53761/1.20.3.02

Crossref Full Text | Google Scholar

Dergaa, I., Chamari, K., Zmijewski, P., and Ben Saad, H. (2023). From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing. Biol. Sport 40, 615–622. doi: 10.5114/biolsport.2023.125623

PubMed Abstract | Crossref Full Text | Google Scholar

Dowling, M., and Lucey, B. (2023). ChatGPT for (finance) research: the Bananarama conjecture. Financ. Res. Lett. 53:103662. doi: 10.1016/j.frl.2023.103662

Crossref Full Text | Google Scholar

Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., et al. (2023). Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. Npj. Digit. Med. 6:75. doi: 10.1038/s41746-023-00819-6

PubMed Abstract | Crossref Full Text | Google Scholar

Gonçalves, I., and David, Y. (2022). A systematic literature review of the representations of migration in Brazil and the United Kingdom. Comunicar 30, 49–61. doi: 10.3916/C71-2022-04

Crossref Full Text | Google Scholar

Hamet, P., and Tremblay, J. (2017). Artificial intelligence in medicine. Metab. Clin. Exp. 69S, S36–S40. doi: 10.1016/j.metabol.2017.01.011

PubMed Abstract | Crossref Full Text | Google Scholar

Kitchenham, B., and Charters, S. (2007). Guidelines for performing systematic literature reviews in SE. Keele University and University of Durham.

Google Scholar

Kooli, C. (2023). Chatbots in education and research: a critical examination of ethical implications and solutions. Sustain. For. 15:5614. doi: 10.3390/su15075614

Crossref Full Text | Google Scholar

Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S. R., and Wang, Z. (2023). ChatGPT and a new academic reality: artificial intelligence-written research papers and the ethics of the large language models in scholarly publishing. J. Assoc. Inf. Sci. Technol. 74, 570–581. doi: 10.1002/asi.24750

Crossref Full Text | Google Scholar

Macdonald, C., Adeloye, D., Sheikh, A., and Rudan, I. (2023). Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis. J. Glob. Health 13:01003. doi: 10.7189/jogh.13.01003

Crossref Full Text | Google Scholar

Májovský, M., Černý, M., Kasal, M., Komarc, M., and Netuka, D. (2023). Artificial intelligence can generate fraudulent but authentic-looking scientific medical articles: Pandora’s box has been opened. J. Med. Internet Res. 25:37. doi: 10.2196/46924

Crossref Full Text | Google Scholar

Manterola, C., Astudillo, P., Arias, E., and Claros, N. (2013). Systematic reviews of the literature: what should be known about them. Cirugía Española 91, 149–155. doi: 10.1016/j.cireng.2013.07.003

Crossref Full Text | Google Scholar

Manterola, C., Vial, M., Pineda, V., and Sanhueza, A. (2009). Systematic review of literature with different types of designs. Int. J. Morphol. 27, 1179–1186. doi: 10.4067/S0717-95022009000400035

PubMed Abstract | Crossref Full Text | Google Scholar

Moher, D., Liberati, A., Tetzlaff, J., and Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339:b2535. doi: 10.1136/bmj.b2535

Crossref Full Text | Google Scholar

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Br. Med. J. 372:n71. doi: 10.1136/bmj.n71

PubMed Abstract | Crossref Full Text | Google Scholar

Razia, B., Awwad, B., and Taqi, N. (2023). The relationship between artificial intelligence (AI) and its aspects in higher education. Dev. Learn. Organ. Int. J. 37, 21–23. doi: 10.1108/dlo-04-2022-0074h

Crossref Full Text | Google Scholar

Rind, M. A., Qudah, M. A. A. L., and Aliyev, P. (2024). Determining the impact artificial intelligence on development of higher education. 2024 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS). Applied Science University in Bahrain, Kingdom of Bahrain.

Google Scholar

Salvagno, M., Taccone, F. S., and Gerli, A. G. (2023). Can artificial intelligence help for scientific writing? Crit. Care 27:75. doi: 10.1186/s13054-023-04380-2

Crossref Full Text | Google Scholar

Vaishya, R., Misra, A., and Vaish, A. (2023). ChatGPT: is this version good for healthcare and research? Diab.Metab. Syndr. 17:102744. doi: 10.1016/j.dsx.2023.102744

Crossref Full Text | Google Scholar

Zawacki-Richter, O., Marín, V. I., Bond, M., and Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? High. Educ. 16:39. doi: 10.1186/s41239-019-0171-0

Crossref Full Text | Google Scholar