Skip to main content

ORIGINAL RESEARCH article

Front. Educ.
Sec. Higher Education
Volume 9 - 2024 | doi: 10.3389/feduc.2024.1429324
This article is part of the Research Topic The Role of Conversational AI in Higher Education View all 3 articles

Comparative Analysis of AI-driven Assistance in Diverse Educational Queries: ChatGPT vs. Google Bard

Provisionally accepted
Mohammad Al Mashagbeh Mohammad Al Mashagbeh 1*Latefa Dardas Latefa Dardas 1Heba Alzaben Heba Alzaben 2Amjad Alkhayat Amjad Alkhayat 3
  • 1 The University of Jordan, Aljubeiha, Jordan
  • 2 Al Hussein Technical University, Amman, Amman, Jordan
  • 3 Al-Balqa Applied University, Al-Salt, Balqa, Jordan

The final, formatted version of the article will be published soon.

    Artificial intelligence tools are rapidly growing in education, highlighting the imperative need for a thorough and critical evaluation of their performance. To this aim, this study tests the effectiveness of ChatGPT and Google Bard in answering a range of questions within the engineering and health sectors. True/False, Multiple Choice Questions (MCQs), matching, short answer, essay, and calculation questions are among the question types investigated. Findings showed that ChatGPT 4 surpasses both ChatGPT 3.5 and Google Bard in terms of creative problemsolving and accuracy across various question types. The highest accuracy achieved by ChatGPT 4 was in True/False questions, reaching 97.5%, while its least accurate performance was noted in calculation questions with an accuracy of 82.5%. Prompting both ChatGPT and Google Bard to provide short responses apparently prevented them from hallucinating with unrealistic or nonsensical responses. The majority of the problems for which ChatGPT and Google Bard provided incorrect answers demonstrated a correct problem-solving approach; however, both AI models struggled to accurately perform simple calculations. In MCQs related to health sciences, ChatGPT seemed to have a challenge in discerning the correct answer among several plausible options. While all three tools managed the essay questions competently, avoiding any blatantly incorrect responses (unlike with other question types), some nuanced differences were noticed. ChatGPT 3.5 consistently adhered more closely to the essay prompts, providing straightforward and essential responses, while ChatGPT 4 demonstrated superiority over both models in terms of adaptability. ChatGPT4 fabricated references, creating nonexistent authors and research titles in response to prompts for sources. While utilizing AI in education holds a promise, even the latest and most advanced versions of ChatGPT and Google Bard were not able to accurately answer all questions. There remains a significant need for human cognitive skills and further advancements in AI capabilities.

    Keywords: ChatGPT, Google Bard, Question types, AI chatbots, Education

    Received: 07 May 2024; Accepted: 06 Sep 2024.

    Copyright: © 2024 Al Mashagbeh, Dardas, Alzaben and Alkhayat. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Mohammad Al Mashagbeh, The University of Jordan, Aljubeiha, Jordan

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.