Skip to main content

ORIGINAL RESEARCH article

Front. Educ.
Sec. Digital Education
Volume 9 - 2024 | doi: 10.3389/feduc.2024.1417900

ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research

Provisionally accepted
Dimitri Prandner Dimitri Prandner 1*Daniela Wetzelhütter Daniela Wetzelhütter 2Sönke Hese Sönke Hese 3
  • 1 Johannes Kepler University of Linz, Linz, Austria
  • 2 Faculty of Medical Technology and Applied Social Sciences, University of Applied Sciences Upper Austria, Linz, Upper Austria, Austria
  • 3 University of Kiel, Kiel, Schleswig-Holstein, Germany

The final, formatted version of the article will be published soon.

    Social scientists are faced with the challenge of designing complex studies and analyzing collected data via various programs such as R, Stata, SPSS, or Python. This often requires the use of analytical procedures and specific software packages that are beyond an individual's established skillsets and technical knowledge. To address these challenges, generative artificial intelligence, such as ChatGPT, can now be employed as 'assistants'-with both associated risks and benefits. Accordingly, this paper explores the potential and pitfalls of using a tool like ChatGPT as an assistant in quantitative data analysis.We investigate the practical use of ChatGPT-3.5 by replicating analyses and findings in everyday scientific research. Unlike previous studies, which have primarily focused optimizing the use of chatbots for code generation, our approach examines an amateur level use of AI tools to support and reference regular research activities, with an emphasis on minimal technical expertise. While we overall conducted three experiments, with the goal to replicate academic papers, the article's focus is on the methodologically most complex one, De Wet et al. (2020). In this case AI is used for the stepby-step replication of the two-dimensional model of value types proposed by Shalom Schwartz (2012).The results of this experiment highlight the challenges of using ChatGPT 3.5 for specific, detailed tasks in academic research, as a tendency for responses to repeat in loops when solutions were not readily available emerged at several stages. Thus, we concluded that there are severe limitations in the AI's ability to provide accurate and comprehensive solutions for complex tasks and emphasize the need for caution and verification when using AI powered tools for complex research procedures.

    Keywords: ChatGPT, Data analisys, AI-support, quantitative analysis, Data analysis - methods

    Received: 16 Apr 2024; Accepted: 13 Dec 2024.

    Copyright: © 2024 Prandner, Wetzelhütter and Hese. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Dimitri Prandner, Johannes Kepler University of Linz, Linz, Austria

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.