Skip to main content

ORIGINAL RESEARCH article

Front. Res. Metr. Anal.
Sec. Research Methods
Volume 9 - 2024 | doi: 10.3389/frma.2024.1432774

AI-Powered Fraud and the Erosion of Online Survey Integrity: An Analysis of 31 Fraud Detection Strategies

Provisionally accepted
Natalia Pinzón Natalia Pinzón 1,2*Vikram Koundinya Vikram Koundinya 3,4*Ryan E. Galt Ryan E. Galt 3,5William O'R. Dowling William O'R. Dowling 2Marcela Baukloh Marcela Baukloh 2Namah C. Taku-Forchu Namah C. Taku-Forchu 6Tracy Schohr Tracy Schohr 4Leslie M. Roche Leslie M. Roche 4,7Samuel Ikendi Samuel Ikendi 8Mark Cooper Mark Cooper 3Lauren E Parker Lauren E Parker 10,9Tapan B. Pathak Tapan B. Pathak 11,4
  • 1 Geography Graduate Group, University of California, Davis, Davis, United States
  • 2 Rhizobia, LLC, San Francisco, United States
  • 3 Department of Human Ecology, University of California, Davis, Davis, United States
  • 4 University of California Cooperative Extension, Davis, California, United States
  • 5 Agricultural Sustainability Institute, College of Agricultural and Environmental Sciences, University of California, Davis, Davis, California, United States
  • 6 Natural Resources Institute, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • 7 Department of Plant Sciences, University of California, Davis, Davis, United States
  • 8 Division of Agriculture and Natural Resources, University of California, Merced, Merced, California, United States
  • 9 California Climate Hub, United States Department of Agriculture (USDA), Davis, California, United States
  • 10 Institute of the Environment, University of California, Davis, Davis, California, United States
  • 11 Department of Civil and Environmental Engineering, University of California, Merced, Merced, California, United States

The final, formatted version of the article will be published soon.

    The proliferation of AI-powered bots and sophisticated fraudsters poses a significant threat to the integrity of scientific studies reliant on online surveys across diverse disciplines, including health, social, environmental and political sciences. We found a substantial decline in usable responses from online surveys from 75% to 10% in recent years due to survey fraud. Monetary incentives attract sophisticated fraudsters capable of mimicking genuine open-ended responses and verifying information submitted months prior, showcasing the advanced capabilities of online survey fraud today. This study evaluates the efficacy of 31 fraud indicators and 6 ensembles using two agriculture surveys in California. To evaluate the performance of each indicator, we use predictive power and recall. Predictive power is a novel variation of precision introduced in this study, and both are simple metrics that allow for non-academic survey practitioners to replicate our methods. The best indicators included a novel email address score, MinFraud Risk Score, consecutive submissions, opting-out of incentives, improbable location,

    Keywords: surveys, online data collection, Fraud detection, survey farms, AI bots

    Received: 14 May 2024; Accepted: 08 Nov 2024.

    Copyright: © 2024 Pinzón, Koundinya, Galt, Dowling, Baukloh, Taku-Forchu, Schohr, Roche, Ikendi, Cooper, Parker and Pathak. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence:
    Natalia Pinzón, Geography Graduate Group, University of California, Davis, Davis, United States
    Vikram Koundinya, Department of Human Ecology, University of California, Davis, Davis, United States

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.