Skip to main content

ORIGINAL RESEARCH article

Front. Comput. Neurosci.
Volume 18 - 2024 | doi: 10.3389/fncom.2024.1516440
This article is part of the Research Topic Advancements in Smart Diagnostics for Understanding Neurological Behaviors and Biosensing Applications View all articles

Multimodal Consumer Choice Prediction Using EEG Signals and Eye Tracking

Provisionally accepted
Aimen Tanveer Aimen Tanveer 1Syed Muhammad Usman Syed Muhammad Usman 2Shehzad Khalid Shehzad Khalid 2Ali Shariq Imran Ali Shariq Imran 3*Muhammad Zubair Muhammad Zubair 4
  • 1 Air University, Islamabad, Islamabad, Pakistan
  • 2 Bahria University, Islamabad, Islamabad, Pakistan
  • 3 Norwegian University of Science and Technology, Trondheim, Norway
  • 4 University of Central Punjab, Lahore, Punjab, Pakistan

The final, formatted version of the article will be published soon.

    Marketing plays a vital role in the success of a business, driving customer engagement, brand recognition, and revenue growth. Neuromarketing adds depth to this by employing insights into consumer behavior through brain activity and emotional responses to create more effective marketing strategies. Electroencephalogram (EEG) has typically been utilized by researchers for neuromarketing, whereas Eye Tracking (ET) has remained unexplored. To address this gap, we propose a novel multimodal approach to predict consumer choices by integrating EEG and ET data. Noise from EEG signals is mitigated using a bandpass filter, Artifact Subspace Reconstruction (ASR), and Fast Orthogonal Regression for Classification and Estimation (FORCE). Class imbalance is handled by employing the Synthetic Minority Oversampling Technique (SMOTE). Handcrafted features, including statistical and wavelet features, and automated features from Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM), have been extracted and concatenated to generate a feature space representation. For ET data, preprocessing involved interpolation, gaze plots, and SMOTE, followed by feature extraction using LeNet-5 and handcrafted features like fixations and saccades. Multimodal feature space representation was generated by performing feature-level fusion for EEG and ET, which was later fed into a meta-learner-based ensemble classifier with three base classifiers, including Random Forest, Extended Gradient Boosting, and Gradient Boosting, and Random Forest as the meta-classifier, to perform classification between buy versus not buy. The performance of the proposed approach is evaluated using a variety of performance metrics, including accuracy, precision, recall, and F1 score. Our model demonstrated superior performance compared to competitors by achieving 84.01% accuracy in predicting consumer choices and 83% precision in identifying positive consumer preferences.

    Keywords: EEG, eye tracking, neuromarketing, CNN-LSTM, multimodal

    Received: 24 Oct 2024; Accepted: 19 Dec 2024.

    Copyright: © 2024 Tanveer, Usman, Khalid, Imran and Zubair. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Ali Shariq Imran, Norwegian University of Science and Technology, Trondheim, Norway

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.