Farzad Saffari
Shobhit Kakaria
Enrique Bigné
Luis E. Bruni2
Sahar Zarei
Thomas Z. Ramsøy- 1Neurons Inc., Høje-Taastrup Municipality, Denmark
- 2Augmented Cognition Lab, Aalborg University, Copenhagen, Denmark
- 3Faculty of Economics, University of Valencia, Valencia, Spain
Introduction: Consumer decision-making processes involve a complex interrelation between perception, emotion, and cognition. Despite a vast and diverse literature, little effort has been invested in investigating the neural mechanism behind such processes.
Methods: In the present work, our interest was to investigate whether asymmetrical activation of the frontal lobe of the brain could help to characterize consumer’s choices. To obtain stronger experimental control, we devised an experiment in a virtual reality retail store, while simultaneously recording participant brain responses using electroencephalogram (EEG). During the virtual store test, participants completed two tasks; first, to choose items from a predefined shopping list, a phase we termed as “planned purchase”. Second, subjects were instructed that they could also choose products that were not on the list, which we labeled as “unplanned purchase.” We assumed that the planned purchases were associated with a stronger cognitive engagement, and the second task was more reliant on immediate emotional responses.
Results: By analyzing the EEG data based on frontal asymmetry measures, we find that frontal asymmetry in the gamma band reflected the distinction between planned and unplanned decisions, where unplanned purchases were accompanied by stronger asymmetry deflections (relative frontal left activity was higher). In addition, frontal asymmetry in the alpha, beta, and gamma ranges illustrate clear differences between choices and no-choices periods during the shopping tasks.
Discussion: These results are discussed in light of the distinction between planned and unplanned purchase in consumer situations, how this is reflected in the relative cognitive and emotional brain responses, and more generally how this can influence research in the emerging area of virtual and augmented shopping.
Introduction
Our everyday behaviors and choices result from a complex interplay between our perceptions, emotions, and cognition. Choices we make range from highly planned to direct and impulsive behaviors. This distinction is often seen in well-controlled studies in psychological and cognitive neuroscience. The distinction between decisions as ranging from cognitive to emotional responses has been investigated in various disciplines, i.e., neuroscience, philosophy, psychology, and commercial studies (Schall, 2001). Most notably, behavioral studies have long held a view of a split between an emotional and impulsive system, and a more cognitively driven deliberation system (Frankish, 2010). This has also been demonstrated in neuroimaging studies showing a distinction between two separate neural systems for immediate and delayed rewards (McClure et al., 2004).
However, less is known about the role of dual-process (an interplay between our emotion and cognition) decisions in real-life situations, especially in the ubiquitous digital reality in which consumer behaviors are embedded in our days. Numerous studies in neuroscience have been examining consumers’ shopping behavior (Martin and Potts, 2009; Ravaja et al., 2013; Gable et al., 2015; Ramsøy et al., 2018; Goto et al., 2019; Liao et al., 2019; Shih et al., 2019; Ozkara and Bagozzi, 2021). With the help of quantitative measurement in neuroscience such as electroencephalogram (EEG), these relatively vague concepts can be analyzed and furtherly explore consumer decisions. The most recent studies in neuromarketing have been focused on consumer’s purchase intention and decisions (purchase vs. no-purchase) (Shang et al., 2020; Wang et al., 2021), or predicting consumer’s purchases (Golnar-Nik et al., 2019; Bak et al., 2022). But fewer studies have gone beyond “buy” and “no buy” and the relative contribution of cognitive and emotional process on consumer motivation (Aditya and Sarno, 2018; Royo-Vela and Varga, 2022) and the distinction of consumers decisions have not been thoroughly investigated.
Human emotion plays a central role in our everyday decisions (Rutherford and Lindell, 2011; Harmon-Jones et al., 2012). Previous research has shown that our positive or negative perceptions could be explained by approach\avoidance behavior (Rutherford and Lindell, 2011; Harmon-Jones et al., 2012). When it comes to human shopping behavior, the reasoning behind our decision becomes more complex and difficult to explain (Ohme et al., 2010; Wixted, 2018; Karmarkar and Plassmann, 2019; Neal and Gable, 2019; Sänger, 2019).
Lateralized function of the brain has been the core of many neuroscientific studies on emotion (Petrantonakis and Hadjileontiadis, 2011; Gable et al., 2015), motivation (Coan et al., 1997; Poole and Gable, 2014), neurological and psychiatric disorders (Keune et al., 2015; Lin et al., 2021). Specifically, frontal asymmetry scores are one of the most promising features that have been extracted from EEG data to quantify our valence and approach-avoidance behavior (Coan et al., 1997; Davidson, 2004; Ravaja et al., 2013; Poole and Gable, 2014; Wixted, 2018).
Frontal asymmetry has been traditionally used by neuroscientists as an indicator of human emotions (Smith et al., 2017), which relies on the theory that the left prefrontal cortex is more activated for emotions with positive valence compared to those with negative valence, which induce relatively less left prefrontal activity. Moreover, the approach/avoidance behavior has been studied in this manner as a motivation index. Most of these studies have been focused on the alpha frequency band, which relates to the inhibitory function of the brain and a higher relative power in this band indicates less cortical activation (Smith et al., 2017). However, less attention has been paid to the other EEG frequency bands such as beta and gamma (Ravaja et al., 2013; Ramsøy et al., 2018; Spironelli et al., 2021).
The theta-alpha ratio (TAR) has been recently used as a cognitive load index which is sensitive to task difficulty and cognitive processing (Trammell et al., 2017; Cabañero et al., 2019; Wang et al., 2020). Here, we assumed that the TAR index would vary as participants switched their shopping behavior from decisions that were mainly emotion-based, and hence frontal asymmetry based, to a more cognitive state of planned behavior, which we expected to be related to stronger TAR scores.
Consumer behaviors have recently changed from being physically based in brick-and-mortar stores to moving online (Cheung et al., 2005), and finally now on the verge of moving fully into mixed reality (MR) (Shen et al., 2021). Recent research in Virtual Reality (VR) tried to combine VR and supermarket as a research tool (Peschel et al., 2022), or as a teaching application for Autistic patients (Thomsen and Adjorlu, 2021). Also, VR supermarkets have been used to investigate consumers’ behavior in choosing healthy food (Eichhorn et al., 2021; Melendrez-Ruiz et al., 2021), or with the help artificial intelligence, virtual supermarket has been studied as a “shopping at home” avenue (Shravani et al., 2021). However, to the best of our knowledge, the effect of virtual supermarkets on consumers’ subconscious response has not been thoroughly investigated via psychophysiological tools such as EEG. Thus, there is a need for a better understanding of consumer behaviors and relative underlying neural responses in general, as well as in the new digital interfaces that consumers are currently moving into.
Recently, different effects of VR on our brain responses have been measured through EEG (D’Errico et al., 2020; Dini et al., 2022). Due to VR engagement and the semi-real environment that these technologies provide, they could facilitate examining neural responses in a more realistic way. In Schaefer et al. (2016) a VR shopping task has been conducted to investigate the effect of price expectation violation on the P300 component of EEG and in Rosenlacher et al. (2018) the effects of a VR store on human shopping behavior have been studied. Besides consumer neuroscience, VR and MR have been used as a treatment tool (Tran et al., 2022), or as an educational avenue (Makransky et al., 2019) in neuroscientific studies. Specifically, as there is now a surge in interest for implementing MR worlds as a solution for everyday situations (Bazzani et al., 2020)—increasingly referred to as the “metaverse”—our knowledge about attentional, emotional, and cognitive responses is woefully lacking. While we can, a priori, assume that the responses we see inside MR are the same as those that we see outside MR, few studies have been conducted to test this assumption. Hence, although numerous neuroscientific studies tried to investigate the neural mechanism behind consumers’ decision-making process, the validity of the finding of those studies have not been tested in a real-life scenario.
In this study, we designed an experiment in a VR supermarket, with minimal experimental control and high ecological validity, to investigate whether there is a difference in consumers’ relative involvement of cognitive and emotional processes in planned vs. unplanned decisions, respectively. Participants went through two experimental phases while being in the VR store. First, they were asked to buy products following a predefined shopping list. Second, they were instructed to also buy whatever they wanted from the store.
We posed two main hypotheses. First, following prior research on frontal asymmetry and choice, we expected stronger frontal asymmetry responses when consumers made choices, compared to phases of product inspection with no choice, regardless of whether the choice was planned or unplanned. More specifically, frontal asymmetry, as a representation of participants’ approach behavior, is assumed to be higher in choice trials compared to the no-choice trials, i.e., periods of the experiment in which participants are navigating or searching for products.
Second, we assumed that planned purchases would rely more on cognitive brain responses implying higher cognitive load, while spontaneous, unplanned purchases would be more driven by emotional brain responses. Therefore, we hypothesized that unplanned purchases would be related to a stronger frontal asymmetry score during product choice, compared to planned purchases. Conversely, we hypothesized that planned purchases would be associated with a relatively lower frontal asymmetry and a higher degree of cognitive load.
Materials and methods
A total of 30 (14 female, 16 male) right-handed subjects (age range 23–44, mean = 31.8, std = 6.6) were recruited in the experiment using Neurons Inc1 online recruitment procedures. Since we aimed to compare the EEG responses in two conditions, we only included the data from the subjects who fulfilled both tasks in the experiment which reduced the number of subjects to 27 (age range 23–44, mean = 31.6, std = 6.9). From these participants that we consider the data for the study, eight of them have previous experience with VR and using the controllers and two of them have participated in a study with virtual supermarket previously. All participants read and signed the consent form, and they were informed about the experimental procedure prior to the data collection. This study was performed in accordance with the Declaration of Helsinki, the rules and laws of the Danish Data Protection Agency, the European Union law of the General Data Protection Regulation, as well as the ethical regulations imposed by the Neuromarketing Science and Business Association, Article 64. Each person’s biometric data, survey responses, and other types of data were anonymized and only contained the log number as the unique identifier. No personally relevant data could be extracted from the log number.
Experimental design
A supermarket environment was designed in Unreal Engine V4.12 and implemented in a VR HTC Vive 5.3 The VR supermarket was comparable to the supermarkets in Denmark in terms of appearance. Participants had two controllers (which appeared as hands in the VR) that they can use to choose the items and teleport through the VR. They could also walk (for one or two steps in the physical space), but we asked them to limit their movement in order to keep them in VR environments. All of the items in the supermarket were accessible and participants could choose whatever they see in the store through free navigation. We have allocated a budget of 250 Danish Kroner (DKK) (∼$35) for each participant to spend on these two shopping tasks. Therefore, participants were free to distribute their budget between these two shopping tasks and prevent the neurological effect of product price suggested by Tang and Song (2019) in online context. We provided a list of products in the VR for the participants, which contained six items (Broccoli, Milk, Cheese, Soda, Cereal, and Chocolate). The range of the cost of these items was between 80 and 120 DKK. Participants need to purchase all of these items, which we call as planned purchase condition. With the remaining amount of budget, they could buy whatever they wanted in the store (Unplanned Purchase Condition). They could leave the environment as soon as they had bought the items they wanted. The number of planned purchases (items from the list) was fixed, and the number of unplanned purchases could be different among the subjects, as they were free to choose any item they wanted if it was within their budget. In Figure 1, the experimental procedure of the experiment is illustrated. In the upper side of the figure, the environment for the virtual supermarket is presented.
Figure 1. The environment for virtual supermarket and EEG data collection procedure is presented. The red arrows show the time points in which they choose a product. We consider the 3 s data before that event as a trial. Afterward the trials for each condition have been concatenated for further analysis.
The EEG data were recorded during the whole shopping task. After mounting the VR and EEG cap, we instructed the participant on how to use the VR so they could start the experiment and navigate through the store. The list of the products that they needed to purchase was available for them by hitting a button on the controller. In the first phase, they were required to buy all the products from the list. Afterward, we informed them about the remaining budget. Then, they could start the second phase of the experiment. On average, participants spent 238.87 ± 85.57 s on the planned purchase condition, and 228.00 ± 107.20 s on the unplanned purchase condition. The time-points of each purchase (i.e., when the participants choose an item) have been recorded during the experiment.
EEG recording
Electroencephalogram data were collected using a monopolar 32 channels (gel-based) EEG device (Brain Products, ActiCap) with 500 Hz sampling rate. The EEG sensor locations with the standard 10–20 system were positioned at Fp1, Fp2, F8, F4, Fz, F3, F7, FT9, FT10, FC5, FC1, FC2, FC6, T7, T8, C3, Cz, C4, CP5, CP1, CP2, CP6, TP9, TP10, P7, P3, Pz, P4, P8, O1, Oz, O2. The ground and reference electrodes were located at Afz and FCz, respectively, for online processing. The data were transmitted wirelessly from the amplifier (LiveAmp32) to a PC using a USB module. The impedance of the electrodes was monitored prior to the data collection, and it was maintained below 15 KΩ.
EEG analysis
For the EEG data analysis, we used the MNE tool (Gramfort, 2013). First, the data were filtered using a FIR bandpass filter with a hamming window for the 0.1 and 100 Hz frequency bands. A notch filter at 50 Hz was also applied to remove the power line noises. Independent Component Analysis (Jung et al., 1998) was applied to clean the data and manually remove the “bad” components by visual inspection. Afterward, common average referencing and baseline correction methods were used for pre-processing procedures.
Based on a recent study (Ramsøy et al., 2018), we considered 3 s epochs of the data before each participant chose an item (in both conditions) for our analysis. The rest of the data, which was not included in choosing the products was also epoched with 3 s length for the consistency of the analysis. The power spectrum of the signal was calculated using a welch method (with 256 numbers of FFT points equivalent to 512 milliseconds) for each frequency band (theta [4–8], alpha [8–13], beta [13–25], gamma [30–40]). For each epoch, the power spectrum was calculated by averaging over frequency bins and then averaged over all epochs to represent the power spectrum of that condition for each subject.
EEG feature extraction
Frontal asymmetry
The frontal asymmetry score is a well-established EEG feature to indicate the lateralization effect of emotional processing in the brain (Lee et al., 2020; Lin et al., 2021; Yang et al., 2021). Frontal asymmetry is calculated by subtracting the log-transformed power values of frontal channel F7 from frontal channel F8 and divided by the sum of the power of both electrodes, as shown in Equation 1. Frontal asymmetry was measured for alpha, beta, and gamma frequency bands. It should be noted that the alpha scores are not multiplied by −1 to indicate the “de-activation” of the alpha band.
Theta-alpha ratio
TAR indexes the cognitive load value by using cross-frequency activity in the frontal and parietal lobes. As shown in Equation 2, power values in the theta frequency band from F7 and F8 channels were normalized to power values in the alpha frequency band at the parietal lobe (P3 and P4) (Wang et al., 2020).
Statistical method
As described in the experimental design section, we asked each participant to purchase six items from a predefined list, and then they could purchase whatever they wanted with the remaining budget. Accordingly, this led to an imbalance of trials, both within each subject (between each condition) and between subjects. To solve this imbalance, we applied a permutation test, which previously has been applied to resolve the imbalance of trials issue in an ERP study (Files et al., 2016). As shown in Figure 2, for each participant, we first calculated the desired feature (asymmetry score in different frequency bands) from the trials of each condition. We subtracted those values of one condition from the other to calculate the difference between the features. By repeating this procedure for all subjects, we computed the average of these values for all the participants to build up the “ground truth” values for that comparison.
Figure 2. The implemented statistical method for considering the imbalanced trials. In section (A), the calculation of feature values has been shown. In the upper section, features from the actual conditions (V1, V2) have been calculated and then subtracted for each subject. In the lower section of (A) the same procedure is happening but with random shuffling of the labels across trials for 1,000 iterations. In section (B) we averaged the outcome of section (A) over subjects which the upper section shows for the actual condition and the lower section is for each iteration. In section (C) the ground truth value from the actual conditions and null distribution from the random shuffling is illustrated in the upper and lower side respectively.
Next, for generating data-driven null distribution, we randomly shuffled the trial’s labels between conditions for each subject, and then repeated the same procedure that we did to calculate the ground truth. By performing this procedure 1,000 times, we could have a null distribution to compare the ground truth value in the two conditions. Ultimately, if the ground truth is far enough (based on the significance level which here is 0.05 before Bonferroni’s correction) from the mean of this distribution, we could state that our findings are not due to randomness. Due to multiple comparisons, the significance level is 0.05/3 = 0.01.
We applied this statistical method to compare the asymmetry scores in alpha, beta, and gamma frequency bands, in the two given conditions: purchase vs. non-purchase, and planned purchase vs. unplanned purchase.
Results
Frontal asymmetry
Looking at the distinction between purchase and non-purchase conditions, we find a significant difference in alpha, beta, and gamma asymmetry scores (F7, and F8 channels) between the two conditions. The results of the beta asymmetry scores for these two conditions are presented in Figure 3A as an illustrative example to show the significance level (P-value < 0.01). For the rest of the frequency bands, the significance plots were similar to Figure 3, therefore, we are not presenting the plots here to avoid repetition. As it is shown in Figure 3, the ground truth of asymmetry scores was far from the mean of null distribution which makes the ground truth value statistically significant.
Figure 3. (A) The distribution and ground truth values of the beta asymmetry score differences in the two conditions (purchase vs. Non-purchase). The distribution derives from calculating the difference of the beta asymmetry score between two conditions for 1,000 times and shuffling the labels among trials. The blue vertical line is our ground truth which indicates the same difference but for the actual conditions. (B) Same as the above plot but for planned and unplanned conditions in gamma frequency band.
In the alpha frequency band, the asymmetry mean scores for purchase and non-purchase conditions over all subjects were negative and the score for the purchase phase (−0.001 ± 0.008) was higher (p-value < 0.001, 1000 permutations) than the non-purchase phase (−0.005 ± 0.006). For beta and gamma frequency bands, the mean asymmetry scores were positive for the purchase phase (0.0001 ± 0.009 and 0.0008 ± 0.011) and negative for the non-purchase phase (−0.005 ± 0.007 and −0.006 ± 0.008) with p-value < 0.001 and 1,000 permutations. The violin plots of these results are presented in Figure 4.
Figure 4. The violin plots of asymmetry scores of three frequency bands for all subjects. The purchase and non-purchase conditions are shown with blue and purple colors, respectively. The red and yellow line in the middle illustrating the mean and median values, respectively. The difference between the asymmetry scores were statistically significant (*p-value < 0.001) in all tree frequency bands.
For the comparison between planned and unplanned purchase, there was a trend toward a significant difference in the range of alpha (P-value = 0.08) and beta (P-value = 0.04) asymmetry scores before correction, between these two conditions. However, in the gamma band, the difference between mean asymmetry scores of the unplanned purchase (0.001 ± 0.011) and planned purchase (−0.002 ± 0.012) was statistically significant (P-value < 0.001, 1000 permutations). The results of the statistical method for the gamma frequency band are presented in Figure 3B, and the violin plots of asymmetry scores for different frequency bands are presented in Figure 5.
Figure 5. The violin plots of asymmetry scores of three frequency bands for all subjects. The planned purchase and unplanned purchase conditions are shown with dark-blue and light-blue colors, respectively. The red and yellow line in the middle illustrating the mean and median values, respectively. The difference between the asymmetry scores was statistically significant (*p-value < 0.001) in gamma frequency band.
The asymmetry scores in alpha, beta, and gamma frequency bands for each of the products in planned purchase are presented in Figure 6 (averaged for all subjects). For each of the six products, the alpha asymmetry scores are positive (considering the multiplication by −1) and chocolate has the highest alpha asymmetry. For beta and gamma asymmetry, the scores are both negative and positive. Cereal and chocolate have the highest beta and gamma asymmetry scores respectively.
Figure 6. Asymmetry scores in alpha, beta, and gamma frequency bands for each item in planned purchase. Asymmetry scores in alpha have been multiply by negative 1 to represent a same behavior as other frequency bands.
In addition, we have compared the asymmetry score in the gamma frequency band of each second (from the 3-s time window) before choosing an item. As it is shown in Figure 7, for both planned and unplanned purchases, as participants become closer to choosing an item, the gamma asymmetry score increases. For unplanned purchase, the asymmetry score starts from negative and become positive when there is a 2 s before choosing an item. However, for the planned purchases, even though the asymmetry score increased as we get closer to choosing a product, the asymmetry score remains negative for each of the three phases before choosing an item.
Figure 7. Gamma asymmetry score (averaged for all subject over all purchased items) for the three 1-s epochs before choosing a product. The horizontal axis shows the three time-intervals (within the 3 s window) before choosing a product.
Theta alpha ratio
Comparing TAR values of planned and unplanned purchase, we found a significant difference between the given conditions with p-value < 0.0001. As represented in Figure 8, the averaged TAR index over all subjects was 7.04 ± 4.35 for planned condition and 4.28 ± 1.92 for unplanned condition.
Figure 8. The violin plots of TAR index across all subjects. The planned purchase and unplanned purchase conditions are shown with dark-blue and light-blue colors, respectively. The red and yellow lines in the middle illustrate the mean and median values, respectively.
Discussion
In this study, we sought to test whether consumer choices in a VR supermarket can modulate brain frontal asymmetry in alpha, beta, and gamma frequency bands. In addition, we were interested in testing whether planned and unplanned choices would be related to similar changes in emotional and cognitive responses such as frontal asymmetry and TAR, respectively.
In doing so, we found that regardless of the types of participants’ purchases, there was a significant difference in alpha, beta, and gamma asymmetry scores comparing trials that involved choosing a product, compared to other phases of store task completion, such as navigating the environment. We also showed that different types of consumer choices (planned and unplanned), were related to a modulation of the gamma asymmetry scores but not the alpha or beta frontal asymmetries.
Our findings for comparison of “choice vs. no-choice” were partially in line with the previous studies regarding frontal asymmetry and approach behavior. Previous research has shown that since alpha oscillation is more correlated with inhibitory activities, relatively higher right frontal activity is related to the approach behavior in this frequency band (Harmon-Jones and Allen, 1998; Pizzagalli et al., 2005; Ravaja et al., 2013; Smith et al., 2017). However, our results showed a contrast in alpha frontal asymmetry, in which there was a relatively higher right frontal activity in the no-choice phase. For the beta and gamma frequency bands, there was a relatively higher frontal asymmetry during the “choice” conditions, which was in line with previous research that relates higher left frontal beta activity compared to right frontal activity to consumers’ purchase decisions (Aftanas et al., 2006; Ramsøy et al., 2018; Zhao et al., 2018; Alarcão and Fonseca, 2019; Lin et al., 2021; Yuan et al., 2022).
Our second hypothesis was concerned with the comparison between planned and unplanned choices. Here, our results confirmed our hypothesis as there was a significantly higher left than right frontal activity in the gamma frequency band during unplanned conditions, relative to planned choices. As some recent studies suggested (Ramsøy et al., 2018; Tarrant and Cope, 2018), we found a greater gamma asymmetry score for unplanned choices compared to planned choices. These findings provide new insights into the role of gamma frequency band activity in decision making particularly in consumer neuroscience.
Moreover, when looking at the cognitive index of the TAR score for the planned and unplanned conditions, our results show that planned purchases were associated with relatively higher levels of the cognitive load than unplanned purchases. This is in line with our assumption and prior findings and suggests that planned behavior is reflected as a theta-alpha activity ratio even in a virtual shopping environment. As opposed to most of the research in the field of consumer neuroscience, which mainly focus on consumers’ “purchase” and “no purchase” (Rosenlacher et al., 2018; Golnar-Nik et al., 2019; Goto et al., 2019; Eichhorn et al., 2021; Melendrez-Ruiz et al., 2021; Horr et al., 2022) we have extended this view by looking deeper to the relative contribution of cognitive and emotional response and the dual-process of consumer’s decision-making process. Additionally, one of the important contributions of this study is that, compared to highly controlled lab experiments, it adds a higher ecological validity to the research by using an immersive VR supermarket. Most previous studies that have investigated emotions and frontal asymmetry have been customarily conducted in lab settings. However, with the current study we can expand the interpretation more widely into our daily decisions. Also, little is known about the effects of MR systems on human emotional and cognitive responses. The present research shows that those findings can be partially reproduced in a VR-based experiment which is a closer simulation of real-life conditions.
As for the limitations of this study that also prompt future research, it should be noted that these findings should be tested in other VR settings to eliminate the sensitivity of the results to a specific environment. Additionally, other studies should seek to do a direct comparison of virtual and real store environments to ensure that there is indeed a high degree of reliability of the identified emotional and cognitive scores across environments. In addition, although the feedback we got from the participants was overally positive, there were some limitations to the environment (e.g., they could not carry the products to the cashier, or they could not find some types of products in the supermarket) which should be improved and lead us to novel hypotheses such as the role of product type on frontal asymmetry.
Another limitation of this is study is that we did not consider personal preferences (which lead to a higher motivation) of the participants for each product. Collecting the data on personal preferences for each product that participants have bought within both planned and unplanned purchases, a comprehensive comparison of participants’ motivation resulted from EEG for each item and its relation to their behavioral response could be conducted.
Another limitation of this study is the imbalance of trials and fewer number of trials for one condition. This limitation was mainly because of our attempt to keep a high degree of ecological validity and thereby providing a shopping experience as realistic as possible for a daily shopping routine. Also, we sought to limit the experiment duration, as we had to consider elements of fatigue which is often caused by staying in a VR environment for a relatively long time. In our analysis, we fill this “gap” with the aforementioned statistical methods, to overcome the imbalance trials comparison. To further abate this, another solution could be to increase the number of participants and make a between-subject study. By doing so, there would be more trials for each condition, and the outcome of the statistical analysis would be less affected by imbalance trials. As a result, the participants would be less likely to experience fatigue during the VR experiments.
Conclusion
In conclusion, we investigated the role of frontal asymmetry and TAR in explaining consumer choices in a virtual store. Although most previous studies have focused on frontal asymmetry in the alpha frequency band, we showed that information in the gamma band can explain consumers’ behavior more precisely. By conducting an experiment in a VR supermarket and two shopping tasks (planned and unplanned purchases) first we compared trials related to consumer choices and trials related to their navigation and searching products. We found a clear difference in alpha, beta, and gamma frontal asymmetry. In addition, we compared trials within the consumer choices (planned and unplanned choices), and results showed that gamma information in frontal asymmetry and TAR could discriminate those choices, which neither alpha nor beta frequency information would be apt to explain the difference.
The presented study tested previous findings related to frontal asymmetry in the consumer decision-making process. In addition, we confirmed previous finding in a semi-realistic VR environment, which makes the current study fruitful for consumer neuroscience and the VR research field. These findings and further research on the dual-process nature of consumer choices in MR systems will become increasingly important as emerging technological paradigms such as the Metaverse become distributed at the level that non-immersive social media have reached today.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Technical Faculty of IT and Design, Aalborg University. The patients/participants provided their written informed consent to participate in this study.
Author contributions
FS was responsible for designing the experiment, collecting the data, pre-processing and analyzing the data, interpreting the result, writing the manuscript, and preparing the figures. SK was being responsible for designing the experiment and collecting the data. EB worked on the study design and contributed to the manuscript. LB and SZ contributed to the manuscript. TR was responsible for study design and interpreting the results contributed to the manuscript. All authors contributed to the article and approved the submitted version.
Funding
This work was supported by Rhumbo (European Union’s Horizon 2020 Research and Innovation Program under the Marie Skłodowska-Curie Grant Agreement No 813234).
Conflict of interest
FS, SZ, and TR were employed by Neurons Inc.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
References
Aditya, D., and Sarno, R. (2018). Neuromarketing: State of the arts. Adv. Sci. Lett. 24, 9307–9310. doi: 10.1166/asl.2018.12261
Aftanas, L. I., Reva, N. V., Savotina, L. N., and Makhnev, V. P. (2006). Neurophysiological correlates of induced discrete emotions in humans: An individually oriented analysis. Neurosci. Behav. Physiol. 36, 119–130. doi: 10.1007/s11055-005-0170-6
Alarcão, S. M., and Fonseca, M. J. (2019). Emotions recognition using EEG signals: A survey. IEEE Trans. Affect. Comput. 10, 374–393. doi: 10.1109/TAFFC.2017.2714671
Bak, S., Jeong, Y., Yeu, M., and Jeong, J. (2022). Brain–computer interface to predict impulse buying behavior using functional near-infrared spectroscopy. Sci. Rep. 12:18024. doi: 10.1038/s41598-022-22653-8
Bazzani, A., Ravaioli, S., Trieste, L., Faraguna, U., and Turchetti, G. (2020). Is EEG suitable for marketing research? A systematic review. Front. Neurosci. 14:594566. doi: 10.3389/fnins.2020.594566
Cabañero, L., Hervás, R., González, I., Fontecha, J., Mondéjar, T., and Bravo, J. (2019). “Analysis of cognitive load using EEG when interacting with mobile devices,” in 13th international conference on ubiquitous computing and ambient intelligence UCAmI 2019, (Basel: MDPI), 70. doi: 10.3390/proceedings2019031070
Cheung, C. M. K., Chan, G. W. W., and Limayem, M. (2005). A critical review of online consumer behavior. J. Electron. Commer. Organ. 3, 1–19. doi: 10.4018/jeco.2005100101
Coan, J. A., B Allen, J. J., Harmon-Jones, E., and B Allen, J. J. (1997). Frontal EEG asymmetry and the behavioral activation and inhibition systems. Psychophysiology 40, 106–114.
D’Errico, F., Leone, G., Schmid, M., and D’Anna, C. (2020). Prosocial virtual reality, empathy, and EEG measures: A pilot study aimed at monitoring emotional processes in intergroup helping behaviors. Appl. Sci. 10:1196. doi: 10.3390/app10041196
Davidson, R. J. (2004). What does the prefrontal cortex “do” in affect: Perspectives on frontal EEG asymmetry research. Biol. Psychol. 67, 219–234. doi: 10.1016/j.biopsycho.2004.03.008
Dini, H., Simonetti, A., Bigne, E., and Bruni, L. E. (2022). EEG theta and N400 responses to congruent versus incongruent brand logos. Sci. Rep. 12:4490. doi: 10.1038/s41598-022-08363-1
Eichhorn, C., Lurz, M., Plecher, D. A., Weber, S., Wintergerst, M., Kaiser, B., et al. (2021). “Inspiring healthy food choices in a virtual reality supermarket by adding a tangible dimension in the form of an augmented virtuality smartphone,” in 2021 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops (VRW), (Lisbon: IEEE), 548–549. doi: 10.1109/VRW52623.2021.00156
Files, B. T., Lawhern, V. J., Ries, A. J., and Marathe, A. R. (2016). A permutation test for unbalanced paired comparisons of global field power. Brain Topogr. 29, 345–357. doi: 10.1007/s10548-016-0477-3
Frankish, K. (2010). Dual-process and dual-system theories of reasoning. Philos. Compass 5, 914–926. doi: 10.1111/j.1747-9991.2010.00330.x
Gable, P. A., Mechin, N. C., Hicks, J. A., and Adams, D. L. (2015). Supervisory control system and frontal asymmetry: Neurophysiological traits of emotion-based impulsivity. Soc. Cogn. Affect. Neurosci. 10, 1310–1315. doi: 10.1093/scan/nsv017
Golnar-Nik, P., Farashi, S., and Safari, M.-S. (2019). The application of EEG power for the prediction and interpretation of consumer decision-making: A neuromarketing study. Physiol. Behav. 207, 90–98. doi: 10.1016/j.physbeh.2019.04.025
Goto, N., Lim, X. L., Shee, D., Hatano, A., Khong, K. W., Buratto, L. G., et al. (2019). Can brain waves really tell if a product will be purchased? Inferring consumer preferences from single-item brain potentials. Front. Integr. Neurosci. 13:19. doi: 10.3389/fnint.2019.00019
Gramfort, A. (2013). MEG and EEG data analysis with MNE-Python. Front. Neurosci. 7:267. doi: 10.3389/fnins.2013.00267
Harmon-Jones, E., and Allen, J. J. B. (1998). Anger and frontal brain activity: EEG asymmetry consistent with approach motivation despite negative affective valence. J. Pers. Soc. Psychol. 74, 1310–1316. doi: 10.1037/0022-3514.74.5.1310
Harmon-Jones, E., Gable, P. A., and Price, T. F. (2012). The influence of affective states varying in motivational intensity on cognitive scope. Front. Integr. Neurosci. 6:73. doi: 10.3389/fnint.2012.00073
Horr, N. K., Han, K., Mousavi, B., and Tang, R. (2022). Neural signature of buying decisions in real-world online shopping scenarios – An exploratory electroencephalography study series. Front. Hum. Neurosci. 15:797064. doi: 10.3389/fnhum.2021.797064
Jung, T. P., Humphries, C., Lee, T. W., Makeig, S., McKeown, M. J., Iragui, V., et al. (1998). “Removing electroencephalographic artifacts: Comparison between ICA and PCA,” in Neural networks for signal processing - Proceedings of the IEEE workshop, (Cambridge: IEEE), 63–72. doi: 10.1109/nnsp.1998.710633
Karmarkar, U. R., and Plassmann, H. (2019). Consumer neuroscience: Past, present, and future. Organ. Res. Methods 22, 174–195. doi: 10.1177/1094428117730598
Keune, P. M., Wiedemann, E., Schneidt, A., and Schönenberg, M. (2015). Frontal brain asymmetry in adult attention-deficit/hyperactivity disorder (ADHD): Extending the motivational dysfunction hypothesis. Clin. Neurophysiol. 126, 711–720. doi: 10.1016/j.clinph.2014.07.008
Lee, M., Shin, G. H., and Lee, S. W. (2020). Frontal EEG asymmetry of emotion for the same auditory stimulus. IEEE Access 8, 107200–107213. doi: 10.1109/ACCESS.2020.3000788
Liao, W., Zhang, Y., and Peng, X. (2019). Neurophysiological effect of exposure to gossip on product endorsement and willingness-to-pay. Neuropsychologia 132:107123. doi: 10.1016/j.neuropsychologia.2019.107123
Lin, I. M., Chen, T. C., Lin, H. Y., Wang, S. Y., Sung, J. L., and Yen, C. W. (2021). Electroencephalogram patterns in patients comorbid with major depressive disorder and anxiety symptoms: Proposing a hypothesis based on hypercortical arousal and not frontal or parietal alpha asymmetry. J. Affect. Disord. 282, 945–952. doi: 10.1016/j.jad.2021.01.001
Makransky, G., Terkildsen, T. S., and Mayer, R. E. (2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learn. Instr. 60, 225–236. doi: 10.1016/j.learninstruc.2017.12.007
Martin, L. E., and Potts, G. F. (2009). Impulsivity in decision-making: An event-related potential investigation. Pers. Individ. Dif. 46, 303–308. doi: 10.1016/j.paid.2008.10.019
McClure, S. M., Li, J., Tomlin, D., Cypert, K. S., Montague, L. M., and Montague, P. R. (2004). Neural correlates of behavioral preference for culturally familiar drinks. Neuron 44, 379–387. doi: 10.1016/j.neuron.2004.09.019
Melendrez-Ruiz, J., Goisbault, I., Charrier, J.-C., Pagnat, K., Dujourdy, L., Arvisenet, G., et al. (2021). An exploratory study combining eye-tracking and virtual reality: Are pulses good “eye-catchers” in virtual supermarket shelves? Front. Virtual Real. 2:655273. doi: 10.3389/frvir.2021.655273
Neal, L. B., and Gable, P. A. (2019). Shifts in frontal asymmetry underlying impulsive and controlled decision-making. Biol. Psychol. 140, 28–34. doi: 10.1016/j.biopsycho.2018.11.002
Ohme, R., Reykowska, D., Wiener, D., and Choromanska, A. (2010). Application of frontal EEG asymmetry to advertising research. J. Econ. Psychol. 31, 785–793. doi: 10.1016/j.joep.2010.03.008
Ozkara, B. Y., and Bagozzi, R. (2021). The use of event related potentials brain methods in the study of Conscious and unconscious consumer decision making processes. J. Retail. Consum. Serv. 58:102202. doi: 10.1016/j.jretconser.2020.102202
Peschel, A. O., Jacobsen, L. F., Frank, D.-A., and Steinmann, S. (2022). “VR retail lab: An immersive virtual reality (VR) supermarket as a flexible research infrastructure,” in Adjunct proceedings of the 2022 Nordic human-computer interaction conference, (New York, NY: ACM), 1–2. doi: 10.1145/3547522.3547711
Petrantonakis, P. C., and Hadjileontiadis, L. J. (2011). A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition. IEEE Trans. Inform. Technol. Biomed. 15, 737–746. doi: 10.1109/TITB.2011.2157933
Pizzagalli, D. A., Sherwood, R. J., Henriques, J. B., and Davidson, R. J. (2005). Frontal brain asymmetry and reward responsiveness. Psychol. Sci. 16, 805–813. doi: 10.1111/j.1467-9280.2005.01618.x
Poole, B. D., and Gable, P. A. (2014). Affective motivational direction drives asymmetric frontal hemisphere activation. Exp. Brain Res. 232, 2121–2130. doi: 10.1007/s00221-014-3902-4
Ramsøy, T. Z., Skov, M., Christensen, M. K., and Stahlhut, C. (2018). Frontal brain asymmetry and willingness to pay. Front. Neurosci. 12:138. doi: 10.3389/fnins.2018.00138
Ravaja, N., Somervuori, O., and Salminen, M. (2013). Predicting purchase decision: The role of hemispheric asymmetry over the frontal cortex. J. Neurosci. Psychol. Econ. 6, 1–13. doi: 10.1037/a0029949
Rosenlacher, P., Tomcik, M., and Brùna, M. (2018). EEG study of the effect of virtual reality. AD ALTA J. Interdiscip. Res. 8, 216–218. doi: 10.33543/0802216218
Royo-Vela, M., and Varga, Á (2022). Unveiling neuromarketing and its research methodology. Encyclopedia 2, 729–751. doi: 10.3390/encyclopedia2020051
Rutherford, H. J. V., and Lindell, A. K. (2011). Thriving and surviving: Approach and avoidance motivation and lateralization. Emot. Rev. 3, 333–343. doi: 10.1177/1754073911402392
Sänger, J. (2019). Can’t take my eyes off you – How task irrelevant pictures of food influence attentional selection. Appetite 133, 313–323. doi: 10.1016/j.appet.2018.11.030
Schaefer, A., Buratto, L. G., Goto, N., and Brotherhood, E. V. (2016). The feedback-related negativity and the P300 brain potential are sensitive to price expectation violations in a virtual shopping task. PLoS One 11:e0163150. doi: 10.1371/journal.pone.0163150
Schall, J. D. (2001). Neural basis of deciding, choosing and acting. Nat. Rev. Neurosci. 2, 33–42. doi: 10.1038/35049054
Shang, Q., Jin, J., Pei, G., Wang, C., Wang, X., and Qiu, J. (2020). Low-order webpage layout in online shopping facilitates purchase decisions: Evidence from event-related potentials. Psychol. Res. Behav. Manag. 13, 29–39. doi: 10.2147/PRBM.S238581
Shen, B., Tan, W., Guo, J., Zhao, L., and Qin, P. (2021). How to promote user purchase in metaverse? A systematic literature review on consumer behavior research and virtual commerce application design. Appl. Sci. 11:11087. doi: 10.3390/app112311087
Shih, D. H., Lu, K. C., and Shih, P. Y. (2019). Exploring shopper’s browsing behavior and attention level with an EEG biosensor cap. Brain Sci. 9:301. doi: 10.3390/brainsci9110301
Shravani, D., Prajwal, Y. R., Atreyas, P. V., and Shobha, G. (2021). “VR supermarket: A virtual reality online shopping platform with a dynamic recommendation system,” in 2021 IEEE International conference on artificial intelligence and virtual reality (AIVR), (Taichung: IEEE), 119–123. doi: 10.1109/AIVR52153.2021.00028
Smith, E. E., Reznik, S. J., Stewart, J. L., and Allen, J. J. B. (2017). Assessing and conceptualizing frontal EEG asymmetry: An updated primer on recording, processing, analyzing, and interpreting frontal alpha asymmetry. Int. J. Psychophysiol. 111, 98–114. doi: 10.1016/j.ijpsycho.2016.11.005
Spironelli, C., Fusina, F., Bortolomasi, M., and Angrilli, A. (2021). EEG frontal asymmetry in dysthymia, major depressive disorder and euthymic bipolar disorder. Symmetry 13:2414. doi: 10.3390/sym13122414
Tang, X., and Song, Z. (2019). Neurological effects of product price and evaluation on online purchases based on event-related potentials. Neurosci. Lett. 704, 176–180. doi: 10.1016/j.neulet.2019.04.019
Tarrant, J., and Cope, H. (2018). Combining frontal gamma asymmetry neurofeedback with virtual reality: A proof-of-concept case study. NeuroRegulation 5, 57–66. doi: 10.15540/nr.5.2.57
Thomsen, L. A., and Adjorlu, A. (2021). “A collaborative virtual reality supermarket training application to teach shopping skills to young individuals with autism spectrum disorder,” in 2021 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops (VRW), (Lisbon: IEEE), 50–55. doi: 10.1109/VRW52623.2021.00015
Trammell, J. P., MacRae, P. G., Davis, G., Bergstedt, D., and Anderson, A. E. (2017). The relationship of cognitive performance and the theta-alpha power ratio is age-dependent: An EEG study of short term memory and reasoning during task and resting-state in healthy young and old adults. Front. Aging Neurosci. 9:364. doi: 10.3389/fnagi.2017.00364
Tran, Y., Austin, P., Lo, C., Craig, A., Middleton, J. W., Wrigley, P. J., et al. (2022). An exploratory EEG analysis on the effects of virtual reality in people with neuropathic pain following spinal cord injury. Sensors 22:2629. doi: 10.3390/s22072629
Wang, J., Wang, A., Zhu, L., and Wang, H. (2021). The effect of product image dynamism on purchase intention for online aquatic product shopping: An EEG Study. Psychol. Res. Behav. Manag. 14, 759–768. doi: 10.2147/PRBM.S313742
Wang, Y., Wang, Q., Du, J., and Lin, Y. (2020). “Quantifying cognitive load in wayfinding information review using EEG,” in Construction research congress 2020, (Reston, VA: American Society of Civil Engineers), 594–602. doi: 10.1061/9780784482865.063
Wixted, J. T. (ed.) (2018). Stevens’ handbook of experimental psychology and cognitive neuroscience. Hoboken, NJ: Wiley, doi: 10.1002/9781119170174
Yang, M., Deng, X., and An, S. (2021). The relationship between habitual use and real-time emotion regulation strategies in adolescents: Evidence from frontal EEG asymmetry. Neuropsychologia 162:108056. doi: 10.1016/j.neuropsychologia.2021.108056
Yuan, G., He, W., and Liu, G. (2022). Is mate preference recognizable based on electroencephalogram signals? Machine learning applied to initial romantic attraction. Front. Neurosci. 16:830820. doi: 10.3389/fnins.2022.830820
Keywords: electroencephalogram, virtual reality, frontal asymmetry, consumer neuroscience, decision making
Citation: Saffari F, Kakaria S, Bigné E, Bruni LE, Zarei S and Ramsøy TZ (2023) Motivation in the metaverse: A dual-process approach to consumer choices in a virtual reality supermarket. Front. Neurosci. 17:1062980. doi: 10.3389/fnins.2023.1062980
Received: 06 October 2022; Accepted: 30 January 2023;
Published: 16 February 2023.
Edited by:
Giulia Cartocci, Sapienza University of Rome, ItalyReviewed by:
Luigi Pavone, Mediterranean Neurological Institute Neuromed (IRCCS), ItalyKostas Karpouzis, Panteion University, Greece
Copyright © 2023 Saffari, Kakaria, Bigné, Bruni, Zarei and Ramsøy. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Thomas Z. Ramsøy, 
dGhvbWFzQG5ldXJvbnNpbmMuY29t