Guanxiong Pei
Taihao Li*- Zhejiang Laboratory, Research Center for Advanced AI Theory, Hangzhou, China
Affect plays an important role in the consumer decision-making process and there is growing interest in the development of new technologies and computational approaches that can interpret and recognize the affects of consumers, with benefits for marketing described in relation to both academia and industry. From an interdisciplinary perspective, this paper aims to review past studies focused on electroencephalography (EEG)-based affective computing (AC) in marketing, which provides a promising avenue for studying the mechanisms underlying affective states and developing recognition computational models to predict the psychological responses of customers. This review offers an introduction to EEG technology and an overview of EEG-based AC; provides a snapshot of the current state of the literature. It briefly presents the themes, challenges, and trends in studies of affect evaluation, recognition, and classification; and further proposes potential guidelines for researchers and marketers.
Introduction
Affective computing (AC) is a continuously growing interdisciplinary research field spanning psychology, computer science, cognitive science, neuroscience, and more (Tao and Tan, 2005). In Picard’s landmark book, AC is defined as “computing that relates to, arises from, or deliberately influences emotion or other affective phenomena” (Picard, 1997). It mainly focuses on how technology can inform and deepen understanding of human affect and how systems can be designed to estimate the affective state using computational models from behavioral and physiological signals (Calvo and D’Mello, 2010).
Affect plays an important role in human cognition, specifically in perception processes, rational decision-making, human communication, and human intelligence (Ammar et al., 2010; Singh et al., 2013). In marketing, understanding and recognizing the affective states of users (or customers) has become a vital theme, which can reveal users’ true preferences and improve and assist in the purchasing process (Malär et al., 2011; Garrido-Morgado et al., 2015).
Conventional assessment methods of AC in marketing depend mostly on personal evaluations, such as surveys, focus groups, and interviews (Ariely and Berns, 2010; Yadava et al., 2017; Lin et al., 2018). However, customers may not express their true opinions because of social desirability bias (Paulhus, 2002; Vecchiato et al., 2011a). They may not say exactly how they are feeling but rather, how they feel others would reply (Calvert and Brammer, 2012). These post hoc analysis tools are also influenced by individuals’ mental states or environments at the time of self-reporting (Nilashi et al., 2020). Due to the limitations of traditional AC techniques, marketers and researchers try to supplement these shortcomings and seek alternative or complementary tools. Neuroscientific tools based on electrophysiological and neuroimaging techniques provide one such alternative, as well as a way to dig deeper into understanding the complex evaluation process and the dynamics of the affective state by directly accessing the physiological signals and fundamental cerebral structure from which an affective state occurs.
Considering its low cost and high temporal resolution (milliseconds), electroencephalography (EEG) has become common and is extensively used in the marketing industry. Furthermore, variations in EEG signals cannot be voluntarily controlled and therefore are a better objective indicator of affect (Singh et al., 2013). EEG-based AC provides a promising avenue for studying the mechanisms underlying affective states and developing recognition computational models to predict the psychological responses of customers. It can, therefore, be widely used to boost sales, advertising, pricing, package design, marketing campaigns, and so forth (Calvo and D’Mello, 2010).
In this review, we focus on EEG-based AC in marketing. First, we stress the need to incorporate the various features of neural signals that contribute to consumers’ affective states and evaluation processes beyond what traditional marketing measures already provide. Second, we state that marketing studies should adopt the methodologies and algorithms used in data processing and prediction modeling that are mature in other fields such as computer science and engineering. We examine AC literature in marketing on the general features extracted from EEG recordings and conclude with a general discussion of the challenges faced by this field, providing several recommended guidelines for the road ahead.
Consumers’ Neural Affective Mechanisms
It is expected that a deeper understanding of how the human brain works and responds will make marketing more effective (Kuan et al., 2014; Nagyová et al., 2014; Caratù et al., 2018). Researchers and marketers have attempted to reveal the neural correlates and neural affective mechanisms that affect decision making in customers and consumer behavior in economic processes.
Event-Related Potentials
As an extremely useful tool for investigating higher-level brain functions, event-related potentials (ERPs) have been widely used to explore the neural mechanisms underlying affect processing in marketing. For example, Ma et al. (2007) found that brand extension preferences were modulated by negative emotions. These underlying mechanisms were indicated by the amplitudes of N270. This research presents a method, which is automatic, objective, and non-verbal, for the use of ERP components as markers for emotional preferences. Jones et al. (2012) shifted their attention toward pricing and discount-related consumer behavior. The enhanced P3 in high math anxiety individuals indicated greater reliance on the emotional and motivational factors involved in the buying process. Social interaction in marketing is another important theme. In a study by Pozharliev et al. (2015), the late positive potential (LPP) was enlarged during shopping for luxury products. It was especially prominent in the presence of another person during buying decisions, suggesting that social interaction magnifies the emotional effect of the brand category. Furthermore, Chen et al. (2010) found that N500 was evoked by emotional conflicts when making counter conformity choices in buying books online.
By taking full advantage of the high temporal resolution of ERPs, a group of scientists conducted multistage experiments and multicomponent detections to decompose the time course of neural cognitive and affective activities. Handy et al. (2010) focused on the emotional appraisal of unfamiliar commercial logos. Using an ERP examination, they found that the judgment of logos can be divided into at least two stages: an initial formation of ones’ impression at the sensory-perceptual level and evaluative analysis at the cognitive-processing level. Jin et al. (2018) demonstrated that eco-labeling induced positive emotions in consumers and reduced cognitive conflict, as reflected by decreased P2 and N2 amplitudes, respectively. In another study, Shang et al. (2020) studied online shopping and focused on the influence of webpage layouts on consumer experience. They demonstrated that a low-order online shopping webpage facilitated consumers’ instant purchase decisions as indexed by increased P2 (attention engagement) and LPP (emotional self-control) amplitudes. Separating affective activities from cognitive activities and capturing independent processes within each stage is helpful to provide a deeper and more nuanced understanding of consumers’ neural affective mechanisms.
EEG Time-Frequency Components
Because EEG signals comprise abundant time-frequency information, multiple characteristic parameters, such as hemispheric asymmetry, event-related desynchronization/synchronization (ERD/ERS), and power spectral density (PSD), are closely related to affective states.
Concerned with the frontal EEG asymmetry, Vecchiato et al. (2010) revealed that the cerebral activity of the theta band on the left frontal hemisphere increased when television commercials were judged to be pleasant. As a follow-up to this, Vecchiato et al. (2014) further highlighted that the index of cortical hemispheric asymmetry, which was also a valid predictor of preferences, was significantly correlated with participants’ pleasantness ratings. Furthermore, the frontal alpha band, as reflected in cerebral hemispheric alpha asymmetry, has been widely used to represent affective responses to advertisements (Ohme et al., 2010; Venkatraman et al., 2015). When two subjects were viewing two versions of the almost same skincare product advertisement (differing only by the inclusion of a very short clip concerning a gesture by a female model), Ohme et al. (2009) found out that each version elicited significantly different emotional impacts, indexed by differences in the frontal alpha activity between the left and the right hemispheres. Similarly, Reeves et al. (1989) reported a significant interaction between hemisphere activity and the emotional content (positive and negative) of television advertisements for frontal alpha power but no interaction for occipital alpha power. It would be more beneficial for future research to specify the role of mediation and moderation that frontal alpha asymmetry plays in relation to advertisements and affective states (Palmiero and Piccardi, 2017; Allen et al., 2018).
With regard to the ERD/ERS indices, Vecchiato et al. (2011b) demonstrated that desynchronization of the left alpha band is positively related to judgments of high pleasantness. Other findings also supported the evidence that synchronization and desynchronization changes were the result of affective arousal and valence (Aftanas et al., 2001; Costa et al., 2006). Moreover, ERD/ERS is a useful tool for understanding variation in oscillatory activity occurring during the decision-making process related to subjective preferences. Khushaba et al. (2013) explored brain activity while subjects were choosing crackers using EEG and eye tracking data. They found clear phase synchronization between the left and right frontal and occipital regions during preference decision making for the different cracker characteristics, which can be used to predict consumers’ future choices and develop effective marketing strategies.
By performing power spectra and PSD analyses, Horska et al. (2016) and Berčík et al. (2016a) studied the emotions of consumers tasting different kinds of wine and used their findings to improve the practical selling strategies. Subsequently, Berčík et al. (2016b) conducted interdisciplinary research on the impact of illumination on the emotional state (valence) of customers in a food store by calculating and comparing alpha and beta spectral power. Lee (2016) explored the emotional mechanism of empathy indexed by the theta-band power spectra of the anterior cingulate cortex (ACC) for customer equity and willingness to pay. Moreover, Vecchiato et al. (2011b) showed that an increase in PSD in the left frontal lobe is negatively related to the degree of perceived pleasantness. These studies indicate that power spectra and PSD are reliable indicators in identifying emotional responses to marketing.
Taken together, the studies discussed above provide evidence that it is helpful to use EEG features and properties to understand neural affective mechanisms, which enables us to objectively reveal hidden consumer reactions and identify critical success factors in the process of buying and consuming certain products (Kuan et al., 2014; Telpaz et al., 2015). We believe that brain wave-based methodologies would further enrich marketing research and help marketers to go beyond traditional marketing paradigms.
Classification and Recognition of Affective States
Feature extraction is a critical step for obtaining an accurate classification result. It is beneficial for simplifying the models and the amount of resources, reducing the cost of data processing, improving data visualization, and avoiding overfitting (Übeyli, 2008; Al-Fahoum et al., 2014). Various methods have been used to extract the neural features from EEG signals. Among these methods are time frequency distributions (appropriate for great continuous segments), fast Fourier transform (suitable for narrowband signal and stationary signal), wavelet transform (good tool for sudden and transient signal changes), auto regressive methods (which are advantageous for short data segments), and so on (Tao and Tan, 2005; Singh et al., 2013; Al-Fahoum et al., 2014; Yadava et al., 2017). Researchers should make clear the signal types and apply the optimum method.
Once the features are extracted, the features will be fed into an affect classification algorithm. Many types of classifiers are commonly used including: support vector machines (SVMs), relevance vector machines (RVMs), logistic model trees (LMTs), Fisher linear discriminant analysis (FLDA), the k-nearest neighbors (KNN), hidden Markov models (HMMs), artificial neural networks (ANNs), artificial bee colonies (ABCs), random forests (RFs), and deep neural networks (DNNs). In the following paragraphs, we present the various types of affective classifiers used in marketing scenarios.
Many studies have used machine learning algorithms to assess the impact of advertising. Friedman et al. (2015) proposed an EEG data-driven approach to measure customers’ emotional valence when processing commercials. Their results indicated that hemispheric asymmetry was a good marker and the LMT algorithm (81.2%) provided better classification rates than the SVM algorithm (77.3%). Wei et al. (2018) explored a new method using low-cost EEG headbands to assess the influence of advertisements on purchasing. The EEG features that were closely related to emotions were gathered into different groups and the SVM method was applied to assess the ability of features to predict possible purchases, achieving an accuracy of 77.3%. Yang et al. (2015) developed an approach to evaluate the temporal patterns of EEG data (PSD) and extract affective indices such as happiness and surprise for TV commercial evaluation. FLDA was used to predict which parts of an advertisement could elicit positive emotional responses in customers. Similarly, Guixeres et al. (2017) attempted to forecast the effectiveness of advertisements during the Super Bowl sports event in the United States, based on EEG signals, including biometric responses such as the z-score of the global field power (GFP), hemispheric asymmetry (pleasantness index), and the relative number of peaks in the beta and theta bands (interest index). The results showed that the ANN was able to precisely classify and estimate the effect of each advertisement on the Internet via biometrics (82.9% of average accuracy). Marketers could consider the proposed approach at the technical design stages of advertising content. From a multimodal perspective, Gauba et al. (2017) developed a notable rating forecast framework for advertisement clips by using both EEG signals and sentiment analysis of online users’ comments from YouTube. The prediction was carried out using the RF regression and was later fused with the sentiment score to improve the overall prediction.
Some scholars pay special attention to the music or jingles of advertisements. Lin et al. (2014) focused on the emotion classification problem when listening to music and used the SVM classifier, generating accuracies of 82.5 and 79.1% for valence and arousal classification, respectively. Gupta and Falk (2016) used EEG graph-theoretic features to classify emotional states while watching music clips with the aid of two classifiers, the SVM and RVM, and the approach achieved a significant improvement in the classification accuracy (The percentage increase in classification performance ranged from 3 to 9%). More recently, Avinash et al. (2018) developed a very accurate tool for understanding consumers’ emotional responses to advertisement jingles. The KNN algorithm was used to classify positive and negative emotions based on theta power signals, achieving an accuracy of 100%.
Other studies have investigated consumers’ purchasing preferences. Chew et al. (2016) studied the aesthetic emotional responses in industrial design and buying decisions using EEG signals for virtual 3D shapes with motion. A classification accuracy of up to 80% was attained using the KNN with the alpha, theta, and delta rhythms as the features taken from frontal electrodes to classify two classes, like and dislike. Lobato and Garza (2017) developed a classification algorithm using neural networks and EEG signals to measure the affective states of “do like” and “don’t like” during buying processes. Yadava et al. (2017) proposed a predictive model using the HMM classifier to analyze EEG signals to understand consumers’ affective states and purchase preferences toward e-commerce goods, and the model achieved an accuracy of 70.33%. They also tested numerous other models, such as the SVM, RF, and KNN. Interestingly, Kumar et al. (2019) creatively put forward a multimodal rating prediction method by fusing the affective ratings from e-commerce websites and EEG data. The ABC and RF models were applied to optimize and compute the ratings from varied data sources, and the results showed that the combined method could achieve a lower Root Mean Square Error (RMSE) in rating prediction compared to a unimodal method. More recently, Aldayel et al. (2020) adopted a deep learning approach to assess consumer preferences (pleasant or unpleasant) by extracting the PSD and valence features. They built four different classifiers, namely, the DNN, RF, SVM, and KNN, which attained accuracies of 94, 92, 62, and 88%, respectively.
In summary, due to significant differences between the experimental design and the paradigm used in these studies, it is difficult to make direct comparisons of the classification accuracies achieved. Furthermore, it seems that there is no particular feature extractor or affective classifier that appears to be the single best choice for all marketing scenarios. In most situations, one should consider as many algorithms as possible from the studies mentioned above and then compare the results with a range of features and algorithms before choosing the one with the best performance for the given marketing application.
Discussion
In this review, we summarize previous studies analyzing EEG signals as biological markers in affective mechanism and recognition in the marketing area (as shown in Table 1). The majority of the studies, especially those using machine learning techniques and algorithms, have been published in the last 10 years. This review provides new directions regarding neuromarketing data analyses and fosters cooperation among scholars from miscellaneous disciplines, such as information science, neuroscience, marketing, and psychology. Although there has been a recent increase in the number of EEG-based AC studies in marketing with no signs of slowing down, theoretical and operational challenges must be settled before moving forward.
Table 1. Summary of current findings on EEG-based affective computing in marketing.
First, it would be helpful to pay more attention to multiclass affective classification. As this review shows, most of the previous studies are based on dimensional emotion theory, typically concerning the dimensions of arousal, valence, liking, and dominance. The state of the art usually relies on the affective polarity of its components (e.g., positive or negative) and proposes approaches that mostly focus on binary affective classification. However, to study the affective states of consumers, it would be more interesting to go deeper into the classification and detect subtle affective changes in marketing. Furthermore, marketing scenarios may induce multiple emotions in customers. The phenomena of coexistence should be considered in affective tagging. We recommend that future studies should focus on two issues to develop a more accurate affective definition and conduct better forecasting: (1) they should aim for a deeper understanding of consumer ambivalence, characterized by the co-occurrence of positive and negative emotions (Kreibig and Gross, 2017; Hu et al., 2019); and (2) consider emotion dyads, namely, a mix of primary emotions, raised by Plutchik (1980).
The multidimensional and multimodal feature fusion can obtain better recognition performance. When studying EEG-affect relationships, EEG-based AC studies assume that EEG signals can sufficiently depict and predict human affective states. However, this hypothesis cannot always be assumed to be true because the relationship between physiological responses and psychological states could be very complex (Cacioppo and Tassinary, 1990; Hu et al., 2019). To achieve precise prediction and improved generalization, first, we suggest decreasing the abundant number of features from EEG signals and further perform feature selection and fusion. The most widely used features include differential symmetry, GFP, PSD, and ERPs. It might be the case that a fusion of features derived from different EEG signal types will lead to better recognition performance (Hakim and Levy, 2019). It is worth noting that future studies should be more cautious regarding the reliability and validity of “one-to-one” relationships (one affective state is associated with one and only one EEG feature) (Bridwell et al., 2018; Hu et al., 2019). Second, recent studies have revealed that multimodal frameworks can effectively increase emotion recognition accuracy and robustness compared to unimodal frameworks (Guixeres et al., 2017; Avinash et al., 2018; Kumar et al., 2019). The advantage of multiple modalities (for example, vision, sound, or smell) helps to increase the validity and usability since the weaknesses of one modality are offset by the strengths of another. Future studies may derive features from modalities other than EEG while collecting and analyzing data by using machine learning, natural language processing, and automatic speech recognition technology, evolving from unimodal analyses to multimodal fusion.
The use of portable wireless EEG devices and virtual reality (VR) technology can alleviate the lack of ecological validity in marketing studies. For EEG hardware devices, the whole-brain coverage, the time-consuming preparation procedure, and the prohibitive cost of a professional headset with wet electrodes make it impractical and difficult to transfer the laboratory to real-world applications in marketing. A group of recent studies has confirmed the feasibility of using consumer-level EEG headsets for AC with promising results. For example, the widely used wireless EPOC headset (e.g., Kuan et al., 2014; Lin et al., 2014; Friedman et al., 2015; Yang et al., 2015; Gauba et al., 2017; Yadava et al., 2017; Kumar et al., 2019), due to its light weight, low price, and ease of use, shows promise. Studies on the EPOC headset seem to agree that it can be applied to acquire reliable EEG signals in marketing, but researchers should pay attention to its relatively low signal-to-noise ratio and poor signal stability (Friedman et al., 2015). We suggest that researchers evaluate the performance of consumer-level devices using the standard testing procedures proposed by Hu et al. (2019). In addition, to bridge the gap between the laboratory environment and real market scenarios, the use of VR is an important trend that can effectively enhance the experience of immersive sensation. It enables consumers to get a direct, intuitive, and concrete understanding of the appearance, quality, and performance of products (Guo and Elgendi, 2013). Furthermore, VR makes it possible to simulate and assess retail and consumption environments under controlled laboratory conditions (Marín-Morales et al., 2017), allowing the isolation and modification of variables in a cost-effective manner.
Studying interactions among multiple customers is critical for understanding the marketing ecosystem, which consists of interrelated trends that shape consumer behaviors. Most AC studies in marketing have concentrated mainly on a single consumer’s EEG activity and may ignore the socio-affective interaction and processes related to consumer behavior (Hasson et al., 2012). The EEG-based hyperscanning technique [for a recent review, see Liu et al. (2018)] provides a way to explore dynamic brain activities between two or more interactive customers and their underlying neural affective mechanisms. In previous hyperscanning studies, interpersonal neural synchronization (INS) has been verified to be a crucial neural marker for different kinds of social interactions, such as communication (Stephens et al., 2010), collaborative decision making (Montague et al., 2002; Hu et al., 2018), and imitation (Pan et al., 2017). As consumer behavior is inherently social and interactive in nature, EEG-based INS could be used to study the biological mechanism for shared intentionality of consumption, panic buying, collective emotion, and group purchase.
Author Contributions
Both authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
Funding
This work was supported by Grant No. 2020M671819 from the China Postdoctoral Science Foundation. This work was also supported by Grant No. 111002-AC2001 from the Major Scientific Project of Zhejiang Lab.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The handling editor declared a shared affiliation with TL and GP at the time of review.
References
Aftanas, L., Varlamov, A., Pavlov, S., Makhnev, V., and Reva, N. (2001). Event-related synchronization and desynchronization during affective processing: emergence of valence-related time-dependent hemispheric asymmetries in theta and upper alpha band. Int. J. Neurosci. 110, 197–219. doi: 10.3109/00207450108986547
Aldayel, M., Ykhlef, M., and Al-Nafjan, A. (2020). Deep learning for EEG-based preference classification in neuromarketing. Appl. Sci. 10:1525. doi: 10.3390/app10041525
Al-Fahoum, A. S., Al-Fraihat, A. A., Oliveira, M. S., Grant, A., and Hinojosa, J. A. (2014). Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains. ISRN Neurosci. 2014:730218. doi: 10.1155/2014/730218
Allen, J. J., Keune, P. M., Schönenberg, M., and Nusslock, R. (2018). Frontal EEG alpha asymmetry and emotion: from neural underpinnings and methodological considerations to psychopathology and social cognition. Psychophysiology 55:e13028. doi: 10.1111/psyp.13028
Ammar, M. B., Neji, M., Alimi, A. M., and Gouardères, G. (2010). The affective tutoring system. Expert Syst. Appl. 37, 3013–3023. doi: 10.1016/j.eswa.2009.09.031
Ariely, D., and Berns, G. S. (2010). Neuromarketing: the hope and hype of neuroimaging in business. Nat. Rev. Neurosci. 11, 284–292. doi: 10.1038/nrn2795
Avinash, T., Dikshant, L., and Seema, S. (2018). Methods of neuromarketing and implication of the frontal theta asymmetry induced due to musical stimulus as choice modeling. Procedia Comput. Sci. 132, 55–67. doi: 10.1016/j.procs.2018.05.059
Berčík, J., Horská, E., Gálová, J., and Margianti, E. S. (2016a). Consumer neuroscience in practice: the impact of store atmosphere on consumer behavior. Periodica Polytechnica Soc.Manag. Sci. 24, 96–101. doi: 10.3311/PPso.8715
Berčík, J., Horská, E., Wang, R. W., and Chen, Y. (2016b). The impact of parameters of store illumination on food shopper response. Appetite 106, 101–109. doi: 10.1016/j.appet.2016.04.010
Bridwell, D. A., Cavanagh, J. F., Collins, A. G., Nunez, M. D., Srinivasan, R., Stober, S., et al. (2018). Moving beyond ERP components: a selective review of approaches to integrate EEG and behavior. Front. Hum. Neurosci. 12:106. doi: 10.3389/fnhum.2018.00106
Cacioppo, J. T., and Tassinary, L. G. (1990). Inferring psychological significance from physiological signals. Am. Psychol. 45, 16–28. doi: 10.1037/0003-066X.45.1.16
Calvert, G. A., and Brammer, M. J. (2012). Predicting consumer behavior: using novel mind-reading approaches. IEEE Pulse 3, 38–41. doi: 10.1109/MPUL.2012.2189167
Calvo, R. A., and D’Mello, S. (2010). Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans. Affect. Comput. 1, 18–37. doi: 10.1109/T-AFFC.2010.1
Caratù, M., Cherubino, P., and Mattiacci, A. (2018). “Application of neuro-marketing techniques to the wine tasting experience,” in Proceedings of the 11th Annual Conference of the EuroMed Academy of Business Valleta, Malta.
Chen, M., Ma, Q., Li, M., Lai, H., Wang, X., and Shu, L. (2010). Cognitive and emotional conflicts of counter-conformity choice in purchasing books online: an event-related potentials study. Biol. Psychol. 85, 437–445. doi: 10.1016/j.biopsycho.2010.09.006
Chew, L. H., Teo, J., and Mountstephens, J. (2016). Aesthetic preference recognition of 3D shapes using EEG. Cogn. Neurodyn. 10, 165–173. doi: 10.1007/s11571-015-9363-z
Costa, T., Rognoni, E., and Galati, D. (2006). EEG phase synchronization during emotional response to positive and negative film stimuli. Neurosci. Lett. 406, 159–164. doi: 10.1016/j.neulet.2006.06.039
Friedman, D., Shapira, S., Jacobson, L., and Gruberger, M. (2015). “A data-driven validation of frontal EEG asymmetry using a consumer device,” in Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction, Xi’an, China (New York, NY: IEEE)
Garrido-Morgado, A., González-Benito, O., Campo, K., and Martos-Partal, M. (2015). “Improving sales of private labels in store,” in Proceedings of the Advances in National Brand and Private Label Marketing. Springer Proceedings in Business and Economics, eds F. Martínez-López, J. Gázquez-Abad, and R. Sethuraman (Cham: Springer), 3–7. doi: 10.1007/978-3-319-20182-5_1
Gauba, H., Kumar, P., Roy, P. P., Singh, P., Dogra, D. P., and Raman, B. (2017). Prediction of advertisement preference by fusing EEG response and sentiment analysis. Neural Netw. 92, 77–88. doi: 10.1016/j.neunet.2017.01.013
Guixeres, J., Bigné, E., AusínAzofra, J. M., Alcañiz Raya, M., ColomerGranero, A., Fuentes Hurtado, F., et al. (2017). Consumer neuroscience-based metrics predict recall, liking and viewing rates in online advertising. Front. Psychol. 8:1808. doi: 10.3389/fpsyg.2017.01808
Guo, G., and Elgendi, M. (2013). A new recommender system for 3D e-commerce: an EEG based approach. J. Adv. Manag. Sci. 1, 61–65. doi: 10.12720/joams.1.1.61-65
Gupta, R., and Falk, T. H. (2016). Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization. Neurocomputing 174, 875–884. doi: 10.1016/j.neucom.2015.09.085
Hakim, A., and Levy, D. J. (2019). A gateway to consumers’ minds: achievements, caveats, and prospects of electroencephalography-based prediction in neuromarketing. Wiley Interdiscip. Rev. Cogn. Sci. 10:e1485. doi: 10.1002/wcs.1485
Handy, T. C., Smilek, D., Geiger, L., Liu, C., and Schooler, J. W. (2010). ERP evidence for rapid hedonic evaluation of logos. J. Cogn. Neurosci. 22, 124–138. doi: 10.1162/jocn.2008.21180
Hasson, U., Ghazanfar, A. A., Galantucci, B., Garrod, S., and Keysers, C. (2012). Brain-to-brain coupling: a mechanism for creating and sharing a social world. Trends Cogn. Sci. 16, 114–121. doi: 10.1016/j.tics.2011.12.007
Horska, E., Bercik, J., Krasnodebski, A., Matysik-Pejas, R., and Bakayova, H. (2016). Innovative approaches to examining consumer preferences when choosing wines. Agric. Econ. Blackwell 62, 124–133. doi: 10.17221/290/2015-AGRICECON
Hu, X., Chen, J., Wang, F., and Zhang, D. (2019). Ten challenges for EEG-based affective computing. Brain Sci. Adv. 5, 1–20. doi: 10.1177/2096595819896200
Hu, Y., Pan, Y., Shi, X., Cai, Q., Li, X., and Cheng, X. (2018). Inter-brain synchrony and cooperation context in interactive decision making. Biol. Psychol. 133, 54–62. doi: 10.1016/j.biopsycho.2017.12.005
Jin, J., Dou, X., Meng, L., and Yu, H. (2018). Environmental-friendly eco-labeling matters: evidences from an ERPs Study. Front. Hum. Neurosci. 12:417. doi: 10.3389/fnhum.2018.00417
Jones, W. J., Childers, T. L., and Jiang, Y. (2012). The shopping brain: Math anxiety modulates brain responses to buying decisions. Biol. Psychol. 89, 201–213. doi: 10.1016/j.biopsycho.2011.10.011
Khushaba, R. N., Wise, C., Kodagoda, S., Louviere, J., Kahn, B. E., and Townsend, C. (2013). Consumer neuroscience: assessing the brain response to marketing stimuli using electroencephalogram (EEG) and eye tracking. Expert Syst. Appl. 40, 3803–3812. doi: 10.1016/j.eswa.2012.12.095
Kreibig, S. D., and Gross, J. J. (2017). Understanding mixed emotions: paradigms and measures. Curr. Opin. Behav. Sci. 15, 62–71. doi: 10.1016/j.cobeha.2017.05.016
Kuan, K. K., Zhong, Y., and Chau, P. Y. (2014). Informational and normative social influence in group-buying: evidence from self-reported and EEG data. J. Manage. Inform. Syst. 30, 151–178. doi: 10.2753/mis0742-1222300406
Kumar, S., Yadava, M., and Roy, P. P. (2019). Fusion of EEG response and sentiment analysis of products review to predict customer satisfaction. Inform. Fusion 52, 41–52. doi: 10.1016/j.inffus.2018.11.001
Lee, E. (2016). Empathy can increase customer equity related to pro-social brands. J. Bus. Res. 69, 3748–3754. doi: 10.1016/J.JBUSRES.2015.05.018
Lin, M. J., Cross, S. N., Jones, W. J., and Childers, T. L. (2018). Applying EEG in consumer neuroscience. Eur. J. Marketing 52, 66–91. doi: 10.1108/EJM-12-2016-0805
Lin, Y., Yang, Y., and Jung, T. (2014). Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening. Front. Neurosci. 8, 94–94. doi: 10.3389/FNINS.2014.00094
Liu, D., Liu, S., Liu, X., Zhang, C., Li, A., Jin, C., et al. (2018). Interactive brain activity: review and progress on EEG-based hyperscanning in social interactions. Front. Psychol. 9:1862. doi: 10.3389/fpsyg.2018.01862
Lobato, B. Y. M., and Garza, A. A. (2017). Classification algorithm for measuring human emotion: “I like it” and “I do not like” in neuromarketing. IEEE Lat. Am. Trans. 15, 2177–2184. doi: 10.1109/TLA.2017.8070424
Ma, Q., Wang, X., Dai, S., and Shu, L. (2007). Event-related potential N270 correlates of brand extension. Neuroreport 18, 1031–1034. doi: 10.1097/WNR.0b013e3281667d59
Malär, L., Krohmer, H., Hoyer, W. D., and Nyffenegger, B. (2011). Emotional brand attachment and brand personality: the relative importance of the actual and the ideal self. J. Marketing 75, 35–52. doi: 10.1509/jmkg.75.4.35
Marín-Morales, J., Torrecilla-Moreno, C., Guixeres Provinciale, J., and Llinares Millán, M. D. C. (2017). Methodological bases for a new platform for the measurement of human behaviour in virtual environments. DYNA Ingeniería e Industria 92, 34–38. doi: 10.6036/7963
Montague, P. R., Berns, G. S., Cohen, J. D., McClure, S. M., Pagnoni, G., Dhamala, M., et al. (2002). Hyperscanning: simultaneous fmri during linked social interactions. Neuroimage 16, 1159–1164. doi: 10.1006/nimg.2002.1150
Nagyová, L., Berčík, J., and Horská, E. (2014). The efficiency, energy intensity and visual impact of the accent lighting in the retail grocery stores. Potravinarstvo Slovak J. Food Sci. 8, 296–305. doi: 10.5219/398
Nilashi, M., Samad, S., Ahmadi, N., Ahani, A., Abumalloh, R. A., Asadi, S., et al. (2020). Neuromarketing: a review of research and implications for marketing. J. Soft Comput. Deci. Support Syst. 7, 23–31.
Ohme, R., Reykowska, D., Wiener, D., and Choromanska, A. (2009). Analysis of neurophysiological reactions to advertising stimuli by means of EEG and galvanic skin response measures. J. Neurosci. Psychol. Econ. 2, 21. doi: 10.1037/a0015462
Ohme, R., Reykowska, D., Wiener, D., and Choromanska, A. (2010). Application of frontal EEG asymmetry to advertising research. J. Econ. Psychol. 31, 785–793. doi: 10.1016/j.joep.2010.03.008
Palmiero, M., and Piccardi, L. (2017). Frontal EEG asymmetry of mood: a mini-review. Front. Behav. Neurosci. 11:224. doi: 10.3389/fnbeh.2017.00224
Pan, Y., Cheng, X., Zhang, Z., Li, X., and Hu, Y. (2017). Cooperation in lovers: an fnirs-based hyperscanning study. Hum. Brain Mapp. 38, 831–841. doi: 10.1002/hbm.23421
Paulhus, D. L. (2002). “Socially desirable responding: the evolution of a construct,” in The Role of Constructs in Psychological and Educational Measurement, eds H. I. Braun, D. N. Jackson, and D. E. Wiley (New Jersey, NJ: Lawrence Erlbaum Associates Publishers).
Plutchik, R. (1980). “A general psychoevolutionary theory of emotion,” in Emotion: Theory, Research and Experience, Theories of Emotion Theories of Emotion, eds R. Plutchik and H. Kellerman (New York, NY: Academic press), 3–33. doi: 10.1016/b978-0-12-558701-3.50007-7
Pozharliev, R., Verbeke, W. J., Van Strien, J. W., and Bagozzi, R. P. (2015). Merely being with you increases my attention to luxury products: using EEG to understand consumers’ emotional experience with luxury branded products. J. Marketing Res. 52, 546–558. doi: 10.1509/jmr.13.0560
Reeves, B., Lang, A., Thorson, E., and Rothschild, M. (1989). Emotional television scenes and hemispheric specialization. Hum. Commun. Res. 15, 493–508. doi: 10.1111/j.1468-2958.1989.tb00196.x
Shang, Q., Jin, J., Pei, G., Wang, C., Wang, X., and Qiu, J. (2020). Low-order webpage layout in online shopping facilitates purchase decisions: evidence from event-related potentials. Psychol. Res. Behav. Manag. 13:29. doi: 10.2147/PRBM.S238581
Singh, M., Singh, M., and Gangwar, S. (2013). Emotion recognition using electroencephalography (EEG): a review. Int. J. Inform. Technol.Knowl. Manag. 7, 1–5. doi: 10.1016/b978-0-12-804490-2.00001-4
Stephens, G. J., Silbert, L. J., and Hasson, U. (2010). Speaker-listener neural coupling underlies successful communication. Proc. Natl. Acad. Sci. U.S.A. 107, 14425–14430. doi: 10.1073/pnas.1008662107
Tao, J., and Tan, T. (2005). “Affective computing: a review,” in Affective Computing and Intelligent Interaction. ACII 2005. Lecture Notes in Computer Science, eds J. Tao T. Tan, and R. W. Picard, Vol 3784, (Berlin: Springer), 981–995. doi: 10.1007/11573548_125
Telpaz, A., Webb, R., and Levy, D. J. (2015). Using EEG to predict consumers’ future choices. J. Marketing Res. 52, 511–529. doi: 10.1509/jmr.13.0564
Übeyli, E. D. (2008). Wavelet/mixture of experts network structure for EEG signals classification. Expert Syst. Appl. 34, 1954–1962. doi: 10.1016/j.eswa.2007.02.006
Vecchiato, G., Astolfi, L., De VicoFallani, F., Toppi, J., Aloise, F., Bez, F., et al. (2011a). On the use of EEG or MEG brain imaging tools in neuromarketing research. Comput. Intell. Neurosci. 2011:643489. doi: 10.1155/2011/643489
Vecchiato, G., Astolfi, L., Fallani, F. D. V., Cincotti, F., Mattia, D., Salinari, S., et al. (2010). Changes in brain activity during the observation of TV commercials by using EEG, GSR and HR measurements. Brain Topogr. 23, 165–179. doi: 10.1007/s10548-009-0127-0
Vecchiato, G., Cherubino, P., Maglione, A. G., Ezquierro, M. T. H., Marinozzi, F., Bini, F., et al. (2014). How to measure cerebral correlates of emotions in marketing relevant tasks. Cogn. Comput. 6, 856–871. doi: 10.1007/s12559-014-9304-x
Vecchiato, G., Toppi, J., Astolfi, L., Fallani, F. D. V., Cincotti, F., Mattia, D., et al. (2011b). Spectral EEG frontal asymmetries correlate with the experienced pleasantness of TV commercial advertisements. Med. Biol. Eng. Comput. 49, 579–583. doi: 10.1007/s11517-011-0747-x
Venkatraman, V., Dimoka, A., Pavlou, P. A., Vo, K., Hampton, W., Bollinger, B., et al. (2015). Predicting advertising success beyond traditional measures: new insights from neurophysiological methods and market response modeling. J. Marketing Res. 52, 436–452. doi: 10.1509/jmr.13.0593
Wei, Z., Wu, C., Wang, X., Supratak, A., Wang, P., and Guo, Y. (2018). Using support vector machine on EEG for advertisement impact assessment. Front. Neurosci. 12:76. doi: 10.3389/fnins.2018.00076
Yadava, M., Kumar, P., Saini, R., Roy, P. P., and Dogra, D. P. (2017). Analysis of EEG signals and its application to neuromarketing. Multimed. Tools Appl. 76, 19087–19111. doi: 10.1007/s11042-017-4580-6
Keywords: electroencephalography, affective computing, marketing, neural affective mechanisms, classification and recognition
Citation: Pei G and Li T (2021) A Literature Review of EEG-Based Affective Computing in Marketing. Front. Psychol. 12:602843. doi: 10.3389/fpsyg.2021.602843
Received: 04 September 2020; Accepted: 19 February 2021;
Published: 16 March 2021.
Edited by:
Tao Liu, Zhejiang University, ChinaCopyright © 2021 Pei and Li. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Taihao Li, lith@zhejianglab.com