Autonomic Nervous System and Recall Modeling in Audiovisual Emotion-Mediated Advertising Using Partial Least Squares-Path Modeling

Interest in improving advertisement impact on potential consumers has increased recently. One well-known strategy is to use emotion-based advertisement. In this approach, an emotional link with consumers is created, aiming to enhance the memorization process. In recent years, Neuromarketing techniques have allowed us to obtain more objective information on this process. However, the role of the autonomic nervous system (ANS) in the memorization process using emotional advertisement still needs further research. In this work, we propose the use of two physiological signals, namely, an electrocardiogram (heart rate variability, HRV) and electrodermal activity (EDA), to obtain indices assessing the ANS. We measured these signals in 43 subjects during the observation of six different spots, each conveying a different emotion (rational, disgust, anger, surprise, and sadness). After observing the spots, subjects were asked to answer a questionnaire to measure the spontaneous and induced recall. We propose the use of a statistical data-driven model based on Partial Least Squares-Path Modeling (PSL-PM), which allows us to incorporate contextual knowledge by defining a relational graph of unobservable variables (latent variables, LV), which are, in turn, estimated by measured variables (indices of the ANS). We defined four LVs, namely, sympathetic, vagal, ANS, and recall. Sympathetic and vagal are connected to the ANS, the latter being a measure of recall, estimated from a questionnaire. The model is then fitted to the data. Results showed that vagal activity (described by HRV indices) is the most critical factor to describe ANS activity; they are inversely related except for the spot, which is mainly rational. The model captured a moderate-to-high variability of ANS behavior, ranging from 38% up to 64% of the explained variance of the ANS. However, it can explain at most 11% of the recall score of the subjects. The proposed approach allows for the easy inclusion of more physiological measurements and provides an easy-to-interpret model of the ANS response to emotional advertisement.

1. Introduction

Interest in improving advertisement impact on potential consumers has increased recently. In this context, Neuromarketing techniques have allowed us to obtain more objective information on how the consumer unconsciously processes the information (Norton et al., 2007; Babiloni, 2015). In recent years, advertisements have tended to use emotion-based approaches to create an emotional link with consumers; the hypothesis is that this emotional engagement will enhance the memorization process and, eventually, the recall (Roberts, 2004; Vecchiato et al., 2010, 2014; Baraybar-Fernández et al., 2017).

Neuromarketing is the application of neurophysiological tools to assess and objectively understand human behavior concerning marketing product development (Lee et al., 2007; Babiloni, 2015). Nowadays, the main research techniques are the following: (1) neuroimaging technologies to assess brain activity, such as functional magnetic resonances (fMRI), electroencephalogram (EEG), and magnetoencephalogram (MNG), for market research (Ioannides et al., 2000; Vecchiato et al., 2010, 2014); and (2) autonomic nervous system (ANS) analysis tools, such as electrodermal analysis (EDA) and heart rate variability (HRV), to study emotion, cognition, and attention (Critchley, 2002; Cartocci et al., 2017). The latest, EDA and HRV, are physiological signals that can be registered with cheap and easy-to-use equipment.

The neuromarketing approach tries to assess, as objectively as possible, the emotional impact on how the memory is formed during the observation of advertisements. It has been shown that this emotional reaction and perception of the stimuli is mainly unconscious. In this context, HRV and EDA assess the emotional status, as indexes reflecting the ANS activity. Several studies used the HRV and EDA to create an emotional index (EI) based on the theoretical works about emotions, particularly regarding the valence and arousal plane of emotions (Mauss and Robinson, 2009; Babiloni, 2015; Cartocci et al., 2017). Some approaches have combined the sympathetic activity estimated from EDA and vagal activity estimated from HRV to propose a reliable index of the sympathovagal balance in cold stress test studies (Ghiasi et al., 2018, 2020). However, there is still a need for an improved understanding of the relationship between the sympathetic and vagal branches of the ANS and the recall induced by different emotions during observation of advertisements.

Different studies have demonstrated that positive or negative emotion stimuli are an important element when establishing memory traces (Kato, 2009; Baraybar-Fernández et al., 2017; Cartocci et al., 2017). Neuromarketing techniques allow us to objectively measure the reaction of consumers to emotion stimulus in advertisements, and it does so namely by assessing cerebral activity variations (Davidson and Irwin, 1999; Vecchiato et al., 2014; Babiloni, 2015) and ANS status modifications (Babiloni, 2015; Cartocci et al., 2017). The ANS plays an essential role in the emotional status of the subjects viewing advertisements. Different studies suggested an increase in the parasympathetic activity for positive emotions, whereas negative emotions (anger, fear, and sadness) result in parasympathetic withdrawal and sympathetic activation (McCraty et al., 1995; Kop et al., 2011).

In this work, we propose a modeling of the activity of the ANS as a result of modulation by sympathetic and vagal branches. Moreover, we model the impact of the ANS on recall. These are unobservable variables (Latent Variables, LV) that cannot be measured directly. We propose the use of several manifest variables (MV), i.e., directly measured indices, that are related to the LV. In this way, LVs represent abstract concepts that are combinations of the observable variables, i.e., MVs. Since we want to estimate ANS influence on recall, we propose the use of HRV- and EDA-based indices to assess the ANS activity. We used partial least squares path modeling approach (PLS-PM), which is an alternative method to covariance-based estimation for structural equation models (SEM) (Wold, 1975; Tenenhaus et al., 2005).

PLS-PM does not rely on strong assumptions for probability distributions of the variables and error. It is a data-driven approach in which the aim is to provide a model to explain the relationship between LVs and MVs. It can be viewed as an explanatory approach more so than a confirmatory one (Esposito et al., 2010; Sánchez, 2013; Serrano et al., 2014). Both properties, the lack of assumptions, and the explanatory approach are well-suited to address the problem we are facing in this study. We propose an assessment of the ANS by HRV indices, which are known to characterize both sympathetic and vagal branches, as well as by EDA indices, which mainly characterize sympathetic tone (Malik et al., 1996; Boucsein, 2012; Greco et al., 2016). These indices will represent the MVs in the PLS-PM. These physiological variables are measured in subjects during observation of advertisements conveying different emotional stimuli. Once finished the observation, we measured recall using a questionnaire, which evaluates the spontaneous and induced recall. Finally, the PLS-PM was complemented with a recall score for each spot.

The structure of the manuscript is as follows. In section 2, the experimental setup and signal acquisition are described. In section 3, the ANS evaluation using HRV and EDA-based indices is stated. In section 4, the statistical modeling of the ANS and the recall using PLS-PM are described. In section 5, the results of the proposed model are presented. Finally, the discussion and our conclusions are presented in section 6.

2. Experimental Setup

The subjects in the experiment were asked to look at a computer screen at first with neutral content (black screen). The physiological recording equipment was correctly placed, and the subject was left for 1 min in this situation to stabilize and serve as a baseline. After that, the subject observed six different advertisements in a row. Once the visualization ended, the subjects were asked to answer a questionnaire that measured both spontaneous and suggested recall.

Six advertising spots of 1 min in length each were used in this experiment. Four of them expressed predominantly one emotion, namely, surprise, sadness, disgust, and anger, and two included formative content (rational messages). The advertisements were selected by a group of eight experts (four academics and four publicists), and the order of the spots was determined randomly to minimize the primacy (first) effect and recency (last) effect. The place of the spots can generate higher recall in the study population (Glanzer and Cunitz, 1966). The final order of the messages was the following: (1) rational, traffic speed in cities; (2) disgust, cocaine use; (3) anger, child abuse; (4) surprise, alcohol and driving; (5) sadness, speed and distractions on the road; and (6) rational, addiction treatments. The videos are part of the Supplementary Material.

The experiment was conducted with a total of 66 participants (30 males and 36 females) aged between 18 and 41 (22.5 ± 4.45 years old). Participants were healthy students of different undergraduate or post-graduate degrees or staff of the Rey Juan Carlos University. All subjects gave their informed consent and were declared to be healthy and without heart conditions. The experiment had the approval of the University ethic committee. ECG and EDA signals were continuously recorded during the experiment. Signals were recorded using BITalino system (Plux Wireless Biosignals S.A. Portugal), with a sampling rate of 1 kHz. The ECG was registered using a derivation from three electrodes, positive and negative on the wrists, and the reference electrode on the forearm of the subject. EDA signal was recorded placing a pair of electrodes on the left palm. After this procedure we manually reviewed the data and discarded subjects with poor signal quality (a lot of artifacts, loose electrodes, interruptions in the recording). The final data set ended up with 43 subjects (25 females).

This questionnaire measures the spontaneous and suggests recall. The questionnaire has three sections:

• Demographic variables: age, gender, education, and nationality.

• Spontaneous recall: participants are asked about their direct remembrance of the main message conveyed in the advertisements as a free text. They are also asked what emotions they felt, in degrees akin to a Lickert scale, and in relation to which advertisements as a free text.

• Suggested recall: different situations are presented, and participants are asked if those situations happened in the spots (yes/no questions). If the answer is yes, participants are asked to select the messages they remember from that particular situation. The list of messages includes those that appeared in the spots, among others, as a multiple-choice question.

The complete questionnaire is available in the Supplementary Material in Spanish.

We quantified recall by assigning to each question an individual score, which was maxed out if the subject recalled correctly (emotions and messages conveyed in the advertisements). The final score was the addition of individual scores. The recall-score ranged from 0 (no recall at all) to 15 points (perfect recall).

3. ANS Characterization Using HRV and EDA

In this section, the processing of the ECG and EDA signals to obtain ANS markers is described. The aim is to describe the indices we finally used as MV in the final PLS-PM model we propose. The interested reader can find a more detailed description in Malik et al. (1996) and Greco et al. (2016).

3.1. ECG-HRV Analysis

The heart behavior is not constant; instead, there exists a variation in the time intervals between consecutive heartbeats. The normal heart rhythm is controlled by the cardiac sinoatrial (SA) node, modulated by innervation from both the sympathetic and the vagal branches of the ANS. The SA node is the final mechanism responsible, through its repetitive nervous impulses, for generating heartbeats.

Both the sympathetic and parasympathetic responses have antagonist roles, i.e., the sympathetic activity increases the heart rate, whereas the vagal activity slows down the heart rate. In rest conditions, there is a dynamic balance state between these systems, which is responsible for the variability in the intervals between consecutive heartbeats. At the same time, the ANS is influenced by many other systems (respiratory system, vasomotor system, central nervous system, or renin-angiotensin system), which also indirectly contribute to modulate the heart rate through it (Malik et al., 1996).

HRV is the variation in the intervals between consecutive heartbeats, and it allows for non-invasive investigation of the ANS state. The most straightforward time signal representation of the HRV is the tachogram, representing the time between consecutive beats (RR-intervals or NN-intervals) vs. the interval number. The NN-intervals are the time distance between R-waves of consecutive beats, excluding the ectopic beats (those that do not have their origin at the SA node).

Several methods have been proposed in the literature to assess HRV. Time-domain and spectral-domain methods are the most widely used indices due to their simplicity and interpretability. A sub-class of time-domain methods, known as statistical methods, involves calculating the standard deviation of either the RR-intervals series or its first difference (Malik et al., 1996; Mietus et al., 2002). Table 1 summarizes the statistical indices computed in this work. Additionally, we use the mean of the NN intervals, AVNN.

TABLE 1

Table 1. Time and frequency domain indices of HRV.

Spectral methods indicate how the power of the signal is distributed as a function of the frequency f. HRV found in healthy subjects during rest is influenced by respiratory activity and the feedback mechanism of the systems for regulation of temperature and blood pressure. The different systems oscillate spontaneously at rest with characteristic frequencies at different intervals. By quantifying the power of the spectral components, information related to autonomic cardiac function may be pointed out (Cerutti et al., 1995; Persson, 1997; Sörnmo and Laguna, 2005).

Four spectral bands are mainly distinguished in the power spectral density of the NN series, namely, the high frequency (HF) band with f ∈ (0.15, 0.4) Hz, low frequency (LF) band with f ∈ (0.04, 0.15) Hz, very low frequency (VLF) band with f ∈ (0.003, 0.04) Hz, and ultra low frequency (ULF) band with f < 0.003 Hz (Akselrod et al., 1991). Frequency-domain indices are calculated based on these spectral bands (Malik et al., 1996). Due to the length limitations of the signals, in this work, we focus on the frequency-domain indices listed in Table 1.

HF power is generally taken as an index of cardiac parasympathetic tone, although some studies have shown that it is, to some degree, also affected by the sympathetic tone and may not be a pure parasympathetic index. The LF index is more controversial. Many studies have pointed out that LF power reflects sympathetic and vagal activity and that the LF/HF ratio reflects sympathovagal balance. In short-term HRV recordings under experimental conditions affecting autonomic response, the relative LF power contribution is often considered a marker of sympathetic activity. However, some studies using pharmacological, physiological, or psychological manipulations affecting the sympathetic activity and HRV have questioned this association. One crucial fact is that vagal blockade strongly reduces LF power, whereas sympathetic blockade has no significant effect. A review of these studies and the corresponding references can be found in Iwase et al. (2017).

3.2. EDA–Analysis

The conductance of the skin is regulated by the sweat gland activity. This modulation is the cause of the EDA. In this context, EDA has been viewed as a sensitive and straightforward way to assess the sympathetic arousal associate with emotion (Vecchiato et al., 2014; Posada-Quintero and Chon, 2020). Since sympathetic function directly modulates the sudomotor activity, EDA is indicated to evaluate sympathetic autonomic function. Indeed, several studies have used the EDA signal, along with HRV, to build an emotional index (EI) in the context of Neuromarketing techniques (Cartocci et al., 2017). The EI represents the valence and arousal response to external stimuli, which is mediated by the ANS.

The EDA signal has two main components: the skin conductance level (SCL) and the skin conductance response (SCR). SLC represents the overall conductance of the tonic component of EDA. It is related to slow shifts of the EDA signal (Greco et al., 2016; Posada-Quintero and Chon, 2020). To obtain the tonic component (SCL), a low-pass filter with cut-frequency 0.1 Hz was applied (Vecchiato et al., 2014). We quantified the contribution of the SCL as the mean value during the observation of the advertisement. The other component of the EDA signal, SCR, is the response to external stimuli. To automatically detect the SCR, we used the algorithm proposed in Kim et al. (2004). In this algorithm, the EDA signal is first downsampled to 20 samples per second. The signal is then differentiated to emphasize high-frequency content, and it is finally convolved with a 20-point Bartlett window. This processing produces a spiky signal in which the spikes correspond to SCRs, and simplify the detection procedure. To account for this part, we proposed the use of the number of SCRs (ns_scr) as an index of the phasic component of the EDA signal for each advertisement.

4. ANS and Recall Modeling Using PLS-PM

PLS-PM is an iterative algorithm that estimates the relationship between MVs and LVs by the weights of multiple and simple regressions. PLS-PM allows to also obtain linear relationship between LVs (Wold, 1975; Tenenhaus et al., 2005). A full path model is comprised of two submodels: (1) the measurement model, which establishes the relationship between each LV and its own MVs; and (2) the structure model, which considers the relationship between LVs (Tenenhaus et al., 2005; Sánchez, 2013).

• LV–Sympathetic: We associated several indices that, in the scientific literature, are related to the sympathetic activity, namely, from HRV analysis, lf, sdnn, and, from EDA analysis ns_scr, mean_scl (Posada-Quintero et al., 2016; Ghiasi et al., 2020; Posada-Quintero and Chon, 2020). Indices from EDA are clearly related to the sympathetic activity, while the HRV indices raise more controversy about its association to sympathetic and vagal branches (Malik and Camm, 1993; Coumel et al., 1994; Sacha, 2014; Medeiros et al., 2018). However, PLS-PM allows us to measure the adequacy of belonging to each LV, and this allows for the convenience of changing membership to another LV.

• LV–Vagal: The MVs associated with this LV are, from HRV analysis, hf, pnn50, rmssd. The evidence from scientific literature supports the relationship between these indices and the Vagal activity (Malik and Camm, 1993; Malik et al., 1996; Medeiros et al., 2018).

• LV–ANS: The MVs associated with this LV are, from HRV analysis, the classical indices that assess the sympatho-vagal balance: lf_hf, avnn (Akselrod et al., 1991; Malik and Camm, 1993; Malik et al., 1996).

• LV–Recall: This LV represents the recall of the subjects, measured by the recall-score described in section 2.

Regarding the structural model, it was assumed that the ANS directly depends on the Sympathetic and Vagal LVs, while the recall construct depends directly on ANS. The complete scheme of the structural and the measurement model can be seen in Figure 1.

FIGURE 1

Figure 1. PLS-PM model of the ANS activity and Recall. Dashed lines represent the links in the measurement model, and solid lines represent links in structure model.

The model was designed such that MVs are considered to be caused by the latent variables, i.e., reflective indicators. This assumption imposes a restriction since all the MVs are measuring the same LVs. All MVs therefore have to be highly correlated (Sánchez, 2013). Consequently, some of the MVs had to change their sign to follow along with the remaining MVs. The structure model is statistically represented by two linear regression models: (1) ANS as a function of Sympathetic and Vagal activity and (2) recall as a function of the ANS. The path coefficients (β) were obtained as classical weights in linear regression, i.e., using a least-square approach (Esposito et al., 2010). The overall fit of the final model was assessed by the goodness-of-fit (Tenenhaus et al., 2004). Gender differences are studied using a permutation resampling procedure to determine whether path coefficients are statistically different (Sánchez, 2013).

5. Results

In this section, we summarize the most relevant results. Firstly, before the analysis of the PLS-PM models, we present the analysis of the recall scores for the different emotions represented in the spots. Next, we present the results of the measurement model, which allows us to identify the adequacy of the MVs as valid indicators of the LV. Finally, we present the results for the structure model and compare the different results for different emotions represented in the spots.

5.1. Recall Scores

Figure 2 shows the distribution of the recall-score grouped by gender. The final score was composed as points obtained when the subject provided a correct answer in the questionnaire. The recall-score measured both spontaneous and induced recall. In this context, the higher the score, the higher the recall. Most emotions generated a similar recall-score; the only exception was the spot with Anger emotion, showing less significant recall-score. The mean value recall-score for all spots but Anger was 6.0, while the recall-score for Anger spot was 4.0. The difference by gender was non-significant, with slightly higher recall-score values for males, in general.

FIGURE 2

Figure 2. Recall-score distribution grouped by gender and for the different emotions conveyed in the advertisements. Emotions are represented in the same order as the subject watched the spots.

5.2. Structure Model: ANS and Recall PLS-PM Model

Figure 3 shows the path coefficients in the structure model. As shown in Figure 3, the structure model is comprised of two relational submodels: (1) the ANS as a function of the Sympathetic and Vagal activity, which is represented in Figure 3 by the two first blocks (Symp → ANS; Vagal → ANS), and (2) recall as a function of the ANS activity, third block (ANS → Recall).

FIGURE 3

Figure 3. Path coefficients in the structure model comparison between emotions conveyed by the advertisements. Values in y-axes should be interpreting as standardize linear regression weights so that a negative value indicates a negative correlation and the higher the value the higher the contribution of the LV.

The behavior is very similar to all emotions except for Rational in the last advertisement. In general, the vagal activity is negatively correlated with the ANS activity, whereas sympathetic activity positively correlated with ANS. As expected, the Vagal branch had a higher impact on ANS behavior. This behavior could be explained because it is more complicated to obtain isolated indices that account only for the Sympathetic activity. Indeed, the vagal LV was always statistically significant for every spot, whereas the sympathetic LV was only statistically significant in Rational 1, Surprise, and Sadness spots (p-value <0 0.05). The ANS LV was only statistically significant explaining the Recall LV in the Sadness spot (p-value < 0.05).

It is worth noting that the model's behavior for the last spot (Rational 2) was completely different. Indeed, this advertisement was utterly different from the others. Even though the first spot was labeled as Rational, it is about hits by cars on urban streets, and it contains real images of hits. Nonetheless, the last spot only contains speeches of doctors about the importance of rehab.

Figure 4 shows the results of the fitted PLS-PM for each spot, in particular, R², GoF, and β path coefficients for the inner model. The coefficient of determination, R² can be understood as the percentage of the variance of the ANS that is explained by the sympathetic and vagal activity, and the percentage of the variance of recall that is explained by ANS. It can be seen the different behavior of Spot 6, which induced a different relationship between LV.

FIGURE 4

Figure 4. Results of fitted PLS-PM. The plots show R², GoF, and β path coefficients for each spot. Significant coefficients are highlighted. Red arrows indicate a negative relationship, whereas black arrows represent a positive relationship. (A) Spot 1–Rational, (B) Spot 2–Disgust, (C) Spot 3–Anger, (D) Spot 4–Surprise, (E) Spot 5–Sadness, (F) Spot 6–Rational.

The PLS-PM model captured a moderate-high variability of ANS behavior, ranging from 38% up to 64% of the explained variance of the ANS. However, the model is only able to account, at most, for 11% of the recall variance (as measured by the recall-score). The overall performance of the model is moderate, with goodness-of-fit values around 0.4 for all spots.

There are no significant differences between male and females, except for Spot 1 (rational), in which the relationships between LV's is the opposite by gender, and the difference is statistically significant for the relationship between Sympathetic and ANS LVs (see Figure 5).

FIGURE 5

Figure 5. Gender differences for Spot 1 (rational) on β path coefficients. Statistically significant differences are indicated with a *.

6. Discussion and Conclusions

We propose a statistical model based on PLS-PM to assess the ANS activity, as described by sympathetic and vagal branches, on subjects during advertisement observation. Subject memory formation was evaluated by filling a survey to measure recall, both spontaneous and assisted. To characterize the ANS, the ECG and EDA were measured for every subject. From these physiological signals, several autonomic indices were computed, namely, HRV time-domain and frequency-domain indices and two indices from EDA. The PLS-PM approach allows us to incorporate contextual knowledge by defining a so-called structural model composed of unobservable variables (LVs), which are constructs defined by the measured indices (MVs). In the proposed model, we set two LVs accounting for sympathetic and vagal activity. An additional LV accounts for the ANS balance, which is related to the previous LVs. Finally, the ANS is associated with a fourth LV accounting for recall.

We conducted an experiment where 66 subjects watched six advertisements while ECG and EDA were recorded. After signal quality review, a data set from 43 subjects (25 females) was available. Each advertisement conveyed a different emotion, namely: rational, disgust, anger, surprise, sadness, and rational.

The emotions caused an interaction between sympathetic and vagal activity in such a way that when the vagal tone increased, the net balance of the ANS decreased, and when the sympathetic tone increased, the net balance of the ANS increased. This interaction was quite the opposite when the advertisements were utterly rational. The recall is an abstract concept that is hard to define and more complicated to quantify. We tried to obtain a quantitative measure of the recall by giving a score based on the survey's answer. However, the model, based on the estimation of the ANS, was only able to explain 12% of the variance of this recall-score.

In this work, we proposed to model recall as a consequence of the emotional state. This modeling can only be achieved thoroughly using a combination of classical and neuromarketing techniques (Shen and Morris, 2016; Baraybar-Fernández et al., 2017). Accordingly, we used questionnaires to measure the degree of recall and neuromarketing tools to assess the emotional state through the ANS response. Our results agree with previous studies in which different induced emotions modify the ANS response (McCraty et al., 1995; Kop et al., 2011). Moreover, our results suggest that the influence of the ANS on the recall is opposite in emotion-based and fact-based advertisements (rational).

Even though HRV is a marker of the ANS control on cardiovascular activity, there are some difficulties in assessing the sympathetic activity from the HRV power spectrum accurately. Some approaches proposed to combine sympathetic information estimated from EDA and vagal information estimated from HRV, yielding a reliable sympathovagal assessment (Ghiasi et al., 2018, 2020).

Both cardiac and respiratory systems are regulated by the ANS, whose mechanisms are not fully understood and are still subject to scientific discussion (Malik and Camm, 1993; Betts et al., 2013; Reyes del Paso et al., 2013). Modulation of the heart rate by respiration, known as respiratory sinus arrhythmia, changes the RR interval duration according to the respiration cycle. Parasympathetic activity is very closely related to respiratory sinus rhythm, and in rest and healthy conditions, it generally contributes to HF power. However, under some conditions, the respiration rate may affect LF power by overestimating sympathetic activity. Furthermore, the parasympathetic activity can be underestimated when the respiratory rate is higher than the HF band (Eckberg, 2009; Karemaker, 2009). Some approaches have been published in the scientific literature trying to separate respiratory influences from the HR for a better estimate of the sympathovagal balance (Varon et al., 2018). In the present study, with healthy subjects in rest conditions, we do not consider a significant influence of the respiratory rate. However, for future works, it is worth analyzing whether the separation of respiratory influences helps knowledge acquisition related to the interactions assessed in this study.

We have proposed an easy-to-interpret model to explain the recall-score and the ANS response to emotional advertisements, which could help to design emotion-based advertisements. An improvement to this approach could be to include physiological information from the brain activity by an EEG, such as incorporating theta and alpha bands power in prefrontal cortex leads (Vecchiato et al., 2014; Cartocci et al., 2017). A limitation is that the model only accounts for linear relationships between MVs and LVs. However, it is well-known that the ANS balanced emotion mediated could have some non-linearity. A simple extension with which to account for non-linearity is the kernelized version of the PLS-PM algorithm (Tenenhaus, 2009).

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Comité de Ética de la Investigación de la Universidad Rey Juan Carlos. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

ÓB-P and RG-E designed the experiment and the idea for the work, they conducted the processing of the signal, analysis of the data, and writing of the manuscript. MC-V and AV-V conducted the experiments with the subjects and gathered, curated the database, and also contributed with the writing of the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This work was partially supported by Projects TEC2016-75361-R and PID2019-105032GB-I00 from the Ministerio de Ciencia e Innovación (Spanish Government).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors kindly appreciate the collaboration of professors Miguel Baños and Antonio Baraybar, for their support designing the experiments, and thorough discussions about the art of communicating using advertisements. The authors kindly appreciate the discussions and comments about heart rate physiology from Dr. Arcadi García-Alberola at Hospital Universitario Virgen de la Arrixaca de Murcia and Dr. Rafael García-Carretero Hospital Universitario de Móstoles.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2020.576771/full#supplementary-material

References

Akselrod, S., Gordon, D., Ubel, F. A., Shannon, D. C., Berger, A. C., and Cohen, R. J. (1991). Power spectrum analysis of heart rate fluctuation: a quantitative probe of beat-to-beat cardiovascular control. Science 213, 220–222. doi: 10.1126/science.6166045

PubMed Abstract | CrossRef Full Text

Babiloni, F. (2015). “Neuromarketing,” in International Encyclopedia of the Social & Behavioral Sciences, eds N. J. Smelser and P. B. Baltes (Elsevier), 698–702. doi: 10.1016/B978-0-08-097086-8.56029-2

CrossRef Full Text | Google Scholar

Baraybar-Fernández, A., Baños González, M., Barquero-Pérez, O., Goya-Esteban, R., and de-la Morena-Gómez, A. (2017). Evaluation of emotional responses to television advertising through neuromarketing. Comunicar 25, 19–28. doi: 10.3916/C52-2017-02

CrossRef Full Text | Google Scholar

Betts, J., DeSaix, P., Johnson, E., Johnson, J., and Korol, O. (2013). Anatomy and Physiology. OpenStax College.

Boucsein, W. (2012). Electrodermal Activity. Boston, MA: Springer US.

Google Scholar

Cartocci, G., Myriam Carat, Ù., Modica, E., Maglione, A. G., Rossi, D., Cherubino, P., et al. (2017). Electroencephalographic, heart rate, and galvanic skin response assessment for an advertising perception study: application to antismoking public service announcements. J. Vis. Exp. 126:55872. doi: 10.3791/55872

CrossRef Full Text | Google Scholar

Cerutti, S., Bianchi, A. M., and Mainardi, L. T. (1995). “Spectral analysis of the heart rate variability,” in Heart Rate Variability, eds M. Malik and A. J. Camm (New York, NY: Futura Publishing Company), 63–74.

Google Scholar

Coumel, P., Maison-Blanche, P., and Catuli, D. (1994). Heart rate and heart rate variability in normal young adults. J. Cardiovasc. Electrophysiol. 5, 899–911. doi: 10.1111/j.1540-8167.1994.tb01130.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Critchley, H. D. (2002). Review: electrodermal responses: what happens in the brain. Neuroscientist 8, 132–142. doi: 10.1177/107385840200800209

PubMed Abstract | CrossRef Full Text | Google Scholar

Davidson, R. J., and Irwin, W. (1999). Neuroanatomy of emotion and affective style. Trends Cogn. Sci. 3:11. doi: 10.1016/S1364-6613(98)01265-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Eckberg, D. L. (2009). Point: counterpoint: respiratory sinus arrhythmia is due to a central mechanism vs. respiratory sinus arrhythmia is due to the baroreflex mechanism. J. Appl. Physiol. 106, 1740–1742. doi: 10.1152/japplphysiol.91107.2008

PubMed Abstract | CrossRef Full Text | Google Scholar

Esposito, V., Trinchera, L., and Amato, S. (2010). “PLS path modelling: from foundations to recent developments and open issues for model assessment and improvement,” in The Handbook of Partial Least Squares: Concepts, Methods and Applications, eds V. Esposito Vinzi, W. W. Chin, J. Henseler, and H. Wang (Berlin: Springer), 47–82. doi: 10.1007/978-3-540-32827-8_3

CrossRef Full Text | Google Scholar

Ghiasi, S., Greco, A., Barbieri, R., Scilingo, E. P., and Valenza, G. (2020). Assessing autonomic function from electrodermal activity and heart rate variability during cold-pressor test and emotional challenge. Sci. Rep. 10:5406. doi: 10.1038/s41598-020-62225-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Ghiasi, S., Greco, A., Nardelli, M., Catrambonel, V., Barbieri, R., Scilingo, E., et al. (2018). “A new sympathovagal balance index from electrodermal activity and instantaneous vagal dynamics: a preliminary cold pressor study,” in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Honolulu, HI: IEEE), 3068–3071. doi: 10.1109/EMBC.2018.8512932

PubMed Abstract | CrossRef Full Text | Google Scholar

Glanzer, M., and Cunitz, A. R. (1966). Two storage mechanisms in free recall. J. Verbal Learn. Verbal Behav. 5, 351–360. doi: 10.1016/S0022-5371(66)80044-0

CrossRef Full Text | Google Scholar

Greco, A., Valenza, G., and Scilingo, E. P. (2016). Advances in Electrodermal Activity Processing With Applications for Mental Health. Cham: Springer International Publishing.

Google Scholar

Ioannides, A. A., Liu, L., Theofilou, D., Dammers, J., Burne, T., Ambler, T., et al. (2000). Real time processing of affective and cognitive stimuli in the human brain extracted from MEG signals. Brain Topogr. 13, 11–19. doi: 10.1023/A:1007878001388

PubMed Abstract | CrossRef Full Text | Google Scholar

Iwase, S., Hayano, J., and Orimo, S. (2017). Clinical Assessment of the Autonomic Nervous System. Chapter 9. Tokyo: Springer.

Karemaker, J. M. (2009). Counterpoint: respiratory sinus arrhythmia is due to the baroreflex mechanism. J. Appl. Physiol. 106, 1742–1743. doi: 10.1152/japplphysiol.91107.2008a

PubMed Abstract | CrossRef Full Text | Google Scholar

Kato, J. (2009). Neural correlates of attitude change following positive and negative advertisements. Front. Behav. Neurosci. 3:9. doi: 10.3389/neuro.08.006.2009

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, K. H., Bang, S. W., and Kim, S. R. (2004). Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. 42, 419–427. doi: 10.1007/BF02344719

PubMed Abstract | CrossRef Full Text | Google Scholar

Kop, W. J., Synowski, S. J., Newell, M. E., Schmidt, L. A., Waldstein, S. R., and Fox, N. A. (2011). Autonomic nervous system reactivity to positive and negative mood induction: the role of acute psychological responses and frontal electrocortical activity. Biol. Psychol. 86, 230–238. doi: 10.1016/j.biopsycho.2010.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, N., Broderick, A. J., and Chamberlain, L. (2007). What is “neuromarketing?” A discussion and agenda for future research. Int. J. Psychophysiol. 63, 199–204. doi: 10.1016/j.ijpsycho.2006.03.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Malik, M., Bigger, J., Camm, A., Kleiger, R., Malliani, A., Moss, A., et al. (1996). Heart rate variability standards of measurement, physiological interpretation, and clinical use. Eur. Heart J. 17, 354–381. doi: 10.1093/oxfordjournals.eurheartj.a014868

PubMed Abstract | CrossRef Full Text | Google Scholar

Malik, M., and Camm, A. (1993). Components of heart rate variability—what they really mean and what we really measure. Am. J. Cardiol. 72, 821–822. doi: 10.1016/0002-9149(93)91070-X

PubMed Abstract | CrossRef Full Text | Google Scholar

Mauss, I. B., and Robinson, M. D. (2009). Measures of emotion: a review. Cogn. Emot. 23, 209–237. doi: 10.1080/02699930802204677

PubMed Abstract | CrossRef Full Text | Google Scholar

McCraty, R., Atkinson, M., Tiller, A., Rein, G., and Watkins, A. D. (1995). The effects of emotions on short-term power spectrum analysis of heart rate variability. Am. J. Cardiol. 76:15. doi: 10.1016/S0002-9149(99)80309-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Medeiros, A. R., Michael, S., and Boullosa, D. A. (2018). Make it easier! Evaluation of the ‘vagal-sympathetic effect’ in different conditions with R–R intervals monitoring. Eur. J. Appl. Physiol. 118, 1287–1288. doi: 10.1007/s00421-018-3855-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Mietus, J. E., Peng, C.-K., Henry, I., Goldsmith, R. L., and Goldberger, A. L. (2002). The pNNx files: re-examining a widely used heart rate variability measure. Heart 88, 378–380. doi: 10.1136/heart.88.4.378

PubMed Abstract | CrossRef Full Text | Google Scholar

Norton, M. I., Frost, J. H., and Ariely, D. (2007). Less is more: the lure of ambiguity, or why familiarity breeds contempt. J. Pers. Soc. Psychol. 92:97. doi: 10.1037/0022-3514.92.1.97

PubMed Abstract | CrossRef Full Text | Google Scholar

Persson, P. B. (1997). Spectrum analysis of cardiovascular time series. Am. J. Physiol. Regul. Integr. Compar. Physiol. 273, 1201–1210. doi: 10.1152/ajpregu.1997.273.4.R1201

CrossRef Full Text | Google Scholar

Posada-Quintero, H. F., and Chon, K. H. (2020). Innovations in electrodermal activity data collection and signal processing: a systematic review. Sensors 20:479. doi: 10.3390/s20020479

PubMed Abstract | CrossRef Full Text | Google Scholar

Posada-Quintero, H. F., Florian, J. P., Orjuela-Cañón, A. D., and Chon, K. H. (2016). Highly sensitive index of sympathetic activity based on time-frequency spectral analysis of electrodermal activity. Am. J. Physiol. Regul. Integr. Compar. Physiol. 311, R582–R591. doi: 10.1152/ajpregu.00180.2016

PubMed Abstract | CrossRef Full Text | Google Scholar

Reyes del Paso, G. A., Langewitz, W., Mulder, L. J., Van Roon, A., and Duschek, S. (2013). The utility of low frequency heart rate variability as an index of sympathetic cardiac tone: a review with emphasis on a reanalysis of previous studies. Psychophysiology 50, 477–487. doi: 10.1111/psyp.12027

PubMed Abstract | CrossRef Full Text | Google Scholar

Roberts, K. (2004). Lovemarks: The Future Beyond Brands. New York, NY: Power House Books.

Google Scholar

Sacha, J. (2014). Interplay between heart rate and its variability: a prognostic game. Front. Physiol. 5:347. doi: 10.3389/fphys.2014.00347

PubMed Abstract | CrossRef Full Text | Google Scholar

Sánchez, G. (2013). PLS Path Modeling With R. Berkeley, CA: Trowchez Editions.

Google Scholar

Serrano, E., López-Soria, S., Trinchera, L., and Segalés, J. (2014). The use of null models and partial least squares approach path modelling (PLS-PM) for investigating risk factors influencing post-weaning mortality in indoor pig farms. Epidemiol. Infect. 142, 530–539. doi: 10.1017/S0950268813001295

PubMed Abstract | CrossRef Full Text | Google Scholar

Shen, F., and Morris, J. D. (2016). Decoding neural responses to emotion in television commercials: an integrative study of self-reporting and fMRI measures. J. Advertis. Res. 56:193. doi: 10.2501/JAR-2016-016

CrossRef Full Text | Google Scholar

Sörnmo, L., and Laguna, P. (2005). Bioelectrical Signal Processing in Cardiac and Neurological Applications. 1st Edn. New York, NY: Elsevier Academic Press.

Google Scholar

Tenenhaus, A. (2009). “Kernel PLS path modelling,” in 6th International Conference on Partial Least Squares and Related (Beijing), 13–16.

Google Scholar

Tenenhaus, M., Amato, S., and Vinzi, V. E. (2004). “A global goodness–of–fit index for PLS structural equation modelling,” in Proceedings of the XLII SIS Scientific Meeting (Padova), 739–742.

Google Scholar

Tenenhaus, M., Vinzi, V. E., Chatelin, Y.-M., and Lauro, C. (2005). PLS path modeling. Comput. Stat. Data Anal. 48, 159–205. doi: 10.1016/j.csda.2004.03.005

CrossRef Full Text | Google Scholar

Varon, C., Lázaro, J., Bolea, J., Hernando, A., Aguiló, J., Gil, E., et al. (2018). Unconstrained estimation of hrv indices after removing respiratory influences from heart rate. IEEE J. Biomed. Health Inform. 23, 2386–2397. doi: 10.1109/JBHI.2018.2884644

PubMed Abstract | CrossRef Full Text | Google Scholar

Vecchiato, G., Astolfi, L., De Vico Fallani, F., Cincotti, F., Mattia, D., Salinari, S., et al. (2010). Changes in brain activity during the observation of TV commercials by using EEG, GSR and HR measurements. Brain Topogr. 23, 165–179. doi: 10.1007/s10548-009-0127-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Vecchiato, G., Maglione, A. G., Cherubino, P., Wasikowska, B., Wawrzyniak, A., Latuszynska, A., et al. (2014). Neurophysiological tools to investigate consumer's gender differences during the observation of TV commercials. Comput. Math. Methods Med. 2014, 1–12. doi: 10.1155/2014/912981

PubMed Abstract | CrossRef Full Text | Google Scholar

Wold, H. (1975). “Path models with latent variables: the NIPALS approach,” in Quantitative Sociology, ed M. H. Blalock (New York, NY: Elsevier), 307–357. doi: 10.1016/B978-0-12-103950-9.50017-4

CrossRef Full Text | Google Scholar