Intragroup Emotions: Physiological Linkage and Social Presence

We investigated how technologically mediating two different components of emotion—communicative expression and physiological state—to group members affects physiological linkage and self-reported feelings in a small group during video viewing. In different conditions the availability of second screen text chat (communicative expression) and visualization of group level physiological heart rates and their dyadic linkage (physiology) was varied. Within this four person group two participants formed a physically co-located dyad and the other two were individually situated in two separate rooms. We found that text chat always increased heart rate synchrony but HR visualization only with non-co-located dyads. We also found that physiological linkage was strongly connected to self-reported social presence. The results encourage further exploration of the possibilities of sharing group member's physiological components of emotion by technological means to enhance mediated communication and strengthen social presence.

Introduction

Emotional contagion—the tendency for emotions between two or more people to converge—is a well-established phenomenon (Barsade, 2002). Not only emotions displayed during face-to-face interaction but also mediated (text-based) emotional cues have been found to elicit similar emotions in others (Salminen et al., 2013). Physiological linkage—the synchronization of physiological activity across individuals—has been suggested as being one underlying mechanism of emotional contagion (Hatfield et al., 1993; Bruder et al., 2012). Bruder et al. (2012) suggested that in addition to physiological linkage, a social appraisal process exists, which also leads to emotional convergence within dyads. While the exact nature of emotions is still an open research question, it is widely agreed that emotions are complex constructs with several subcomponents: feelings, expressions and physiological states—and synchrony can occur on all these different levels. In the present study we investigated how technologically mediating two different components of emotion—expression and physiological state—to group members affects physiological linkage and various self-reports in a small group during video viewing. In different conditions the availability text chat (communicative expression) and visualization of group level physiological heart rates and their dyadic linkage (physiology) was varied.

Physiological Linkage

Psychophysiological measurements can provide real time data on the physiological states of participants that are directly related to their emotional states (Cacioppo et al., 2000) which in turn are affected by reactions to media (Ravaja, 2004). In a social group setting these methods can also be utilized in studying the effects of social dynamics on media experience. Physiological linkage refers to the extent to which physiological signals of two or more people are associated with each other, such as a mutual increase in heart rate (HR) during a shared experience. Physiological signals in general, but also linkage indices specifically, can provide information on the emotional and cognitive state of the user or a group of users that would normally remain unobservable.

The physiological linkage can occur through various processes. In addition to pure chance, the most common cause for joint changes in physiology within dyads in a given situation is simply the shared external stimulus to which they are both reacting in similar manner, e.g., the movie clip itself in a shared viewing situation. This type of physiological linkage is not the type of linkage we are currently interested in as it does not reveal anything about the connection of physiological linkage and social presence, or emotional contagion. Consequently this source of linkage should mostly be controlled in an experimental setup when studying other causes of linkage, as the causes cannot be distinguished from one another if they are all present. Bruder et al. (2012) distinguishes two other paths for emotional convergence that lead to physiological linkage: contagion-based and appraisal-based.

The mirror neuron system (Rizzolatti et al., 1996; Rizzolatti and Craighero, 2004) involved in imitating the perceived movements of other people, is a plausible neurophysiological mechanism underlying contagion-based physiological linkage. According to the embodied cognition theory (see Niedenthal et al., 2005; Barsalou, 2008; Mahon and Caramazza, 2008), in order to understand for example the facial expressions of others, the same brain areas of the observer must be activated that are used in producing them (Niedenthal et al., 2001). Mimicry of facial expressions as such is well-known phenomenon (e.g., Dimberg and Öhman, 1996; Korb et al., 2014), and within embodied cognition framework the basis of this phenomenon is that the perception of a smile causes similar brain activation as when the person would be smiling herself. This in essence means that the perception and mimicry of other person's emotional expressions leads to convergence of emotional states in the observer (Hatfield et al., 1993; Rizzolatti and Craighero, 2004). This in turn leads to the phenomenon of emotional contagion where emotional states are transferred from one person to another (Hatfield et al., 1993; Barsade, 2002; Bruder et al., 2012).

Appraisal-based emotional convergence occurs through social appraisal processes (Bruder et al., 2012) where individuals assess other people's emotional expressions in a given situation and based on them cognitively form a more shared understanding of the emotional situation, that then leads to emotional convergence, and the linkage of the physiological component of emotion. Appraisal-based path to emotional convergence assumably is less strongly connected to physiological linkage than contagion-based convergence, as it is not directly caused by physiological mimicry but is a result of higher social appraisal functions.

According to Emotions as Social Information (EASI) model, the social role of emotions is emphasized in ambiguous situations where the amount of more explicit social information is limited (Van Kleef, 2009, 2010; Van Kleef et al., 2010). Indeed, the amount of utilizable information is technologically mediated interaction is typically limited, and previous studies have found it to hinder the perception and mimicry of the other person's emotional states (Garcia et al., 1999). Often mediated interaction and communication takes place when participants are in physically separate locations, which naturally limits the channels of information, and the impact of merely being physically present with someone in the same space is cut off. Studies suggest however that even when physically separated, the social context and motives have an impact on emotional expressions and feelings (Hess et al., 1995; Jakobs et al., 1999a,b; Bruder et al., 2012). In such situations appraisal-based paths of convergence are arguably emphasized over contagion-based. Therefore, providing additional social information, such as expressive communicative social signals (e.g., text chat) and information on emotions through signal visualizations of physiological states (e.g., heart rates), may provide usable social information to an ambiguous situation and facilitate mimicry and emotional contagion between the participants.

Physiological linkage was first used in analysis of marital interactions, where several linkage indices were associated with conflict conversations (Levenson and Gottman, 1983). Later studies have shown that physiological linkage is also related to e.g., empathy (Levenson and Ruef, 1992, 1997) and performance (Henning et al., 2001). These studies highlight the fact that linkage is not associated only with negative interactions. A more recent suggestion is that linkage captures the intensity of social interactions that is elevated in, but not specific to, interpersonal conflicts (Chanel et al., 2012). Similarly, sense of social presence (Biocca and Harms, 2002)—a sense of dyadic interconnectedness or being together with other people in a given context—is linked to physiological linkage (Ekman et al., 2012; Järvelä et al., 2013).

Present Study

In the present investigation, we studied how providing socially utilizable information on two different components of emotion (expression and physiology) affected the emotional and social experiences of the participants, e.g., social presence, and especially physiological linkage between dyads during movie watching in small groups. Specifically, two different types of socially utilizable information and methods of mediating and presenting it through technological means were chosen for the experiment: (1) text chat on a second screen provided an expressive social information channel which was shared to all participants and only displayed information they contributed voluntarily, and (2) heart rate visualization displayed socially utilizable information on physiological state of the participants and their dyadic linkage continuously to the participants. Ordinarily physiological linkage between persons is not directly observable, but here a visualization of synchrony index that was measured continuously in real time was shown to the participants (see Methods for details). In a sense this enables social appraisal of a complex physiological state and appraisal-based emotional convergence. Such visualization enables the examination of whether the conscious acknowledgment of linkage has an effect on feelings of social presence and could increase emotional contagion within the group.

It has been found that although engaging in text chat during movie watching requires additional attention, it also increases liking and feeling of closeness within the group (Weisz et al., 2007). One of the key characteristics of text chat is asynchronous production where the actual writing of the message is not in synchrony with the actual interactive discussion; that is, the message is written first, sent later, and possibly read and replied by others sometime afterwards. This combined with the possibility for each contributor to write messages at the same time often splits chats into threads where replies to messages do not instantly follow but older messages can be replied to O'Neill and Martin (2003). This asynchronous nature of text chat makes it an activity that does not cause physiological linkage in group members just by providing rhythmic synchronous activity or stimulus—if chat increases physiological linkage it is because of its social aspects and the information it is used to communicate, that is, it is social appraisal-based.

Heart rate is one of the most common and arguably one of the most well-known measures in psychophysiology. Depending on the context, heart rate changes have been used to index increased attention, emotional arousal, and cognitive effort (Ravaja, 2004; Cacioppo et al., 2007). The synchronization of heart beats between two persons is a phenomenon which has been studied for example between patients and therapists, and between mothers and children (see Levenson and Ruef, 1997 for a review). Unlike most other neurophysiological measures, such as electroencephalography EEG or electrodermal activity etc., heart rate measures (e.g., beats per minute, BPM) are rather intuitive to understand and people commonly have a preliminary grasp on what they imply (e.g., arousal) (Janssen et al., 2013). This intuitiveness is essential when visualizing biosignals in an attempt to provide meaningful information to participants concerning their own physiological state. Intuitiveness allows using rather straightforward visualizations (e.g., beating heart icons for heart rate) whereas less intuitive ones would require more elaborated metaphorical visualizations (e.g., clock's hands moving at different speed representing different bands of EEG activity). Although heart rate visualization itself is probably not necessarily enough to cause heart beat synchronization by itself, it can mediate heart rate information to others and increase physiological linkage between two or more people through higher-order social processes. One possible channel for how biosignal visualizations would support physiological linkage is by increasing interoceptive awareness (Craig, 2002)—the awareness of your own bodily state—of all participants, and bringing their attention to the group process of linkage. It can be seen either as making contagion-based convergence easier by boosting interoceptive awareness, or by transforming physiological information into a form that can be utilized in social appraisals. Increased interoceptive awareness has also been found to amplify the experience itself (Dunn et al., 2010).

It would be expected that text chat evokes a sense of human connection and provides emotional cues and information regarding other people (e.g., their feelings toward the movie, opinions, and interests), thereby potentially leading to an increased similarity between the emotional states of the participants. This increased interconnectedness is assumed to manifest as higher reported social presence and physiological linkage. In accordance with previous studies (e.g., Wagner et al., 2015) we expect participants to report increased pleasantness and arousal when chatting as the possibility share emotions and to interact with group members is presumed to be positive experience by default, but HR visualization is not expected to produce such an effect by itself. However, heart rate visualization is expected to increase both physiological linkage and social presence as it provides socially utilizable information on emotional states of the participants. In accordance with the EASI model of emotion (Van Kleef, 2009, 2010; Van Kleef et al., 2010) which states that the social role of emotions is emphasized in ambiguous situations, a pronounced effect is expected in regard to both chat and heart rate visualization when the participants are not physically co-located and where the amount of other socially utilizable information is lower. When spatially separated, some information channels are not in use and the whole situation is more ambiguous and this heightens the importance of the social and emotional cues provided by either chat or heart rate visualization.

The results of this study provide insight on how intragroup emotions are influenced by sharing different emotion components between group members by technological means. It also explores if a visualization of a physiological signal is sufficient emotional information to increase dyadic physiological linkage. In addition, on a more applied level the results contribute to how shared media experience can be enhanced by providing features that strengthen the social presence between group members during media enjoyment. Media is increasingly often enjoyed in situations where the group members are physically in separate locations and they are interacting through technical means. As the second screen phenomenon where e.g., television viewers use tablets to enhance their media experience through various means (Courtois and D'heer, 2012), is quickly spreading, the technical solutions through which these features can be implemented are opening up.

Methods

Participants

Participants were 62 (21 males and 41 females) Finnish university students whose age ranged from 19 to 35 years (M = 24.2 years). All participants provided informed consent prior to the beginning of the experiment. Due to some participants not arriving at the experiment and data lost due to technical reasons, the number of subjects in different analyses varied from 52 to 57.

Stimuli

The stimuli shown in the present study consisted of four video clips, whose duration ranged from 5 min 59 s to 6 min 16 s. The video clips contained no spoken narrative and they were selected to elicit varying emotional valence (unpleasant to pleasant) and arousal (calmness to excitement) levels. Themes consisted of religion (pleasant low-arousal), parkour (pleasant high-arousal), poverty (unpleasant low-arousal), and climbing (unpleasant high-arousal). Religion and poverty clips were selected from the movie “Baraka” (Magidson Films 1992; directed by Ron Fricke), and the parkour and climbing clips were obtained from YouTube Internet service (http://www.youtube.com).

Procedure

Participants arrived to the experiment in 16 four-participant groups. Two of the four participants formed one dyad at the beginning of the experiment so that one dyad was physically located in the same room, and the other two participants were located in two separate rooms. This setup of one co-located dyad and two separated individual participants aimed at comparing the effects of physical (co-located) and mediated (non-co-located) presence while they all were interacting together as a group (Figure 1).

FIGURE 1

Figure 1. The experimental setup.

Due to participant cancellations, three of the groups had only three participants. In one of these groups two participants were assigned to the same room (co-located condition), and in two groups two participants were assigned to different rooms (non-co-located condition). The remaining third subject took part in the experiment but was not included in the analysis. After instructions and a demonstration of the stimulus presentation system, baseline physiological measurements were recorded during a 5-min rest period. The participants were sitting in a chair facing the television screen (co-located dyad were sitting side by side) where video stimuli was presented in each room, and mobile devices were used for providing textual feedback from the participants. Participants listened to the video clip soundtracks via headphones, so that the co-located participants were not able to hear each other. The co-located dyad could see each other, but were mainly facing the television screen, and the individual participants in separate rooms did not have visual contact to other participants during the experiment.

For each group, the four video stimuli were assigned randomly to four display conditions defined by the inclusion of chat and heart rate (HR) displays (both off, only chat display on, only HR display on, and both on). The presentation order of conditions was randomized for all groups. During chat display, all the participants were able to read and write messages online (Figure 2B). During HR display, the heart rates of all participants, as well as the linkage between participant pairs, were displayed (Figure 2A). Linkage was defined as the correlation between a pair's HR signals, calculated on-the-fly within a 30-s moving time window. A 30-s time window was adopted in order to continuously visualize relatively recent changes in HR synchrony while allowing enough data points (typically, 20–60 heart beats) for calculating the correlation. The extent of HR correlation between each pair of participants was visualized with line thickness and color coding (gray for positive and blue for negative correlation; see Figure 2A).

FIGURE 2

Figure 2. The presentation view on the large screen when both chat and heart rate visualization are on (A) and the chat display on the mobile device (B).

To facilitate social interaction within the participant groups during stimulus presentation, questions related to the contents of each presented video clip (e.g., “Why do many religions encourage women to wear veils?”) were presented every 2 min (a total of three questions per video clip), participants were given a restricted amount of time (1 min 30 s) for providing their answers, and all answers were then displayed on the large screen until the beginning of next question (i.e., for 30 s). The facilitation was implemented to ensure at least a minimum amount of social interaction between the participants on a group level as the Finnish culture is not particularly extroverted.

After each video clip session, the participants filled a series of self-report questionnaires (see section Self-reports). After the questionnaires were filled by all participants, the next video was shown.

Heart Rate Measurement and Technical Setup

The heart rates of all participants were measured with prototypical Polar Band heart rate monitors (Polar Electro; http://www.polar.fi). Videos were displayed on 40″ plasma TV screens, approximately 150–200 cm in front of the participants. Nokia N900 mobile devices (Nokia Corporation; http://www.nokia.com) operating on Maemo software platform (http://www.maemo.org) were used for presenting questions related to the video clips, providing answers to these questions, as well as for writing textual messages to other participants during chat display (see Section Procedure).

Stimulus presentation, HR and chat displays, and data collection were controlled with a specialized presentation system PRESEMO (see Figure 2; Kuikkaniemi et al., 2010), which has been developed for presenting various multimedia material (e.g., videos, text, and graphs) while allowing the audience to interact with the presentation and with each other via mobile devices. All data from Nokia N900 mobile devices and Polar Band monitors were transferred wirelessly over a Bluetooth connection to a centralized PRESEMO server (for further technical details, see ibid).

Self-reports

Emotional Evaluations

Participants rated their own emotional reactions to the video viewing sessions in terms of valence, arousal, and dominance on 9-step graphical scales. These scales were similar to Lang's (1980) Self-Assessment Manikin (SAM).

Interpersonal Evaluations

The participants were asked to evaluate a series of 17 items measuring social presence (Biocca and Harms, unpublished)—that is, the degree to which they felt they were sharing a common experience—with their assigned (co-located or non-co-located) pair during video viewing. The following facets of social presence were measured: co-presence (e.g., “I often felt as if my partner and I were together in the same room”), attentional engagement (e.g., “I paid close attention to my partner”), emotional contagion (e.g., “I was sometimes influenced by my partner's moods”), comprehension (e.g., “I was able to communicate my intentions clearly to my partner”), and behavioral interdependence (e.g., “My actions were often dependent on my partner's actions”). For each 17 items they evaluated on 7 point scale (1 = I strongly disagree, 7 = I strongly agree). Participants also answered a series of eight questions measuring physical presence (e.g., “When the broadcast ended, I felt like I came back to the “real world” after a journey”) (Kim and Biocca, 1997). In contrast to social presence which refers explicitly to socially shared experiences, physical presence refers to the feeling of being physically present in the depicted virtual environment (Lee, 2004). To evaluate the sense of attraction with their pairs, participants were additionally asked to answer 13 questions on a 5-point scale, e.g., boring vs. interesting, cold vs. warm (Moreland and Beach, 1992). All of these scales have been shown to exhibit sufficient reliability (Moreland and Beach, 1992; Kim and Biocca, 1997; Harms and Biocca, 2004).

Data Pre-processing and Analysis

A fundamental difference between the experimental HR visualization and the post-experimental HR data analysis was that the former was updated continuously on-the-fly, whereas the latter was done retrospectively for all the recorded data. HR measurements obtained from Polar Band devices were pre-processed in Matlab (version 7.10.0). HR data was first resampled to 32 Hz. Unrealistic values (3 standard deviations from mean, considering only values between 45 and 145 bpm) were replaced by interpolation. Cubic splines were used for both interpolation and resampling. Frequencies below 0.04 Hz were filtered out by removing a moving average cubic polynomial component from each individual data series. Resultant data series were smoothed with cubic polynomial in a 500 ms time window, and series mean was removed from each participant's data. When quantifying the HR linkage between the two members of each dyad, to allow some temporal lag between the physiological reactions, a ± 5-s temporal window was used in calculating the cross correlations (between the dyad members) for each HR sample (we assumed that the co-occurrence of physiological reactions with a longer temporal lag than 5 s is unlikely to be related to social processes). The highest cross-correlation value within this window was selected for the analysis. Mean cross-correlation coefficients were calculated separately for each film. Finally, Fisher transformation was applied to normalize the distribution of resultant values.

Conventional statistical methods such as analysis of variance (ANOVA) would not have been appropriate for the present data, which were hierarchical such that participants were nested within participant dyads, which were further nested within groups of two dyads. Instead, we adopted a multilevel modeling procedure that is a generalization of the more restricted ANOVA method (Quené and van den Bergh, 2004; Hoffman and Rovine, 2007; see (Hayes, 2006) for an excellent introduction on multilevel models), and which is particularly useful for the analysis of dyadic data (Kenny et al., 2006). Specifically, the data were analyzed by the Linear Mixed Models procedure in SPSS (version 18) with maximum likelihood estimation. With HR data, cross-correlations had been calculated for movie conditions that were presented repeatedly to subject pairs. Respectively, subject pair identifiers were specified as the subject variable and movie (four different movies) as the repeated variable. Unstructured variance-covariance structure (UN) was selected for the residuals based on best fit to the data (estimated with -2 log likelihood function). To account for the hierarchy of pairs within groups, a random intercept was specified with groups as the subject variable. A fixed-effects model was specified with main effects for movie (four levels), location (two levels: co-located and non-co-located), chat display (two levels: on, off), and HR display (two levels: on, off); as well as two 2-way interactions “location × chat display” and “location × HR display.”

Self-reports were available from all participants. Therefore, when analysing questionnaire data, participant identifiers were specified as the subject variable, and an additional random intercept was defined for subject pairs to account for the hierarchy between participants and pairs. The analysis remained otherwise identical to that of the HR data. When examining the association of self-reported social presence with HR cross-correlations, social presence scores were first averaged over both members in each pair and grand-mean centered. The HR data analysis was then repeated with a fixed-effects model that included only a main effect for this continuous covariate.

Results

Results from LMM analyses for emotional and interpersonal evaluations are shown in Tables 1, 2, respectively.

TABLE 1

Table 1. Linear mixed model analyses for emotional evaluations.

TABLE 2

Table 2. Linear mixed model analyses for interpersonal evaluations.

Manipulation Checks

Videos exerted significantly different effects on emotional valence and arousal ratings (Table 1). Pairwise post-hoc comparisons with Bonferroni correction confirmed that a priori pleasant videos (religion: M = 6.3; parkour: M = 6.2) elicited higher valence ratings than a priori unpleasant videos (poverty: M = 4.2; climbing: M = 5.5; SE = 0.2 for all videos). In contrast, a priori high-arousal videos (parkour: M = 4.6, SE = 0.2; climbing: M = 4.8, SE = 0.3) failed to elicit significantly higher arousal ratings than a priori low-arousal videos (religion: M = 4.1, SE = 0.2; poverty: M = 5.1, SE = 0.2). To control for any confounds caused by the different videos, the main effect of video was retained in all analyses. Significant video effects emerged for some variables (Table 2). Post-hoc comparisons showed that the poverty video elicited higher attraction ratings than climbing and parkour videos (Ms = 3.5, 3.3, and 3.3), higher physical presence than parkour video (Ms = 2.8 and 2.4), and higher perceived behavioral interdependence ratings than parkour video (Ms = 2.2 and 1.8, SE = 0.1 for all effects). Preliminary analyses showed that interactions between the video condition and chat display, HR display, and location conditions were non-significant for all dependent variables (ps > 0.05).

Emotional Evaluations

Chat display exerted significant effects on all emotional evaluations (Table 1). Mean evaluations for emotional evaluations in chat and HR display conditions can be seen in Figure 3A. The participants reported feeling more aroused, in control of the situation, and more pleasant, when chat was available. In contrast, the HR visualization did not have a similar effect on these components. The location of pairs did not interact significantly with the chat or HR (see Figure 4A and Table 1) display conditions.

FIGURE 3

Figure 3. Mean evaluations for emotional (A) and interpersonal evaluations (B,C) by chat and HR display conditions. Interpersonal evaluations are shown separately for variables measured on 5-step (B) and 7-step scales (C). Error bars refer to standard errors of the mean.

FIGURE 4

Figure 4. Mean evaluations for emotional (A) and interpersonal evaluations (B,C) by location and chat display conditions. Interpersonal evaluations are shown separately for variables measured on 5-step (B) and 7-step scales (C). Error bars refer to standard errors of the mean.

Interpersonal Evaluations

Social presence subscales all showed sufficient reliability (co-presence, Cronbach's α = 0.77, 4 items; Perceived Emotional Contagion, α = 0.90, 4 items; Perceived Comprehension, α = 0.92, 3 items; and Perceived behavioral interdependence, α = 0.89, 3 items) except for the 3 item Perceived Attentional Engagement subscale, that had Cronbach'sα = 0.57. Despite relatively low reliability, scores for Perceived Attentional Engagement are still reported here as they showed very similar results to other social presence subscales. Physical Presence (α = 0.87, 8 items) and Attraction (α = 0.87, 13 items) scales showed high reliability.

Chat display exerted significant effects for all interpersonal evaluations, whereas the effects of HR display were all non-significant (Table 2). As can be seen in Figures 3B,C, chat display elicited greater attraction, physical presence, and social presence (as measured by all of the five social presence subscales) ratings.

A significant interaction between location and chat display (Table 2) demonstrated that the effect of chat display on co-presence evaluation was more pronounced with the non-co-located pairs (Figure 4C). In contrast, there were no significant interactions between location and HR display (see Table 2). Contrary to expectations, chat display increased evaluated physical presence more with co-located rather than with non-co-located pairs (Table 2 and Figure 4B), but this is possibly because increased social presence also emphasized the physical presence of being in the same room. Unexpectedly, non-co-located pairs reported significantly greater social presence as measured with emotional contagion and comprehension subscales (Table 2 and Figure 4C). However, for emotional contagion there was also a non-significant trend (p < 0.10) toward a greater chat display effect for non-co-located pairs. Given that this effect was similar to that of co-presence, it is possible that these location main effects may have stemmed from interaction effects between location and chat display.

Physiological Linkage

Mean HR cross-correlations between subject pairs for chat and HR display conditions are presented in Figure 5. The main effect of chat display, as well as the interaction effects between location and chat display and location and HR display were significant (Table 3). In general, HR cross-correlations were higher when the chat display was enabled but were not affected by the HR display. Importantly, however, both the effects of chat and HR displays were more pronounced for the non-co-located pairs (Figure 5). Simple effect analyses for non-co-located pairs indicated significant chat display, F_{(1, 53.20)} = 17.26, p < 0.001, and HR display effects, F_{(1, 58.41)} = 4.32, p = 0.04. With co-located pairs, non-significant effects were observed for both chat, F_{(1, 60.23)}, = 0.02, and HR displays, F_{(1, 62.22)} = 1.41. Table 4 displays statistical tests for the associations between HR cross-correlations and the social presence evaluations. The results demonstrated that the HR synchrony between participant pairs showed a significant positive correlation with all evaluated social presence scales, which emphasizes how social presence and physiological linkage are connected.

FIGURE 5

Figure 5. Mean HR cross-correlation values by location and display conditions. Display conditions are shown separately for chat and HR displays. Error bars refer to standard errors of the mean.

TABLE 3

Table 3. Linear mixed model analyses for heart rate cross-correlations.

TABLE 4

Table 4. Linear mixed model analyses for the associations between HR cross-correlations and social presence evaluations.

Discussion

In our experiment we set out to examine how providing more socially utilizable emotional information to participants in a group media consumption situation would affect their experience. The text chat option provided sporadic voluntary communicative emotional expressions to the group while heart rate visualization showed continuous involuntary information on group member's physiological state and their dyadic linkage to other group members.

In this setting, text chat was clearly an effective method of affecting the experience. The subjects reported higher feelings of valence, arousal, attraction, social, and physical presence. The HR visualization by itself did not have such an effect. Simple biosignal visualization of group members' physiological state was not enough to significantly affect self-reported feelings. However, HR visualization and text chat both increased physiological linkage (heart rate cross-correlation) when the participants were physically non-co-located. The idea of physiological linkage as a measure of social presence was supported as they were positively correlated with every subscale of self-reported social presence. These results are in accordance with our initial hypotheses, except that HR visualization had weaker effects throughout than expected. When physically co-located, the effects were weaker, especially with HR visualization, which did not have noticeable effect. Our interpretation for this is that it was because the visualization of a physiological component of emotion is harder to interpret and of lesser information value than expressive social signals such as text chat. A possible interpretation for the lack of effect for HR visualization in co-located condition is that contagion-based emotional convergence is a more natural path to utilize the information on physiological state, but that path was already fully in operation when the participants were physically co-located, thus the visualization did not provide anything more by converting the physiological state into form that social appraisal processes can utilize. Another possibility is that in non-co-located condition when the amount and type of social information is lesser, those that are available are emphasized, and consequently the HR visualization is more effective when other forms of information are not available. This would also explain why HR synchrony was higher in non-co-located situations and not just close to co-located situation. It can be concluded that the availability of socially utilizable information, whether it was text chat or HR visualization, increased physiological linkage and associated social presence when the amount of socially utilizable regular information was low, e.g., when communication between subjects was only by technological means. This finding supports the EASI model of emotion, which states that the social role of emotions is emphasized in ambiguous situations. The solid connection between physiological linkage and self-reported social presence supports the idea that social presence could, at least partly, be the subjective feeling component of physiological linkage—however this hypothesis naturally requires further research into the topic.

There were some challenges during the process, mainly with the data quality of the consumer grade heart rate monitor, which is why the data had to be processed rather heavily before the analysis. With a higher quality data, more advanced HR indices could have been calculated and a shorter time window used for linkage calculations. In general, the quality of the non-filtered heart signals might explain the absence of effects of the HR visualization. However, the data quality was sufficient for the results presented here Optimally, the number or participants should have been larger to compensate for the small effect sizes. Now some of the results lack statistical power, and perhaps even more solid results would have been acquired with a larger sample size. Also, as the two information types (text chat and HR visualization) varied in more ways than one (e.g., voluntary vs. involuntary, real-time vs. delay, sporadic vs. continuous) exact interpretations for the results are difficult. Also, the HR visualization provided information not only on the heart rates of group members but also the dyadic linkage between them, and it is impossible to separate the effects from each other. In addition, in a sense linkage scores were used both as dependent and independent variables in the setup. We might not be able to precisely say what caused the difference between chat and HR visualization, but examining those two still provides us with an overview of how a typical mediated communication affects social and emotional states and also how it can be still enhanced with less common biosignal visualizations providing usable emotional and social information.

The positive results acquired in this experiment raise several questions for future studies. For example would a different type of heart rate visualization or a different biosignal altogether produce different effects? Or does the type of emotion (e.g., positive or negative valence, different discrete emotions) experienced affect how physiological synchrony is associated with feelings of social presence? How would the manipulation of the social context (e.g., cooperation vs. competition) affect physiological linkage? Can different paths of convergence be experimentally separated, e.g., do they require different time scales to operate?

Overall, our interpretation is that technological augmentation provides emotional cues and socially utilizable information, and affects intragroup emotions especially when regular communication is somehow limited. For example, text chat is effective when talking is prohibited or considered disturbing (like during movie watching), and sharing indices of group's shared physiological synchrony is effective when the group members are physically separated from each other. In a sense, these technical solutions compensate for the lack of emotional cues and information that exist in typical face-to-face communication. Their promising potential for augmenting various group situations should be further studied and experimented with. In addition to providing practical solutions for modern technologically mediated communication, this line of research will reveal more fundamental dynamics how group-level emotional expressions and their sharing affects group emotions, and how they manifest in physiological responses and their synchrony.

Author Contributions

GC designed the study and performed the experiment, PH pre-processed the data and calculated synchrony indices, JK analyzed the data and wrote the manuscript with SJ and NR.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This research was partly funded by TEKES (Project diary number 417/31/09)—the Finnish Funding Agency for Innovation.

References

Barsade, S. G. (2002). The ripple effect: emotional contagion and its influence on group behavior. Admin. Sci. Q. 47, 644–675. doi: 10.2307/3094912

CrossRef Full Text | Google Scholar

Barsalou, L. W. (2008). Grounded cognition. Annu. Rev. Psychol. 59, 617–645. doi: 10.1146/annurev.psych.59.103006.093639

PubMed Abstract | CrossRef Full Text | Google Scholar

Biocca, F., and Harms, C. (2002). “Defining and measuring social presence: contribution to the networked minds theory and measure,” in Proceedings of PRESENCE 2002 (Porto).

Google Scholar

Bruder, M., Dosmukhambetova, D., Nerb, J., and Manstead, A. S. (2012). Emotional signals in nonverbal interaction: dyadic facilitation and convergence in expressions, appraisals, and feelings. Cogn. Emot. 26, 480–502. doi: 10.1080/02699931.2011.645280

PubMed Abstract | CrossRef Full Text | Google Scholar

Cacioppo, J. T., Tassinary, L. G., and Berntson, G. G. (2000). “Psychophysiological science,” in Handbook of Psychophysiology, 2nd Edn., eds J. T. Cacioppo, L. G. Tassinary, and G. G. Berntson (New York, NY: Cambridge University Press), 3–26.

Cacioppo, J. T., Tassinary, L. G., and Berntson, G. G. (2007). Handbook of Psychophysiology. New York, NY: Cambridge University Press.

Google Scholar

Chanel, G., Kivikangas, J. M., and Ravaja, N. (2012). Physiological compliance for social gaming analysis: cooperative versus competitive play. Interact. Comput. 24, 306–316. doi: 10.1016/j.intcom.2012.04.012

CrossRef Full Text | Google Scholar

Courtois, C., and D'heer, E. (2012). “Second screen applications and tablet users: constellation, awareness, experience, and interest.” EuroiTV ′12 Proceedings of the 10th European Conference on Interactive Tv and Video (Berlin), 153–156. doi: 10.1145/2325616.2325646

CrossRef Full Text | Google Scholar

Craig, A. D. (2002). How do you feel? interoception: the sense of the physiological condition of the body. Nat. Rev. Neurosci. 3, 655–666. doi: 10.1038/nrn894

PubMed Abstract | CrossRef Full Text | Google Scholar

Dimberg, U., and Öhman, A. (1996). Behold the wrath: psychophysiological responses to facial stimuli. Motiv. Emot. 20, 149–82. doi: 10.1007/BF02253869

CrossRef Full Text | Google Scholar

Dunn, B. D., Galton, H. C., Morgan, R., Evans, D., Oliver, C., Meyer, M., et al. (2010). Listening to your heart - how interoception shapes emotion experience and intuitive decision making. Psychol. Sci. 21, 1835–1844. doi: 10.1177/0956797610389191

PubMed Abstract | CrossRef Full Text | Google Scholar

Ekman, I., Chanel, G., Järvelä, S., Kivikangas, J. M., Salminen, M., and Ravaja, N. (2012). Social interaction in games: measuring physiological linkage and social presence. Simul. Gaming 43, 321–338. doi: 10.1177/1046878111422121

CrossRef Full Text | Google Scholar

Garcia, O., Favela, J., and MacHorro, R. (1999). “Emotional Awareness in Collaborative Systems,” in 6th International Symposium on String Processing and Information Retrieval. 5th International Workshop on Groupware (Cat. No.PR00268) (Cancun: IEEE). doi: 10.1109/SPIRE.1999.796607

CrossRef Full Text | Google Scholar

Harms, C., and Biocca, F. (2004). “Internal consistency and reliability of the networked minds measure of social presence measure,” in Seventh Annual International Workshop: Presence 2004, eds M. Alcaniz and B. Rey (Valencia).

Google Scholar

Hatfield, E., Cacioppo, J. T., and Rapson, R. L. (1993). Emotional contagion. Curr. Dir. Psychol. Sci. 2, 96–100.

PubMed Abstract

Hayes, A. F. (2006). A primer on multilevel modeling. Hum. Commun. Res. 32, 385–410. doi: 10.1111/j.1468-2958.2006.00281.x

CrossRef Full Text | Google Scholar

Henning, R. A., Boucsein, W., and Gil, M. C. (2001). Social–physiological compliance as a determinant of team performance. Int. J. Psychophysiol. 40, 221–232. doi: 10.1016/S0167-8760(00)00190-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Hess, U., Banse, R., and Kappas, A. (1995). The intensity of facial expression is determined by underlying affective state and social situation. J. Pers. Soc. Psychol. 69, 280–288. doi: 10.1037/0022-3514.69.2.280

CrossRef Full Text | Google Scholar

Hoffman, L., and Rovine, M. J. (2007). Multilevel models for the experimental psychologist: foundations and illustrative examples. Behav. Res. Methods 39, 101–117. doi: 10.3758/BF03192848

PubMed Abstract | CrossRef Full Text | Google Scholar

Jakobs, E., Manstead, A. S. D., and Fischer, A. H. (1999a). Social motives, emotional feelings, and smiling. Cogn. Emot. 13, 321–345. doi: 10.1080/026999399379212

CrossRef Full Text | Google Scholar

Jakobs, E., Manstead, A. S. D., and Fischer, A. H. (1999b). Social motives and emotional feelings as determinants of facial displays: the case of smiling. Pers. Soc. Psychol. Bull. 25, 424–435. doi: 10.1177/0146167299025004003

CrossRef Full Text | Google Scholar

Janssen, J. H., Ijsselsteijn, W. A., Joyce, W., Tacken, P., and de Vries, G.-J. (2013). The tell-tale heart. Int. J. Synt. Emot. 4, 65–91. doi: 10.4018/jse.2013010103

CrossRef Full Text | Google Scholar

Järvelä, S., Kivikangas, J. M., Kätsyri, J., and Ravaja, N. (2013). Physiological linkage of dyadic gaming experience. Simul. Gaming 45, 24–40. doi: 10.1177/1046878113513080

CrossRef Full Text | Google Scholar

Kenny, D. A., Kashy, D. A., and Cook, W. L. (2006). Dyadic Data Analysis. New York, NY: The Guilford Press.

Google Scholar

Kim, T., and Biocca, F. (1997). Telepresence via television: two dimensions of telepresence may have different connections to memory and persuasion. J. Comput. Med. Commun. 3. Available online at: http://onlinelibrary.wiley.com/doi/10.1111/j.1083-6101.1997.tb00073.x/abstract

Korb, S., With, S., Niedenthal, P., Kaiser, S., and Grandjean, D. (2014). The perception and mimicry of facial movements predict judgments of smile authenticity. PLoS ONE 9:e99194. doi: 10.1371/journal.pone.0099194

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuikkaniemi, K., Kosunen, I., Turpeinen, M., Saari, T., Laitinen, T., and Lievonen, P. (2010). “Designing biofeedback for games and playful applications,” in CHI'10: Workshop at the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, GA).

Google Scholar

Lang, P. J. (1980). “Behavioral treatment and bio-behavioral assessment: computer applications,” in Technology in Mental Health Care Delivery Systems, eds J. B. Sidowski, J. H. Johnson, and T. A. Williams (Norwood, NJ: Ablex Publishing), 119–137.

Lee, K. M. (2004). Presence, explicated. Commun. Theory 14, 27–50. doi: 10.1111/j.1468-2885.2004.tb00302.x

CrossRef Full Text | Google Scholar

Levenson, R. W., and Gottman, J. M. (1983). Marital interaction: physiological linkage and affective exchange. J. Pers. Soc. Psychol. 45:587. doi: 10.1037/0022-3514.45.3.587

PubMed Abstract | CrossRef Full Text | Google Scholar

Levenson, R. W., and Ruef, A. M. (1992). Empathy: a physiological substrate. J. Pers. Soc. Psychol. 63:234. doi: 10.1037/0022-3514.63.2.234

PubMed Abstract | CrossRef Full Text | Google Scholar

Levenson, R. W., and Ruef, A. M. (1997). “Physiological aspects of emotional knowledge and rapport.” in Empathic Accuracy, ed W. Ickes (New York, NY: The Guilford Press), 44–72.

Mahon, B. Z., and Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. J. Physiol. 102, 59–70. doi: 10.1016/j.jphysparis.2008.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Moreland, R. L., and Beach, S. R. (1992). Exposure effects in the classroom: the development of affinity among students. J. Exp. Soc. Psychol. 28, 255–276.

Google Scholar

Niedenthal, P. M., Barsalou, L. W., Winkielman, P., Krauth-Gruber, S., and Ric, F. (2005). Embodiment in attitudes, social perception, and emotion. Pers. Soc. Psychol. Rev. 9, 184. doi: 10.1207/s15327957pspr0903_1

PubMed Abstract | CrossRef Full Text | Google Scholar

Niedenthal, P. M., Brauer, M., Halberstadt, J. B., and Innes-Ker, Å. H. (2001). When did her smile drop? facial mimicry and the influences of emotional state on the detection of change in emotional expression. Cogn. Emot. 15, 853–864. doi: 10.1080/02699930143000194

CrossRef Full Text | Google Scholar

O'Neill, J., and Martin, D. (2003). “Text chat in action,” in Proceedings of the 2003 International ACM SIGGROUP Conference on Supporting Group Work—GROUP ′03 (New York, NY: ACM Press). doi: 10.1145/958160.958167

CrossRef Full Text | Google Scholar

Quené, H., and van den Bergh, H. (2004). On multi-level modeling of data from repeated measures designs: a tutorial. Speech Commun. 43, 103–121. doi: 10.1016/j.specom.2004.02.004

CrossRef Full Text | Google Scholar

Ravaja, N. (2004). Contributions of psychophysiology to media research: review and recommendations. Media Psychol. 6, 193–235. doi: 10.1207/s1532785xmep0602_4

CrossRef Full Text | Google Scholar

Rizzolatti, G., and Craighero, L. (2004). The mirror-neuron system. Ann. Rev. Neurosci. 27, 169–192. doi: 10.1146/annurev.neuro.27.070203.144230

PubMed Abstract | CrossRef Full Text | Google Scholar

Rizzolatti, G., Fadiga, L., Gallese, V., and Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cogn. Brain Res. 3, 131–141. doi: 10.1016/0926-6410(95)00038-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Salminen, M., Ravaja, N., Kallinen, K., and Saari, T. (2013). Mediated cues of group emotion during knowledge-work tasks: effects on subjective and physiological responses. Interact. Comput. 25, 60–73. doi: 10.1093/iwc/iws006

CrossRef Full Text | Google Scholar

Van Kleef, G. A. (2009). How emotions regulate social life the emotions as social information (EASI) model. Curr. Dir. Psychol. Sci. 18, 184–188. doi: 10.1111/j.1467-8721.2009.01633.x

CrossRef Full Text | Google Scholar

Van Kleef, G. A. (2010). The emerging view of emotion as social information. Soc. Personal. Psychol. Compass 4, 331–343. doi: 10.1111/j.1751-9004.2010.00262.x

CrossRef Full Text | Google Scholar

Van Kleef, G. A., de Dreu, C. K. W., and Manstead, A. S. R. (2010). An interpersonal approach to emotion in social decision making: the emotions as social information model. Adv. Exp. Soc. Psychol. 42, 45–96. doi: 10.1016/S0065-2601(10)42002-X

CrossRef Full Text | Google Scholar

Wagner, U., Galli, L., Schott, B. H., Wold, A., van der Schalk, J., Manstead, A. S. R., et al. (2015). Beautiful friendship: social sharing of emotions improves subjective feelings and activates the neural reward circuitry. Soc. Cogn. Affect. Neurosci. 10, 801–808. doi: 10.1093/scan/nsu121

PubMed Abstract | CrossRef Full Text | Google Scholar

Weisz, J. D., Kiesler, S., Zhang, H., Ren, Y., Kraut, R. E., and Konstan, J. A. (2007). “Watching together: integrating text chat with video,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, CA: ACM), 877–886.