- 1Faculty of Psychology, Beijing Normal University, Beijing, China
- 2Department of Psychology, University of Wisconsin–Whitewater, Whitewater, WI, United States
- 3Beijing Key Laboratory of Behavior and Mental Health, School of Psychological and Cognitive Sciences, Peking University, Beijing, China
The present study explored how eye contact at different levels of visual awareness influences gaze-induced joint attention. We adopted a spatial-cueing paradigm, in which an averted gaze was used as an uninformative central cue for a joint-attention task. Prior to the onset of the averted-gaze cue, either supraliminal (Experiment 1) or subliminal (Experiment 2) eye contact was presented. The results revealed a larger subsequent gaze-cueing effect following supraliminal eye contact compared to a no-contact condition. In contrast, the gaze-cueing effect was smaller in the subliminal eye-contact condition than in the no-contact condition. These findings suggest that the facilitation effect of eye contact on coordinating social attention depends on visual awareness. Furthermore, subliminal eye contact might have an impact on subsequent social attention processes that differ from supraliminal eye contact. This study highlights the need to further investigate the role of eye contact in implicit social cognition.
Introduction
As a powerful social stimulus, eye contact communicates emotions and social intentions, such as approach-oriented motivations and general interests in the observer (Macrae et al., 2002; Adams and Kleck, 2005; Adams et al., 2006; George and Conty, 2008). Eye contact has been shown to capture attention more readily compared to averted gazes (Senju and Hasegawa, 2005; Vuilleumier et al., 2005; Conty et al., 2006), evoke positive affective responses (Bindemann et al., 2008; Willis et al., 2011; Chen et al., 2017), facilitate face recognition (Vuilleumier et al., 2005), positively affect attractiveness evaluation (Ewing et al., 2010), bias face preference (Jones et al., 2006), trigger self-referential processes (Conty et al., 2016; Hietanen and Hietanen, 2017), and eventually enhance interpersonal synchronization (Patterson, 1982). In most above-mentioned and many other laboratory studies, researchers often operationalize eye contact by presenting a direct gaze from a central face image to an observer. Although eye contact often functions as a signal facilitating social interaction, it can be also perceived as a threatening signal (Ellsworth et al., 1972; Emery, 2000) or has negative connotations. For instance, angry faces were rated less approachable and trustworthy when displaying direct gazes compared to averted gazes (Willis et al., 2011), and angry faces with direct gazes were categorized quicker and their emotion status judged to be more intense than those with averted gazes (Graham and LaBar, 2007; Sander et al., 2007; Bindemann et al., 2008).
Given the importance of eye contact, it is not surprising that eye contact can be processed at a subliminal level to modulate basic cognitive processes. For instance, subliminally presented eye contact facilitates visual awareness compared to averted gaze in interocular suppression, and is more capable of capturing attention as well as attracting eye movement than averted gaze (Stein et al., 2011; Chen and Yeh, 2012; Madipakkam et al., 2015). However, there is less investigation regarding whether and how eye contact modulates social cognition when the eye contact was presented without visual awareness. Given the functional diversity of eye contact, an intriguing question is whether the role of a subliminally presented eye contact would be different from that of a supraliminal eye contact. There has been evidence on different functions of eye contact with and without awareness in social decision making (Luo et al., 2016), but it is not clear whether such a dissociation is specific to that task, or can also be observed in other aspects of social cognition.
We explore this question by focusing on a fundamental aspect of social cognition: gaze-induced joint attention. Gaze-induced joint attention refers to the phenomenon that people automatically follow the averted gaze of others to align their attention (Driver et al., 1999; Nuku and Bekkering, 2008; Xu et al., 2011). As an essential non-verbal channel of interpersonal coordination, gaze-induced joint attention has been shown to emerge as early as in the first 3 months in infancy (Charman et al., 2000), and occurs automatically even if the averted gazes are task-irrelevant (Driver et al., 1999; Friesen et al., 2004; Frischen et al., 2007; Chen et al., 2017) or subliminally presented (e.g., Sato et al., 2007; Xu et al., 2011; Bailey et al., 2014; Mitsuda and Masaki, 2017; but see Al-Janabi and Finkbeiner, 2012 for evidence of the involvment of volitional control). Further, gaze-induced joint attention has been demonstrated to be specifically linked to the social relevance of gaze. It shows features distinct from that of non-social central attentional cues (Friesen et al., 2004), and it is modulated by social context such as the perceived intentionality of the gazer (Wiese et al., 2012, 2014; Wykowska et al., 2014). Also, gaze-induced joint attention recruits brain areas putatively involved in “Theory of Mind” (George and Conty, 2008), such as superior temporal sulcus and intraparietal sulcus (Vuilleumier et al., 2002; Calder et al., 2007) in addition to brain regions typically involved in attention-based orientation.
In the present study, we are interested in how this specific type of social attention might be affected if eye contact was presented prior to the averted gaze cues, and whether awareness of eye contact plays a moderating role. Supraliminal eye contact has been shown facilitating gaze-induced joint attention. Eye contact between two individuals can subsequently generate quicker attention orienting of one person following the gaze direction of the other, compared to when there is no such eye contact (Bristow et al., 2007). Also, gaze-induced joint attention following eye contact is accompanied by increased neural activity in regions critical in understanding communicative intent, such as medial prefrontal cortex (Schilbach et al., 2010; Caruana et al., 2015). Enhanced gaze-following behavior following eye contact is even found among 4–6 month-old infants (Farroni et al., 2003; Senju and Csibra, 2008). Given these findings, one might wonder whether such a potential impact of direct gaze may unconsciously occur, as eye-contact effects are thought to be partially triggered by subcortical regions (Senju and Johnson, 2009), which is less dependent on visual awareness.
To answer this question, we adopted a gaze-cueing paradigm, in which an averted gaze was used as an uninformative cue for a joint-attention task. Prior to the onset of an averted-gaze cue, either supraliminal (Experiment 1) or subliminal (Experiment 2) eye contact may occur. Following the literature (e.g., Kampe et al., 2003; Conty et al., 2006; Hietanen et al., 2008; Stein et al., 2011), we operationalized eye contact as a central face gazing directly at the observer. Experiment 1 extended previous research about eye contact’s facilitation effect on joint attention in that it examined the effect of task-irrelevant eye contact on subsequent joint attention. When Bristow et al. (2007) reported a facilitation effect of eye contact, their task was to judge the congruency of subsequent gaze shift in relation to a target. This task may have encouraged the participants to purposefully process gaze direction. In contrast, the present study rendered both eye contact and averted gaze cue task-irrelevant and therefore uninformative to participants, so as to examine whether and how eye contact affects subsequent joint attention automatically, without being intentionally processed. We expected to find the gaze-cueing effect, i.e., a shorter reaction time on detecting the location of a target following the congruent compared to the incongruent gaze cues. Furthermore, this gaze-cueing effect would be enhanced by the preceding supraliminal eye contact, despite of its task irrelevancy. Experiment 2 investigated the effect of subliminal eye contact on gaze-induced joint attention by rendering eye contact subjectively invisible with the technique of interocular suppression. We hypothesized that joint attention can also be modulated by subliminal eye contact.
Experiment 1: the Effect of Supraliminal Eye Contact
In Experiment 1, participants viewed face stimuli containing averted gaze cues that were task-irrelevant, and reported the location of subsequent target stimuli. Half of the trials presented a face with direct gaze, which established an eye contact with the participant (the eye-contact condition); no such eye contact was established in the other half of the trials (the no-contact condition). Then the face displayed averted gaze, and participants were asked to report as quickly and accurately as possible the appearance of a target and its location, either left or right to the face. The direction of the averted gaze was either toward (in the congruent trials) or away from (in the incongruent trials) the target. The gaze direction was therefore independent of the location of the target and uninformative for participants to complete the task. Thus Experiment 1 adopted a 2 (Prior Eye Contact: Eye-contact vs. No-contact) × 2 (Gaze Congruency: Congruent vs. Incongruent) within-subject factorial design.
Participants
Twenty-two students at Peking University participated in Experiment 1 (15 females and 7 males; 18–25 years, M = 21.8 years). The sample size was decided a priori based on earlier studies examining the behavioral effects of supraliminal eye contact (e.g., Xu et al., 2011). All participants had normal or corrected-to-normal regular and stereo vision.
Written informed consent was obtained from each participant before Experiments 1 (and 2). The experiments were approved by the ethics review committee at the Peking University.
Stimuli
FaceGen (Copyright© 2009, Singular Inversions Inc.)’s Eastern Asian average male face with neutral emotional expression gazing slightly downward was used as the baseline face. It was trimmed to be 3° × 3°, showing the region from the top of the head to the upper neck. All other face stimuli were the same as the baseline face, except for their respective gaze directions. In the eye-contact face, the face image gazed straight ahead to form eye contact with the participants. In the no-contact face, the gaze direction was further down than in the baseline face, changed by the same degree as in the eye-contact condition. Two other face images, one gazing left and one gazing right, were used as the left and right averted gaze cues pointing to the possible locations of the target in each trial. The target was a picture of a toy duck, subtending visual angles of 1.5° × 1.5°, presented 8° left or right to the center of face stimuli. A light gray outer frame (19° × 19°) surrounded the face stimuli and the target throughout each block.
The stimuli were presented by Matlab 2009 with Psychtoolbox 3 on a 19 inch Viewsonic Professional Series P97f+ monitor (1024 × 768, visual angle 44.4° × 36.1°) connected to a Windows XP computer, against a dark gray (40, 40, 40 RGB) background.
Procedure
Participants completed the experiment individually with their heads supported by a chin rest at a viewing distance of 46 cm from the computer screen. They went through a practice block and five formal blocks. The blocks were separated by 1-min rest periods. The formal blocks contained 280 trials in total. Each formal block contained 28 critical trials, 7 catch trials, and 21 filler trials, all in a random order, in addition to four warm-up trials (not analyzed). Each critical trial (Figures 1A,B) started with a 400 ms presentation of a fixation cross. Then the baseline face appeared for 300–900 ms (randomly varied across trials), followed by two intermediate faces being presented for 100 ms each, and a 1700 ms presentation of the third face. In the eye-contact condition, the intermediate faces gazed gradually upward and the third face gazed straight ahead, forming eye contact with the participants. In the no-contact condition, however, the intermediate faces and the third face gazed further downward, making no eye contact with the participants. Then 300 ms before the onset of the target (a picture of a toy duck, 1.5° × 1.5° visual angles), another three faces with gradually averted gazes (left or right) were sequentially presented to create a perception of gaze shift. The target would randomly appear either left or right to the face.
FIGURE 1. The procedures of Experiment 1. (A) An eye-contact + congruent trial (left-oriented gaze cue and left target). (B) A no-contact + incongruent trial (right-oriented gaze cue and left target). (C) A catch trial in which an eye mask was imposed onto the baseline face.
Participants were instructed to indicate the location of the target (left or right) as quickly and accurately as possible, by pressing the “F” or “J” key with their left or right index finger, respectively. They were also informed in advance that the gaze direction was irrelevant to their task, and they should fixate on the central face, pay attention to bilateral areas on the screen, and avoid shifting their gaze during each trial. Both the target and the averted-gaze face image remained on the screen until a response was made or until 2000 ms had elapsed. Participants’ reaction time (RT) and accuracy of judgment were recorded. An incorrect response would prompt a warning message.
Filler trials were included to prevent participants from possible confounding effect on detecting the target, i.e., associating the onsets of the averted gaze and the target. One third of these trials presented an averted gaze cue but no target, and the rest presented a target but no gaze cue.
Catch trials were included to control for a potential confounding factor, i.e., the participants associating the onsets of the averted gaze with the target. These trials did not present any target or gaze cue; rather an eye mask would be imposed onto the baseline face 400 ms after its onset (Figure 1C). The participants were required to press the space key to report “masks,” to ensure that they were constantly paying attention to the eye region of the central face. To prevent guessing, the participants were warned beforehand that if their wrong location judgments or false alarms in the critical trials, or misses in the catch trials exceeded three times in a block, the block would terminate and restart1.
Results and Discussion
The trials with incorrect response, with RTs shorter than 100 ms or outside three interquartiles’ range of each participant’s mean RT, were discarded (1.0% of all trials). A 2 × 2 repeated measures ANOVA was conducted with prior eye contact and gaze congruency as within-subject factors. The results revealed a significant main effect of gaze congruency, F(1,21) = 27.52, p < 0.001, = 0.57, with participants responding quicker in detecting the targets in the congruent condition than in the incongruent condition (MD = 25.7 ms), and a main effect of prior eye contact, F(1,21) = 5.34, p = 0.03, = 0.20, with the responses being quicker in the eye-contact condition than in the no-contact condition (MD = 7.6 ms). More importantly, the interaction between prior eye contact and gaze congruency was significant, F(1,21) = 7.81, p = 0.011, = 0.27. Simple effect analysis revealed that responses in the eye-contact trials were quicker in the congruent condition than in the incongruent condition, F(1,21) = 38.80, p < 0.001, = 0.65, MD = 32.0 ms, and this gaze congruency effect was also observed when no eye contact was presented, F(1,21) = 11.83, p = 0.002, = 0.36, MD = 19.4 ms (Figure 2A). The paired sample t-test was conducted to directly compare the sizes of the gaze-cueing effect (RTincongruent – RTcongruent) in the eye-contact and the no-contact conditions. The gaze-cueing effect was significantly larger when preceded by the eye-contact than the no-contact faces, t(21) = 2.80, p = 0.01, Cohen’s dz = 0.60 (Figure 2B).
FIGURE 2. The results of Experiment 1. (A) Participants’ reaction times in Experiment 1. Reactions were quicker in the congruent than the incongruent trials in both the eye-contact and the no-contact conditions. (B) The gaze-cueing effect following supraliminal eye contact was larger than that in the no-contact condition in Experiment 1. The asterisks mark the statistically significant differences, p < 0.05.
The results of Experiment 1 showed that supraliminal eye contact enhanced the subsequent gaze-induced joint attention compared to the no-contact condition. The enhancement of the gaze-cueing effect is in line with previous literature examining supraliminal eye contact (Bristow et al., 2007; Senju and Csibra, 2008). In Experiment 2 we proceeded to test whether subliminal eye contact would produce the same effect on the subsequent gaze-cueing effect.
Experiment 2: The Effect of Subliminal Eye Contact
Prior research has demonstrated existence of subliminal processing of eye contact (Stein et al., 2011; Chen and Yeh, 2012; Luo et al., 2016; Chen et al., 2017). However, it is not clear whether and how subliminal eye contact affects subsequent joint attention. Using an interocular suppression technique termed Continuous Flash Suppression (CFS, Tsuchiya and Koch, 2005), Experiment 2 presented the first endeavor to investigate in particular the effect of subliminal eye contact on gaze-induced joint attention. In CFS, awareness of a stimulus presented to one eye is suppressed by a stream of noise images presented to the other eye. Compared to other paradigms such as masking and binocular rivalry, a remarkable advantage of CFS is that it induces robust and prolonged suppression, thus allows the suppressed stimuli to remain invisible for up to several seconds.
Using the same 2 × 2 within-subject factorial design (Gaze Congruency: congruent vs. incongruent; Prior Eye Contact: Eye-contact vs. No-contact) as in Experiment 1, Experiment 2 employed the same spatial-cueing paradigm, in which the presence of eye contact varies prior to a non-informative averted gaze cue. Unlike Experiment 1, the eye-contact face and the no-contact face were suppressed from participants’ awareness. The dependent variable was still the reaction time of target detection.
Participants
Forty-six undergraduate students participated in Experiment 2 as paid volunteers. Data from 2 participants were excluded because of incomplete data recording; 12 participants were excluded because they did not pass the awareness test (see section “Procedure”). Data analysis was conducted on a final sample of 32 participants (19 females and 13 males; 18–25 years, M = 21.4 years). All participants had normal or corrected-to-normal regular and stereo vision.
The sample size of Experiment 2 was based on a priori power analysis. The effect size of eye contact on the gaze-cueing effect (Experiment 1) was used as a crude estimation of the same effect in Experiment 2, and a priori power analysis estimated a required sample of 32 participants to reach a power level of 0.9. Data collection was stopped after the number of valid participants reached this target. Although there was no past research on how subliminally presented eye contact affects subsequent gaze-induced joint attention for reference, our sample size was similar to those in previous studies investigating the gaze-cueing effect induced by subliminal averted gaze cues (e.g., Bailey et al., 2014; Mitsuda and Masaki, 2017; but see Al-Janabi and Finkbeiner, 2012 for smaller samples).
Stimuli
The face stimuli and target stimuli used in Experiment 2 were essentially the same as in Experiment 1. The only exception is that the faces used in the experimental session of Experiment 2 were enlarged (7° × 7°), including only the area from the forehead to the lower part of the jaw (Figure 3). This change in the face stimuli was to make the gaze stimuli stand out more so that any effect of eye gaze would likely be captured when the highly salient dynamic noise images were simultaneously presented during interocular suppression.
FIGURE 3. (A) A schematic drawing of the arrangement of the stereoscope and screen in Experiment 2. The stereoscope delivered different stimuli to each eye. The dashed line showed the path of light from the screen to the eyes of a participant. (B) An illustration of a critical trial in the eye-contact + congruent condition in Experiment 2. The target was presented left to the central face.
Noise images were generated for interocular suppression in sets prior to the experiment according to the following specifications: Each set of noise images started with a different Mondrian image containing randomly located squares of various sizes and colors (luminance: 30.8 cd/mm2), and then each subsequent noise image had more squares added to it, again of random colors and sizes and in random locations, to cover 10% of the image area. During the experiment, each set of images were presented in succession at the rate of monitor refreshing (85 Hz, i.e., each image lasted about 12 ms). Our earlier research has confirmed that such dynamic noise patterns generate robust interocular suppression for eye-gaze stimuli (Xu et al., 2011).
The interocular suppression was induced with the use of a stereoscope consisting of a set of mirrors, through which the stimuli were projected to the two eyes of the participant (see Figure 3A for the arrangement of the stereoscope). Specifically, the face stimuli were presented to the suppressed eye (the non-dominant eye of each participant) while the dynamic noise images of the same size presented to the corresponding location of the suppressing eye (the dominant eye of each participant, Figure 3), resulting in the interocular suppression (Tsuchiya and Koch, 2005; Jiang and He, 2006) that made the face stimuli (eye contact or not) invisible.
Procedure
Upon arrival, participants were tested to determine their dominant eye, or randomly assigned one eye as the suppressed eye (vs. the suppressing eye) for those who did not show clear eye dominance. Each participant went through an experimental session flanked by two awareness test sessions, and always viewed the screen through a stereoscope. The viewing distance was the same as Experiment 1.
The experimental session was similar to Experiment 1. Five formal blocks contain 225 trials in total. Each formal block contained five warm-up trials, 36 critical trials, 3 catch trials, and 6 filler trials. The motivation of including the catch trials and the filler trials was the same as in Experiment 1, but the number of these trials were reduced to control the experiment duration and prevent fatigue, which may affect the strength of interocular suppression. In each critical trial, after a 400-ms binocularly presented fixation, the baseline face was presented for 200 – 600 ms (randomly varied across trials) to the suppressed eye of the participant, followed by either the eye-contact or no-contact face presented for 1000 ms, while dynamic noise images were flashed to the suppressing eye. Such a technique of interocular suppression made the face stimuli (the eye-contact face or the no-contact face) invisible (Tsuchiya and Koch, 2005; Jiang and He, 2006). Then, the face images with uninformative averted gaze cues were presented binocularly and thus visible to the participants, followed by the target after 529.4 ms. This presentation resulted in a perception that at the beginning of each trial, only dynamic noise images were presented, followed by the presentation of a face with left or right gaze (i.e., the averted gaze cue) and the target (Figure 3B). The contrast of the stimuli was adjusted individually to obtain reliable suppression effect, and the intermediate face images were omitted to simplify the stimuli presentation. The task was to indicate the location of the target (left or right) by pressing the corresponding keys as quickly and accurately as possible, the same as in Experiment 1. The participants were warned beforehand for the accuracy criterion, which was the same as in Experiment 1.
The awareness test sessions were to ensure that the participants included in data analysis experienced successful interocular suppression. Each awareness test consisted of 80 forced-choice trials, with half of the trials presenting the suppressed face images and the dynamic noise images in the same way as in the experimental session, and the other half replacing the suppressed face images with scrambled face images. The participants guessed at the end of each trial whether an intact face was presented, and if so whether it was an eye-contact face or not. Comparing a participant’s judgments in such a forced-choice task with the chance level has been recognized as a rigorous manipulation check for awareness (Merikle et al., 2001). If the responses deviated from the chance level in either awareness test session (chi-square tests, ps < 0.05), the participant’s data would be excluded from the final analysis.
Results and Discussion
Thirty-two participants out of the 44 tested participants performed at chance level in the awareness tests. We reported the results of these 32 participants. Specifically, the mean accuracy of these participants on guessing whether there was an intact face presented in a trial was 50.7% (SD = 3.63), and the mean accuracy regarding the gazed direction was 53.40% (SD = 6.17)2. For the 32 valid participants, neither the sensitivity indices (d′) of discriminating between intact and scrambled faces (Md′ = -0.10, SD = 0.19) nor the d′ of discriminating between the eye-contact and the no-contact gazes (Md′ = -0.08, SD = 0.27) differed from 0 (binomial tests, ps > 0.10). These results demonstrated effective suppression of eye contact, therefore final analysis was conducted on the data in the experimental session from these participants.
In the experimental session, trials with incorrect response, with RTs shorter than 100 ms or outside three interquartiles’ range of each participant’s mean RT, were discarded (2.9% of all trials). The 2 (Gaze Congruency: congruent vs. incongruent) × 2 (Eye-contact vs. No-contact) within-subject ANOVA revealed a significant main effect of gaze congruency with quicker responses in the congruent condition than in the incongruent condition (MD = 10.9 ms), F(1,31) = 27.42, p < 0.001, = 0.47. The main effect of prior eye contact was not significant, F(1,31) < 1, p = 0.39, = 0.02. Further, the interaction between prior eye contact and gaze congruency was significant, F(1,31) = 12.10, p = 0.002, = 0.28, but with a pattern different from that of Experiment 1. Simple effect analysis revealed that the responses in the congruent trials were quicker than the incongruent trials in the no-contact condition, F(1,31) = 39.30, p < 0.001, = 0.56, MD = 15.73 ms, as well as in the eye-contact condition, F(1,31) = 5.76, p = 0.023, = 0.16, MD = 6.00 ms (Figure 4A). To compare the sizes of the gaze-cueing effect in the eye-contact and the no-contact conditions directly, a paired sample t-test was conducted, which revealed that the gaze-cueing effect was significantly smaller in the eye-contact condition compared to that in the no-contact condition, t(31) = -3.48, p = 0.002, Cohen’s dz = 0.61, different from the pattern observed in Experiment 1 (Figure 4B).
FIGURE 4. The results of Experiment 2. (A) Participants’ reaction times in Experiment 2. Reactions were quicker in the congruent than the incongruent trials in both the eye-contact condition and the no-contact condition. (B) Gaze-cueing effect following subliminal eye contact was smaller than that in the no-contact condition in Experiment 2. The asterisks mark the statistically significant differences, p < 0.05.
Experiment 2 revealed that subliminal eye contact reduced the subsequent gaze-induced joint attention compared to the no-contact condition, an impact opposite to that of supraliminal eye contact (Experiment 1). These distinct patterns may reflect a qualitative difference between the impacts of subliminal and supraliminal eye contact on subsequent joint attention. This result cannot be explained as a lack of processing of subliminal eye contact; otherwise there should not be a difference between the eye-contact and the no-contact conditions in Experiment 2. Second, it is unlikely that participants’ different response patterns between experiments were due to the difference in experimental setups, such as the presence of the dynamic noises in Experiment 2, since the dynamic noise images were similarly presented in the no-contact condition and the eye-contact condition. In addition, similar to Experiment 1, Experiment 2 revealed a significant gaze-cueing effect in the no-contact condition, suggesting sufficient processing of averted-gaze cues in this experiment. Another factor to consider is that Experiment 2 used a SOA different from Experiment 1 to accommodate interocular suppression. Both SOAs are within the range in which stable gaze-induced joint attention has been found (e.g., from 200 to 700 ms. Friesen and Kingstone, 1998; Driver et al., 1999; Frischen et al., 2007), and the longer SOAs may explain the shorter RTs in Experiment 2 compared to those in Experiment 1. Such reduction in average RTs in Experiment 2 may be the result of a standard foreperiod effect (Friesen and Kingstone, 1998; Bertelson, 1967). However, the change in average RTs cannot easily explain the even smaller gaze-cueing effect in the eye-contact condition compared to the no-contact condition in Experiment 2, because this factor is likely to affect both the eye-contact and the no-contact conditions, instead of producing a reversed effect of eye contact in Experiment 2.
General Discussion
The present study reported dissociable effects of supraliminal and subliminal eye contact on gaze-induced joint attention: supraliminal eye contact increased subsequent gaze-cueing effect, but subliminal eye contact reduced it, compared with the corresponding no-contact conditions. The enhancement of the gaze-cueing effect following supraliminal eye contact is in line with previous literature (Bristow et al., 2007; Senju and Csibra, 2008), demonstrating the facilitation role of supraliminal eye contact in the alignment of social attention and interpersonal coordination (Patterson, 1982).
More importantly, Experiment 2 found reduced gaze-cueing effect following subliminally presented eye contact. Notably, the impact of subliminal eye contact was not a mere weakened version of that of supraliminal eye contact, neither did awareness simply amplify the effect of eye contact. Instead, the dissociation is in the directions of subliminal and supraliminal eye-contact effects. One speculation of ours is that subliminal eye contact may have social implications different from supraliminal eye contact. Existing research revealed that eye contact affects behavior in various ways, social facilitation being only one of it. For instance, eye contact can also be perceived as a threatening signal (Ellsworth et al., 1972; Emery, 2000). Subliminal eye contact evoked a larger negative potential at 200 – 300 ms in the partial-frontal area (Yokoyama et al., 2013), similar to subliminally presented fearful faces (Kiss and Eimer, 2008), suggesting that it probably conveys a negative rather than congenial social message. In line with this explanation, Luo et al. (2016) found that subliminally presented eye contact drove the individuals with a proself social value orientation to make less cooperative decisions compared to the no-contact condition, probably due to the ancient alarm reflex being evoked by the threatening implication of subliminal eye contact. Taking these findings, we speculated that the reduced gaze-cueing effect following subliminal eye contact might come from a mechanism similar to delayed attention disengagement induced by various threatening visual stimuli, including threatening words, emotionally threatening pictures, as well as supraliminal and subliminal fearful and angry faces (Fox et al., 2001; Yiend and Mathews, 2001; Carlson and Mujica-Parodi, 2015). In our study, subliminal eye contact might have functioned as a salient warning signal, thus similarly held attention and interfered with subsequent processes (such as the processing of averted gaze cues), and, consequently, reduced the following joint attention effect (the relative advantage of the congruent trials compared to the incongruent trials). If this speculation stands, our results would join existing evidence (Luo et al., 2016) in suggesting that the social meaning of eye contact may be contingent on visual awareness, and that the social-facilitation impact of eye contact may change remarkably in the absence of awareness. From this perspective, such a qualitative difference in the impact of eye contact at different levels of visual awareness may reflect the complexity of social cognition and the rich social implication of stimuli such as eye contact.
How can such awareness contingency of eye-contact effect exist? We speculated that visual awareness modulates the neural correlates of eye-contact processing. According to the fast-track modulator model (Senju and Johnson, 2009), eye contact is jointly processed by a subcortical pathway and a slow, deliberate cortical social brain network that is further subject to factors such as task demands and social contexts. Many brain regions in the cortical network are impacted by visual awareness. For instance, subliminal face stimuli elicited weaker neural activities in the prefrontal regions and the fusiform face area compared to supraliminal stimuli (Leopold and Logothetis, 1996; Tong et al., 1998; Jiang and He, 2006). In contrast, regions in the subcortical pathway have been demonstrated to respond to eye contact even in the absence of visual awareness (Madipakkam et al., 2015; Rothkirch et al., 2015). Thus, the effect of subliminal eye contact may mainly result from basic, reflective processing via the subcortical pathway, while the modulation from sociocultural norms may be weakened due to the limited access to certain cortical regions, as found in implicit social attitudes tasks (Greenwald et al., 1998; Nosek et al., 2002; Kiefer and Sekaquaptewa, 2007). On the other hand, the processing of supraliminal eye contact is subject to the modulation from top-down, context-specific inputs via the cortical network. As a result, different from the prosocial impact of supraliminal eye contact, a more reflexive warning function may manifest when the eye contact was subliminal, leading to a delay of attention disengagement or other types of fear response. In the present study this may lead to the reduction of subsequent gaze following response. This speculation can be further examined by directly testing the affective impact of subliminal eye contact or investigating the neural correlates of subliminal eye contact in the attention networks, the limbic system and other brain regions critically involved in affective processing.
The social account of the subliminal eye contact effect is but one potential explanation. The design of our study does not exclude alternative non-social explanations of our findings. For instance, in Experiment 1, it is possible that the gaze shift from the baseline face to the no-contact face might have distracted attention away from the central face, hence resulted in a smaller gaze cueing effect in this condition. Alternatively, the transitions from the baseline face to the eye-contact face or the no-contact face might be perceived as upward or downward biological motion, respectively, and the downward biological motion might be perceived as relatively subtle and may not facilitate the processing of subsequent averted gaze cue as much as the upward biological motion in the eye-contact condition. The subsequent gaze-cueing effect may have consequently reduced in the no-contact condition compared to the eye-contact condition. Further, in Experiment 2, the absence of visual awareness may compromise the perception of upward biological motion and affect the subsequent gaze-cueing effect. It may also be possible that the dissociations between upward and downward gaze shifts or biological motion are contingent on visual awareness or other aspects of the experimental setting. Exactly whether and how these factors may lead to the change of the direction of eye contact effect requires future research, for instance to investigate the neural activities in the attention networks and the cortical regions critically involved in gaze processing or the processing of biological motion while orthogonally manipulating the presence of gaze cues and visual awareness. Also, it is possible that inhibition of return (IOR) may have started to affect the gaze-cueing effect with the longer SOA in Experiment 2 (529.4 ms) than that in Experiment 1 (300 ms). Since IOR of centrally presented gaze cues was typically reported with much longer SOAs, (e.g., longer than 1 s, Frischen and Tipper, 2004; Frischen et al., 2007; McKee et al., 2007; Marotta et al., 2013; Hudson and Skarratt, 2016), future research is needed to explore whether IOR also affects responses as early as ∼500 ms, and further whether such an IOR effect can be moderated by eye contact and result in the reduced gaze-cueing effect compared to the no-contact condition.
Although there is also a study that failed to find evidence of the dissociation between subliminal and supraliminal eye contact (Chen et al., 2017), we speculate that it may be due to the specific research design with an affective priming task. The functional dissociation between supraliminal and subliminal eye contact may exist in certain, but not all, domains of social cognition. Taken together, research thus far indicates intricacy in the relation between visual awareness and social cognition that involves eye contact, and calls for future investigation to examine the functions of subliminal and supraliminal eye contact in specific contexts before generalizing findings from any single paradigm. Research in this direction might even shed light on the relation between awareness and sociality in general.
Conclusion
Our study extends previous research investigating the potential dissociation between explicit and implicit social cognition into the research of eye-contact processing (Ewing et al., 2010; Luo et al., 2016). Utilizing one of the most basic, yet essential non-verbal form of social cognition, joint attention, we illustrated how the impact of eye contact may be contingent on visual awareness. We speculated that this contingency is because subliminal and supraliminal eye contact has different affective or social implications, a direction future research should continue to explore.
Ethics Statement
This study was carried out in accordance with the recommendations of the Ethics Review Committee of Peking University with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the ethics review committee at the Peking University.
Author Contributions
SX and HG proposed the ideas and designed the experiments. SX performed the experiments and analyzed the data. SX, SZ, and HG wrote the manuscript.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This work was supported by the National Natural Science Foundation of China under grant numbers 31671131, 31271202, and 31600925 and the China Postdoctoral Science Foundation under grant number 2016M590055. We would like to thank the assistance of Ran Tao, Chao Shi, Chenyuan Zhang, Gege Cui, and Xuefei Yuan with data collection.
Footnotes
- ^Two participants in Experiment 1 and four in Experiment 2 had to restart either the practice block or a formal block. No participant reported any difficulty in suppressing the gaze shift in the debriefing.
- ^In contrast to the 32 valid participants, the participants who cannot pass the awareness tests performed well above the chance level with the mean accuracy in guessing intact faces were 86.37%, SD = 13.17, and the mean accuracy in guessing gaze direction was 63%, SD = 12.64.
References
Adams, R. B. Jr., Ambady, N., Macrae, C. N., and Kleck, R. E. (2006). Emotional expressions forecast approach-avoidance behavior. Motiv. Emot. 30, 177–186. doi: 10.1007/s11031-006-9020-2
Adams, R. B. Jr., and Kleck, R. E. (2005). Effects of direct and averted gaze on the perception of facially communicated emotion. Emotion 5, 3–11. doi: 10.1037/1528-3542.5.1.3
Al-Janabi, S., and Finkbeiner, M. (2012). Effective processing of masked eye gaze requires volitional control. Exp. Brain Res. 216, 433–443. doi: 10.1007/s00221-011-2944-0
Bailey, P. E., Slessor, G., Rendell, P. G., Bennetts, R. J., Campbell, A., and Ruffman, T. (2014). Age differences in conscious versus subconscious social perception: the influence of face age and valence on gaze following. Psychol. Aging 29, 491–502. doi: 10.1037/a0036249
Bertelson, P. (1967). The time course of preparation. Q. J. Exp. Psychol. 19, 272–279. doi: 10.1080/14640746708400102
Bindemann, M., Burton, A. M., and Langton, S. R. H. (2008). How do eye gaze and facial expression interact? Vis. Cogn. 16, 708–733. doi: 10.1080/13506280701269318
Bristow, D., Rees, G., and Frith, C. D. (2007). Social interaction modifies neural response to gaze shifts. Soc. Cogn. Affect. Neurosci. 2, 52–61. doi: 10.1093/scan/nsl036
Calder, A. J., Beaver, J. D., Winston, J. S., Dolan, R. J., Jenkins, R., Eger, E., et al. (2007). Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule. Curr. Biol. 17, 20–25. doi: 10.1016/j.cub.2006.10.052
Carlson, J. M., and Mujica-Parodi, L. R. (2015). Facilitated attentional orienting and delayed disengagement to conscious and nonconscious fearful faces. J. Nonverbal Behav. 39, 69–77. doi: 10.1007/s10919-014-0185-1
Caruana, N., Brock, J., and Woolgar, A. (2015). A frontotemporoparietal network common to initiating and responding to joint attention bids. Neuroimage 108, 34–46. doi: 10.1016/j.neuroimage.2014.12.041
Charman, T., Baron-Cohen, S., Swettenham, J., Baird, G., Cox, A., and Drew, A. (2000). Testing joint attention, imitation, and play as infancy precursors to language and theory of mind. Cogn. Dev. 15, 481–498. doi: 10.1016/S0885-2014(01)00037-5
Chen, T., Helminen, T. M., and Hietanen, J. K. (2017). Affect in the eyes: explicit and implicit evaluations. Cogn. Emot. 31, 1070–1082. doi: 10.1080/02699931.2016.1188059
Chen, Y.-C., and Yeh, S.-L. (2012). Look into my eyes and I will see you: unconscious processing of human gaze. Conscious. Cogn. 21, 1703–1710. doi: 10.1016/j.concog.2012.10.001
Conty, L., George, N., and Hietanen, J. K. (2016). Watching eyes effects: when others meet the self. Conscious. Cogn. 45, 184–197. doi: 10.1016/j.concog.2016.08.016
Conty, L., Tijus, C., Hugueville, L., Coelho, E., and George, N. (2006). Searching for asymmetries in the detection of gaze contact versus averted gaze under different head views: a behavioural study. Spat. Vis. 19, 529–545. doi: 10.1163/156856806779194026
Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., and Baron-Cohen, S. (1999). Gaze perception triggers reflexive visuospatial orienting. Vis. Cogn. 6, 509–540. doi: 10.1080/135062899394920
Ellsworth, P. C., Carlsmith, J. M., and Henson, A. (1972). The stare as a stimulus to flight in human subjects: a series of field experiments. J. Pers. Soc. Psychol. 21, 302–311. doi: 10.1037/h0032323
Emery, N. (2000). The eyes have it: the neuroethology, function and evolution of social gaze. Neurosci. Biobehav. Rev. 24, 581–604. doi: 10.1016/S0149-7634(00)00025-7
Ewing, L., Rhodes, G., and Pellicano, E. (2010). Have you got the look? Gaze direction affects judgements of facial attractiveness. Vis. Cogn. 18, 321–330. doi: 10.1080/13506280902965599
Farroni, T., Mansfield, E. M., Lai, C., and Johnson, M. H. (2003). Infants perceiving and acting on the eyes: tests of an evolutionary hypothesis. J. Exp. Child Psychol. 85, 199–212. doi: 10.1016/S0022-0965(03)00022-5
Fox, E., Russo, R., Bowles, R., and Dutton, K. (2001). Do threatening stimuli draw or hold visual attention in subclinical anxiety? J. Exp. Psychol. Gen. 130, 681–700. doi: 10.1037/0096-3445.130.4.681
Friesen, C. K., and Kingstone, A. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychon. Bull. Rev. 5, 490–495. doi: 10.3758/BF03208827
Friesen, C. K., Ristic, J., and Kingstone, A. (2004). Attentional effects of counterpredictive gaze and arrow cues. J. Exp. Psychol. Hum. Percept. Perform. 30, 319–329. doi: 10.1037/0096-1523.30.2.319
Frischen, A., Bayliss, A. P., and Tipper, S. P. (2007). Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychol. Bull. 133, 694–724. doi: 10.1037/0033-2909.133.4.694
Frischen, A., and Tipper, S. P. (2004). Orienting attention via observed gaze shift evokes longer term inhibitory effects: implications for social interactions, attention, and memory. J. Exp. Psychol. Gen. 133, 516–533. doi: 10.1037/0096-3445.133.4.516
George, N., and Conty, L. (2008). Facing the gaze of others. Neurophysiol. Clin. 38, 197–207. doi: 10.1016/j.neucli.2008.03.001
Graham, R., and LaBar, K. S. (2007). Garner interference reveals dependencies between emotional expression and gaze in face perception. Emotion 7, 296–313. doi: 10.1037/1528-3542.7.2.296
Greenwald, A. G., McGhee, D. E., and Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: the implicit association test. J. Pers. Soc. Psychol. 74, 1464–1480. doi: 10.1037/0022-3514.74.6.1464
Hietanen, J. K., Leppanen, J. M., Nummenmaa, L., and Astikainen, P. (2008). Visuospatial attention shifts by gaze and arrow cues: an ERP study. Brain Res. 1215, 123–136. doi: 10.1016/j.brainres.2008.03.091
Hietanen, J. O., and Hietanen, J. K. (2017). Genuine eye contact elicits self-referential processing. Conscious. Cogn. 51, 100–115. doi: 10.1016/j.concog.2017.01.019
Hudson, M., and Skarratt, P. A. (2016). Peripheral cues and gaze direction jointly focus attention and inhibition of return. Cogn. Neurosci. 7, 67–73. doi: 10.1080/17588928.2015.1053444
Jiang, Y., and He, S. (2006). Cortical responses to invisible faces: dissociating subsystems for facial-information processing. Curr. Biol. 16, 2023–2029. doi: 10.1016/j.cub.2006.08.084
Jones, B. C., Debruine, L. M., Little, A. C., Conway, C. A., and Feinberg, D. R. (2006). Integrating gaze direction and expression in preferences for attractive faces. Psychol. Sci. 17, 588–591. doi: 10.1111/j.1467-9280.2006.01749.x
Kampe, K. K., Frith, C. D., and Frith, U. (2003). “Hey John”: signals conveying communicative intention toward the self activate brain regions associated with “mentalizing,” regardless of modality. J. Neurosci. 23, 5258–5263.
Kiefer, A. K., and Sekaquaptewa, D. (2007). Implicit stereotypes, gender identification, and math-related outcomes. Psychol. Sci. 18, 13–18. doi: 10.1111/j.1467-9280.2007.01841.x
Kiss, M., and Eimer, M. (2008). ERPs reveal subliminal processing of fearful faces. Psychophysiology 45, 318–326. doi: 10.1111/j.1469-8986.2007.00634.x
Leopold, D. A., and Logothetis, N. K. (1996). Activity changes in early visual cortex reflect monkeys’ percepts during binocular rivalry. Nature 379, 549–553. doi: 10.1038/379549a0
Luo, Y., Zhang, S., Tao, R., and Geng, H. (2016). The power of subliminal and supraliminal eye contact on social decision making: an individual-difference perspective. Conscious. Cogn. 40, 131–140. doi: 10.1016/j.concog.2016.01.001
Macrae, C. N., Hood, B. M., Milne, A. B., Rowe, A. C., and Mason, M. F. (2002). Are you looking at me? Eye gaze and person perception. Psychol. Sci. 13, 460–464. doi: 10.1111/1467-9280.00481
Madipakkam, A. R., Rothkirch, M., Guggenmos, M., Heinz, A., and Sterzer, P. (2015). Gaze direction modulates the relation between neural responses to faces and visual awareness. J. Neurosci. 35, 13287–13299. doi: 10.1523/jneurosci.0815-15.2015
Marotta, A., Pasini, A., Ruggiero, S., Maccari, L., Rosa, C., Lupiáñez, J., et al. (2013). Inhibition of return in response to eye gaze and peripheral cues in young people with Asperger’s syndrome. J. Autism Dev. Disord. 43, 917–923. doi: 10.1007/s10803-012-1636-3
McKee, D., Christie, J., and Klein, R. (2007). On the uniqueness of attentional capture by uninformative gaze cues: facilitation interacts with the Simon effect and is rarely followed by IOR. Can. J. Exp. Psychol. 61, 293–303. doi: 10.1037/cjep2007029
Merikle, P. M., Smilek, D., and Eastwood, J. D. (2001). Perception without awareness: perspectives from cognitive psychology. Cognition 79, 115–134. doi: 10.1016/S0010-0277(00)00126-8
Mitsuda, T., and Masaki, S. (2017). Subliminal gaze cues increase preference levels for items in the gaze direction. Cogn. Emot. doi: 10.1080/02699931.2017.1371002 [Epub ahead of print].
Nosek, B. A., Banaji, M. R., and Greenwald, A. G. (2002). Math = male, me = female, therefore math≠ me. J. Pers. Soc. Psychol. 83, 44–59. doi: 10.1037/0022-3514.83.1.44
Nuku, P., and Bekkering, H. (2008). Joint attention: inferring what others perceive (and don’t perceive). Conscious. Cogn. 17, 339–349. doi: 10.1016/j.concog.2007.06.014
Patterson, M. L. (1982). A sequential functional model of nonverbal exchange. Psychol. Rev. 89, 231–249. doi: 10.1037/0033-295X.89.3.231
Rothkirch, M., Madipakkam, A. R., Rehn, E., and Sterzer, P. (2015). Making eye contact without awareness. Cognition 143, 108–114. doi: 10.1016/j.cognition.2015.06.012
Sander, D., Grandjean, D., Kaiser, S., Wehrle, T., and Scherer, K. R. (2007). Interaction effects of perceived gaze direction and dynamic facial expression: evidence for appraisal theories of emotion. Eur. J. Cogn. Psychol. 19, 470–480. doi: 10.1080/09541440600757426
Sato, W., Okada, T., and Toichi, M. (2007). Attentional shift by gaze is triggered without awareness. Exp. Brain Res. 183, 87–94. doi: 10.1007/s00221-007-1025-x
Schilbach, L., Wilms, M., Eickhoff, S. B., Romanzetti, S., Tepest, R., Bente, G., et al. (2010). Minds made for sharing: initiating joint attention recruits reward-related neurocircuitry. J. Cogn. Neurosci. 22, 2702–2715. doi: 10.1162/jocn.2009.21401
Senju, A., and Csibra, G. (2008). Gaze following in human infants depends on communicative signals. Curr. Biol. 18, 668–671. doi: 10.1016/j.cub.2008.03.059
Senju, A., and Hasegawa, T. (2005). Direct gaze captures visuospatial attention. Vis. Cogn. 12, 127–144. doi: 10.1080/13506280444000157
Senju, A., and Johnson, M. H. (2009). The eye contact effect: mechanisms and development. Trends Cogn. Sci. 13, 127–134. doi: 10.1016/j.tics.2008.11.009
Stein, T., Senju, A., Peelen, M. V., and Sterzer, P. (2011). Eye contact facilitates awareness of faces during interocular suppression. Cognition 119, 307–311. doi: 10.1016/j.cognition.2011.01.008
Tong, F., Nakayama, K., Vaughan, J. T., and Kanwisher, N. (1998). Binocular rivalry and visual awareness in human extrastriate cortex. Neuron 21, 753–759. doi: 10.1016/S0896-6273(00)80592-9
Tsuchiya, N., and Koch, C. (2005). Continuous flash suppression reduces negative afterimages. Nat. Neurosci. 8, 1096–1101. doi: 10.1038/nn1500
Vuilleumier, P., Armony, J. L., Clarke, K., Husain, M., Driver, J., and Dolan, R. J. (2002). Neural response to emotional faces with and without awareness: event-related fMRI in a parietal patient with visual extinction and spatial neglect. Neuropsychologia 40, 2156–2166. doi: 10.1016/S0028-3932(02)00045-3
Vuilleumier, P., George, N., Lister, V., Armony, J., and Driver, J. (2005). Effects of perceived mutual gaze and gender on face processing and recognition memory. Vis. Cogn. 12, 85–101. doi: 10.1037/a0030348
Wiese, E., Wykowska, A., and Müller, H. J. (2014). What we observe is biased by what other people tell us: beliefs about the reliability of gaze behavior modulate attentional orienting to gaze cues. PLOS ONE 9:e94529. doi: 10.1371/journal.pone.0094529
Wiese, E., Wykowska, A., Zwickel, J., and Müller, H. J. (2012). I see what you mean: how attentional selection is shaped by ascribing intentions to others. PLOS ONE 7:e45391. doi: 10.1371/journal.pone.0045391
Willis, M. L., Palermo, R., and Burke, D. (2011). Social judgments are influenced by both facial expression and direction of eye gaze. Soc. Cogn. 29, 415–429. doi: 10.1521/soco.2011.29.4.415
Wykowska, A., Wiese, E., Prosser, A., and Müller, H. J. (2014). Beliefs about the minds of others influence how we process sensory information. PLOS ONE 9:e94339. doi: 10.1371/journal.pone.0094339
Xu, S., Zhang, S., and Geng, H. (2011). Gaze-induced joint attention persists under high perceptual load and does not depend on awareness. Vision Res. 51, 2048–2056. doi: 10.1016/j.visres.2011.07.023
Yiend, J., and Mathews, A. (2001). Anxiety and attention to threatening pictures. Q. J. Exp. Psychol. A 54, 665–681. doi: 10.1080/713755991
Keywords: eye contact, joint attention, gaze processing, implicit social cognition, visual awareness
Citation: Xu S, Zhang S and Geng H (2018) The Effect of Eye Contact Is Contingent on Visual Awareness. Front. Psychol. 9:93. doi: 10.3389/fpsyg.2018.00093
Received: 21 September 2017; Accepted: 22 January 2018;
Published: 07 February 2018.
Edited by:
Sebastian Loth, Bielefeld University, GermanyReviewed by:
Anna Pecchinenda, Sapienza Università di Roma, ItalyKohske Takahashi, The University of Tokyo, Japan
Copyright © 2018 Xu, Zhang and Geng. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Haiyan Geng, aHlnZW5nQHBrdS5lZHUuY24=