Islamic headdress influences how emotion is recognized from the eyes

Previous research has shown a negative bias in the perception of whole facial expressions from out-group members. Whether or not emotion recognition from the eyes is already sensitive to contextual information is presently a matter of debate. In three experiments we tested whether emotions can be recognized when just the eyes are visible and whether this recognition is affected by context cues, such as various Islamic headdresses vs. a cap or a scarf. Our results indicate that fear is still well recognized from a briefly flashed (100 ms) image of a women wearing a burqa with less than 20% transparency of the eye region. Moreover, the type of headdress influences how emotions are recognized. In a group of participants from non-Islamic background, fear was recognized better from women wearing a niqāb than from women wearing a cap and a shawl, whereas the opposite was observed for happy and sad expressions. The response patterns showed that fearful and anger labels were more often attributed to women with a niqāb vs. a cap and a shawl and again, an opposite pattern was observed for the happy response. However, there was no general response bias: both correct and incorrect responses were influenced by the facial expression as well. Anxiety levels and/or explicit negative associations with the Islam as measured via questionnaires did not mediate the effects. Consistent with the face literature, we conclude that the recognition of emotions from the eyes is also influenced by context.

Introduction

Seeking a stable and predictable world is a basic social need (Hechter and Horne, 2003). From this viewpoint, it is not difficult to imagine how a person or group can form a threat by being different. History has shown that especially in times of economic uncertainty, people seek support by their kin and kith (henceforth in-group; Fiske, 2002), whereas fear toward the out-group, i.e., people who are different, increases. Populist politicians use the public fear in their favor, fueling anxiety, and distrust with heated discussions targeting the out-group, such as for example is the case in the Islamic headscarf discussion. Proponents of the prohibition of headscarfs argue that a headscarf blocks communication and that emotions and intentions cannot be recognized. The aim of this paper is to challenge this view. We hypothesize that emotions can be read from a face (partly) covered by a veil and that misrecognitions follow from prejudice and negative associations with the Islam rather than from pure coverage.

People regularly see individuals of different backgrounds, religion, and cultures. Often, however, true interactions are avoided due to uncomfortable feelings, misunderstandings, and uncertainty about proper codes of conduct for interacting (Amir, 1969). Indeed, previous research has shown that emotional expressions are more accurately recognized from in-group members than from out-group members (van der Schalk et al., 2011); which are more often interpreted negatively (Elfenbein and Ambady, 2002). For example, emotion recognition is typically faster for positive than negative emotions, but the reverse holds when judging out-group members (Hugenberg and Bodenhausen, 2003). Weisbuch and Ambady (2008) reported that negative automatic responses were activated in response to out-group expressions of joy and in-group expressions of fear (Weisbuch and Ambady, 2008). Moreover, van der Schalk et al. (2011) showed that perceivers are more likely to mimic the negative emotions of in-group members than of out-group members (van der Schalk et al., 2011). Despite this general pattern to value emotions from in-groups better than from out-groups, large individual differences exist. Prejudice, for example, has shown to strengthen this bias (Hugenberg and Bodenhausen, 2004) and the level of intergroup anxiety amplifies individuals’ threat appraisal, anger, and offensive action tendencies toward the out-group (Van Zomeren et al., 2007).

Intergroup anxiety may especially be elicited in response to clear overt signs of being different. Muslim women express their religious identity by wearing a headscarf (Moors and Salih, 2009) and may thereby trigger negative stereotypes in Western individuals (Fischer et al., 2011). However, in some cases, large parts of the face become invisible, which may also have effects on emotion inferences. From the literature about face-processing, it is known that facial expressions are processed as an integrated whole, rather than that information from individual features is accessed serially (de Gelder et al., 1997; White, 1999). Therefore, a headscarf may hamper holistic face-processing.

The Bubbles paradigm has provided new insights with regard to which image locations and spatial frequencies are critical for discriminating between stimuli (Gosselin and Schyns, 2001). This paradigm has been widely applied in the field of face perception and has demonstrated the importance of the eye region in affective communication (Caldara et al., 2005; Spezio et al., 2007; Langner et al., 2009). Emotion-driven complex musculature changes, such as the raising and lowering of eyelids and eyebrows enables perceivers to decode emotions from just the eye region (Baron-Cohen et al., 1997). By using chimeric faces that consisted of neutral eyes combined with a fearful mouth or fearful eyes above a neutral mouth, Morris et al. (2002) showed that participants’ emotion recognition was better when the eyes vs. the mouth expressed fear. Moreover, they showed that fearful eyes alone are sufficient to evoke increased neural responses in human amygdala (Morris et al., 2002). These neuroimaging data accord with behavioral findings in both monkeys (Nahm et al., 1997; Emery, 2000) and humans (de Bonis et al., 1999; Eisenbarth and Alpers, 2011) that eyes have a critical role in conveying fear in facial expressions. We may therefore expect that information from the eyes would be sufficient to recognize emotions. If this is a robust phenomenon, it would resist influence from a surrounding headscarf. The question is therefore whether the expression of the eyes is sensitive to context factors (helmet, medical mask, beard, or a hat, etc.) or not.

Recent evidence suggests that as far as the whole face is concerned, the perception of expressions is influenced by context more than had previously been assumed. In fact, under certain conditions, the presence of a task-irrelevant naturalistic scene can dramatically shift the emotional category recognized in basic facial expressions and bias the valence judgment of facial expressions toward this information (Meeren et al., 2005; Righart and de Gelder, 2006, 2008; Tamietto et al., 2006; Van den Stock et al., 2007; Aviezer et al., 2008, 2009, 2011; Koji and Fernandes, 2010). A recent study also explored the influence of context on the perception of emotions from the eyes (Fischer et al., 2011). In that study, the visibility of expressive cues was manipulated by showing films displaying female targets whose faces were either fully visible, covered by a niqāb¹, or partially occluded by two black bars. Emotion perception was affected by an absence of expressive cues, but there was no difference between the niqāb and black bar condition. However, the niqāb condition was compared with a somewhat unnatural condition (black bars) and the two conditions differed in the size of the face part that was visible, which could have been controlled better, had static images been used. Therefore, it may be premature to rule out the idea that context plays a role in the perception of emotions from the eyes only.

In three experiments we investigated if different headdresses influence emotion recognition from the eyes. In Experiment 1, we tested to what extent emotions can be read from the eyes when an Islamic veil covers the rest of the face. In Experiment 2, we compared two Islamic headdresses, the hijāb and niqāb, with two in-group headdresses, a cap and a cap with scarf. Importantly, the exact size of the visible face area was carefully controlled. In Experiment 3, we aimed to replicate Experiment 2, but increased task difficulty by presenting the stimuli 40 ms instead of 100 ms. In addition, correlations between “negative attitudes toward the Islam” and negative biases were investigated.

Experiment 1

The goal of Experiment 1 was to test to what extent emotions can be read from the eyes when a veil covered all other parts.

Methods

Participants

Twenty-five participants with no neurological or psychiatric history and with normal or corrected-to-normal vision (seven male; mean age: 20 years, range 18–25 years) participated. To avoid interactions between cultural background of the participant and stimulus, two Muslim participants were excluded from analyses. All other participants were Caucasian. The experimental procedures were in accordance with the Helsinki Declaration and approved by Tilburg University.

Research design

The study had a 4 within-subjects (face: full face, niqāb, burqa with 80% noise on eye region, burqa with 90% noise on eye region) × 4 within-subjects (emotion: happy, anger, fear, sadness) design.

Stimulus materials

We used six female identities showing happy, angry, sad, and fearful expressions, recognized above 80%, from a well-validated face set (www.macbrain.org/resources.htm). Of the six female actors who were included, three had brown eyes and three green/blue eyes. The stimuli were included in Experiment 1–3 after they had been validated thoroughly.

Validation study 1. The MacBrain set was chosen not only because it is an often used face set, but also because to us, the actors have a quite ambiguous cultural background. The uncovered neutral faces of the six identities were shown to 12 independent raters who were asked which country they thought these women came from. Indeed, as expected, a large percentage of the answers (41%) included non-western countries including Peru, Mexico, Syria, Armenia, Turkey, Pakistan, Bulgaria, Hungary, Bosnia, and Herzegovina.

Validation study 2. Different types of headdresses were combined with the facial expression using Adobe Photoshop CS5. For the stimuli used in Experiment 1, these headdresses included a hijāb where the whole face was visible, a niqāb where only the eyes were visible, a niqāb with 80 or 90% random noise on the eye region, i.e., a burqa (see Figure 1). The headdresses of the stimuli used in Experiment 2 included a cap, a hijāb, a cap with scarf and a niqāb (see Table 1). Stimuli measured 16.4° × 12.0° visual angle. The newly created stimuli were used in a second validation study in which the stimuli of Experiment 2–3 were also included. Twenty-seven participants of Tilburg University that did not participate in any of the experiments described in this manuscript, watched the randomly presented stimuli for a maximum of 5 s and made a choice among an angry, happy, fearful, or sad label. See Table 1 for accuracy rates.

FIGURE 1

Figure 1. Perception of emotion from the eye region in a cultural context. The eye region was presented in four different contexts: (1) as part of a whole face (which is not shown in this figure but is depicted in Table 1), (2) in the context of a niqāb (a headscarf which only leaves the eyes visible), (3) in the context of a burqa where 80% of this region consisted of random noise, and (4) a burqa with 90% random noise.

TABLE 1

Table 1. Stimulus examples and their validation.

Procedure

In our previous experiments in which we investigated the influence of context on the perception of facial or bodily expressions of emotion, strong effects were observed when the stimuli were presented very briefly (100 ms; Kret and de Gelder, 2010). For this reason, we again use a presentation duration of 100 ms. The presentation duration was checked with a light meter and was stable. A trial started with a white fixation cross (positioned at the height of the actors nose) on a gray screen (300 ms), a stimulus (100 ms), followed by a gray screen shown until a response had been made (which usually occurred between 700 and 1200 ms). Stimuli were randomly presented on a CRT screen with a 100-Hz refresh rate.

Participants were seated at a table in a dimly lit sound reduced booth. Distance to the computer screen was 60 cm. Instructions were given verbally and via an instruction screen. Participants were given a forced-choice categorization task using four emotions and were instructed to respond as accurately and rapidly as possible, to use their index and middle fingers and not to change the position of their fingers on the response-box during the experiment. As in the study of Fischer et al. (2011), we opted for a forced-choice approach. Our goal was to keep the task as simple as possible without giving participants multiple response alternatives that would slow down their reaction and replace the initial (expected) fear response by a more cognitively controlled response. Participants were told that the stimuli consisted of female faces with different emotions. No reference was made to religion.

Data analysis

Recognition performance as a factor of emotion and type of headdress depends on both the reaction time and error data. To evaluate recognition performance accounting for speed–accuracy trade-offs and shifts in criterion, inverse efficiency scores (IES: mean reaction times divided by proportion correct responses; Townsend and Ashby, 1983) were analyzed in a 4 × 4 ANOVA [4 emotions, 4 face parts (whole face, niqāb, burqa 80%, burqa 90%)]. Incorrect responses and responses <200 or >2500 ms were discarded from the reaction times. Interactions were followed up by Bonferroni-corrected two-tailed t-tests (12 pairs were tested: hijāb > niqāb; niqāb > burqa 20%; burqa 20% > burqa 10% for the four emotion conditions). Since the presence of an interaction may influence the interpretation of main effects, main effects are not reported if there was a significant orthogonal interaction (Moore et al., 2004).

Results

Face part and emotion were interacting [F(9, 216) = 7.140, p < 0.001, ${\eta }_{\text{p}}^2=0.23$]. In contrast to sad and happy faces, which were better recognized when the full face was visible (in the hijāb condition) than when only the eyes were visible, fear, and anger were equally well recognized in the hijāb as in the niqāb condition (p > 0.1). Recognition of fear did not differ between the two burqa conditions (p > 0.1) and even when only 10% of the eyes was visible, accuracy rates were still well above chance [t(24) = 6.730, p < 0.001]. See Figure 2.

FIGURE 2

Figure 2. Experiment 1. Inverse efficiency scores for the recognition of facial expressions in an Islamic context. Fearful and angry faces were equally well recognized in the hijāb as the niqāb condition. Fear was still recognized well in the burqa conditions and recognition did not differ between the two noise levels. Sadness was recognized poorly from just the eye region, especially when random noise was added.

Discussion

Angry and fearful faces were not recognized better when the whole face vs. only the eyes were visible but sadness and happiness were hard to recognize from the eyes only. Although most emotions could be recognized in the niqāb condition, recognition in the burqa conditions was severely impaired, especially for sadness. Recognition of fear from the eyes in the context of a burqa was still high. This may indicate that the participants more readily associated fearful expressions with women wearing Islamic headdresses. Alternatively, visibility of the eye-white may have played a facilitating role, as earlier studies already suggested (Whalen et al., 2004). To test this explanation, in the next experiment, we compare the effect of Islamic headdresses on emotion perception with the effect of in-group headdresses (caps and scarfs) and kept the size of uncovered face parts constant.

Experiment 2

To investigate whether the perception of emotions from the eye region is sensitive to contextual factors, effects of in- and out-group headdresses were directly compared.

Methods

Participants

Twenty-eight participants (five male; mean age: 20 years, range 18–25 years), all Caucasian and of Dutch origin, took part in the experiment.

Nine participants were Christian and 19 indicated to be non-religious. On average, they indicated to have had 2.25 (SD 1.60) children with an Islamic background in their class at school. Their overall experience with these children was moderately positive 3.00 (SD 1.72) on a scale from 1 (positive) to 7 (negative). At the time of testing, they did not have that much contact with people with an Islamic background 2.11 (SD 1.37) on a scale from 1 (not at all) to 7 (very much).

Research design

The design of the study is shown in Table 2 and includes an “emotion” condition (with the same four emotions as in Experiment 1 and in addition one blurred face), a “face part” condition (full face vs. eyes only were visible), and an in/out-group condition (Islamic or non-Islamic headdress).

TABLE 2

Table 2. Research design.

Stimulus materials

The same facial expressions were used as in Experiment 1. Four different headdresses were used; a hijāb and a niqāb, representing out-group headdresses; a fleece cap and a fleece cap and knitted scarf, representing in-group headdresses. To make the stimuli more realistic, the headdresses were taken from photographs found on the Internet and the cap and scarf, while being worn by a person, were photographed by the first author. These pictures were edited using Adobe Photoshop CS5. The original faces were erased and the cap and scarf were repositioned so that the distance from the eye to the top, bottom, and side border of the cap and scarf were equal to the niqāb condition. The space beside the outer corners of the eyes was kept constant². Stimuli were turned to grayscale to make them less culturally definable by skin color. Two conditions were included in which a niqāb and a cap and scarf were shown, but the eye region was blurred to specifically test the emotions that participants associate with the headdresses.

Procedure

The task and procedure was the same as in Experiment 1. Participants had to define the emotional expression as fast and accurately as possible using the response-box. They were told that the pictures would be presented in a brief flash and that in some cases information was missing and it would be difficult to see the expression. They were told that even though they did not see any expression, they should still follow their first impression.

Data analysis

There were no correct or incorrect answers when linking emotion labels to the headgear templates (Figure 3). Therefore, pure responses to these stimuli were analyzed in four separate t-tests. For each emotion label, we tested whether it was chosen more often after seeing a niqāb vs. a cap and scarf template.

FIGURE 3

Figure 3. Proportion of answers that were given to the headgear templates. A cap and scarf were more often associated with happiness than a niqāb.

Like in Experiment 1, IES were calculated as indicator of recognition performance of the facial expressions in the context of the different headgear templates. These scores were analyzed in an ANOVA with four emotions, two cultural contexts (out-group vs. in-group headdress), and two face parts (whole face vs. eyes only).

Results

Emotions associated with headgear templates

A cap and scarf were more often associated with happiness than the niqāb [t(27) = 2.57, p < 0.05]. An opposite, non-significant tendency was observed for fear. See Figure 3.

Recognition performance

The IES showed an interaction between face part and emotion [F(3, 81) = 13.22, p < 0.001, ${\eta }_{\text{p}}^2=0.33$]. In contrast to the other emotions, fear was equally well recognized from the eyes only as from the whole face (p > 0.1). There was no three-way interaction. However, since we had clear predictions based on a pilot version of this experiment (see the earlier footnote) that fear would be better recognized from out-group members and happiness from in-group members, we split the main ANOVA into separate two-way ANOVAs per emotion condition. When looking at the emotions sadness and anger, only main effects for coverage were observed [F(1, 27) = 18.785, p < 0.001, ${\eta }_{\text{p}}^2=0.41$; F(1, 27) = 23.599, p < 0.001, ${\eta }_{\text{p}}^2=0.47$]. However, when only happy images were included in the analysis, apart from an effect of coverage [F(1, 27) = 20.675, p < 0.001, ${\eta }_{\text{p}}^2=0.43$], we also observed an interaction between culture and coverage [F(1, 27) = 6.995, p < 0.05, ${\eta }_{\text{p}}^2=0.21$] showing that happiness was better recognized from women wearing a cap and a shawl than from women wearing a niqāb [t(27) = 2.55, p < 0.01]. There was no difference between the hijāb and cap conditions, possibly because performance here was often at ceiling. When looking at the results of the fear-ANOVA, a main effect for culture was observed [F(1, 27) = 3.896, p = 0.059, ${\eta }_{\text{p}}^2=0.13$] showing that this emotion was better recognized from out-group members. We conducted two planned comparisons and compared the recognition performance for fear from women wearing the niqāb vs. a cap and shawl and from women wearing the hijāb vs. a cap. The first comparison was numerically consistent with our predictions, but did not reach significance [t(27) = 0.88, p = 0.19], the second was significant [t(27) = 2.03, p < 0.05]. See Figure 4.

FIGURE 4

Figure 4. **p < 0.001. All t-tests were two-tailed. (A) Inverse efficiency scores. The lower the number, the better the performance. Happiness was better recognized from women wearing a cap and a shawl than from women wearing a niqāb. Fear was slightly better recognized from women wearing out-group rather than in-group headdresses, irrespective of weather this was a niqāb or hijāb. (B) The graph shows the same data but difference scores are calculated of “in-group minus out-group” for inverse efficiency scores. The upper part of the graph shows conditions where the out-group was better recognized; the lower part where the in-group was better recognized.

Discussion

The results of Experiment 2 show two important findings. First, a niqāb was less often associated with happiness than a cap and scarf. So, a first confrontation with a niqāb and with the person who wears it may be more negative from the start and may influence how the wearer is seen. Indeed, the second important finding is that a niqāb does hide happiness: this emotion was better recognized from faces with a cap and a scarf than a niqāb even though the exact same face parts were covered or visible. This suggests that a niqāb is less associated with happiness and more associated with fear. If so, people who have strong negative associations with the Islam, or who are anxious in general, might show even stronger biases. This will be tested in the next experiment. The high ceiling effects in this experiment may have weakened the effects. To be able to analyze the response patterns in more detail, in the next experiment, we reduced exposure times.

Experiment 3

The aim of Experiment 3 was first, to relate recognition performance to participants’ attitudes about the Islam and generalized anxiety levels. A second goal was to replicate the findings of the previous experiment. We kept the experiment very similar to Experiment 2, but made a few modifications. Because of the high performance rates of the participants, the task was made more difficult by presenting the stimuli shorter (40 ms instead of 100 ms). Since we had no reason to expect any differences in the interpretation of the headgear templates after decreasing exposure time, we did not include the templates again in the current experiment.

Methods

Participants

Twenty-eight participants, all Caucasian, took part in the experiment (six male; mean age: 21 years old, range 19–36 years old).

Twelve participants were Christian and 14 indicated to be non-religious. On average, they indicated to have had 2.36 (SD 3.78) children with an Islamic background in their class at elementary school and 2.00 (SD 3.78) at high school. Their overall experience with these children was quite positive 2.36 (SD 1.22) on a scale from 1 (positive) to 7 (negative). At the time of testing, they did not have that much contact with people with an Islamic background 2.75 (SD 1.90) on a scale from 1 (not at all) to 7 (very much). The attitudes they held toward the Islam are summarized in Table 3. The full questionnaire, including the average responses can be found in Appendix.

TABLE 3

Table 3. Attitudes toward women wearing Islamic clothing and toward the Islam in general.

Research design

The study had a 2 within-subjects (in/out-group: Islamic, non-Islamic) × 2 within-subjects (face part: full face, eyes only) × 4 within-subjects (emotion: happy, anger, fear, sadness) design.

Stimulus materials

The stimuli were the same as the previous experiment, except that the headgear templates without facial expressions were not included.

Other measures

We created a questionnaire measuring “negative attitudes toward the Islam,” consisting of two parts. The first part is an adapted version of Stephan et al.’s (2002) prejudice scale (Stephan et al., 2002), measuring negative attitudes toward the homeless (12 items, α = 0.85), realistic threat (4 items, α = 0.70), symbolic threat (6 items, α = 0.78), and intergroup anxiety (6 items, α = 0.91). We changed “toward the homeless” into “toward women wearing a hijāb/niqāb.” Furthermore, we measured frequency of negative contact (six items, α = 0.82) to validate its relationship with intergroup anxiety (Stephan and Stephan, 1989). The second part is an adaptation of the modern racism prejudice questionnaire (Ekehammar et al., 2000), measuring attitudes toward the Islam in general. The participants of Experiment 2 filled out the questionnaire. Items with a corrected item-total correlation below 0.4 were removed. The final questionnaire can be found in Appendix. The two parts of the questionnaire correlated (r = 0.81, p < 0.001). Cronbach alpha for the complete questionnaire was 0.93. The total score of the negative attitudes toward the Islam-questionnaire did not correlate with the STAI.

Procedure

The procedure was the same as in the previous experiment with three exceptions. First, because in Experiment 2, some participants performed at ceiling, task difficulty was increased by reducing stimulus presentation duration to 40 ms. Second, we only included emotional faces with different headdresses, not the templates without a face. Third, after the experiment, participants filled out a questionnaire about their attitudes toward the Islam. Generalized anxiety was measured with the STAI (Spielberger, 1983). We provided the questionnaires after the experiment for two reasons. First, prior to the experiment, we did not tell participants that they would see Muslim and non-Muslim women. We kept the procedure as neutral as possible by saying that they would see female facial expressions whose faces were sometimes occluded. This way, we avoided that answering the questionnaire would influence the results of the main experiment. Second, by giving the attitudes questionnaire after the experiment, participants were more likely to have similar exemplars in mind when answering the questions.

Data analysis

Inverse efficiency scores were calculated as indicator of recognition performance of the facial expressions in the context of the different headgear templates. These scores were analyzed in an ANOVA with four emotions, two cultural contexts (out-group vs. in-group headdress), two face parts (whole face vs. eyes only).

In a 4 × 2 × 2 ANOVA, we investigated the likelihood of participants choosing a particular emotion given the presence of the different types of headgear. We were specifically interested whether participants would more often label a woman as experiencing fear when she wears a niqāb vs. a cap and a shawl. We expected an opposite response pattern for the emotion happiness.

Pearson correlations were calculated between scores on the two questionnaires and (1) difference IES of the fearful niqāb minus the fearful cap and shawl condition; (2) difference IES of the happy cap and shawl minus happy niqāb condition.

Results

Performance

There was an interaction between face part, emotion, and in/out-group headdress [F(3, 81) = 4.62, p < 0.005, ${\eta }_{\text{p}}^2=0.15$]. This interaction was supported by three two-way interactions: emotion × face part [F(3, 81) = 13.16, p < 0.001, ${\eta }_{\text{p}}^2=0.33$]; emotion × in/out-group headdress[F(3, 81) = 3.56, p < 0.05, ${\eta }_{\text{p}}^2=0.12$]; face part × in/out-group headdress[F(3, 81) = 4.82, p < 0.05, ${\eta }_{\text{p}}^2=0.15$]. Since we had clear expectations based on a pilot experiment and on Experiment 2, we made three planned comparisons. Fearful women wearing a niqāb were recognized better than fearful women with a cap and scarf [t(27) = 1.920, p < 0.05]. Happiness and sadness were recognized better in women wearing a cap and scarf vs. a niqāb [t(27) = 2.607, p < 0.05; t(27) = 1.899, p < 0.05]. See Figures 5A,B.

FIGURE 5

Figure 5. (A) Inverse efficiency scores. The lower the number, the better the performance. Fear was better recognized from women wearing a niqāb vs. a cap and a shawl. Happiness and sadness were better recognized from women wearing a cap and a shawl than from women wearing a niqāb. (B) The graph shows difference scores from the “in-group minus out-group” inverse efficiency scores. The upper part of the graph shows conditions where the out-group was better recognized; the lower part where the in-group was better recognized, (C) the proportion of an emotion label given to a facial expression in different headdress-contexts (i.e., the number of fear responses to a niqāb divided by the total number of fear responses), (D) the proportion of an emotion label given to emotional eyes in the context of a niqāb or cap and shawl (i.e., the number of fear responses to a fearful face covered with a niqāb divided by the number of fear responses to a fearful face covered with a niqāb plus a cap and a scarf.

Response patterns

When dividing the number of emotion responses in a certain headgear condition by the total number of the emotion responses, we observed a three-way interaction between emotion × face part and in/out-group headdress[F(3, 81) = 2.775, p < 0.05, ${\eta }_{\text{p}}^2=0.09$] which was further supported by a two-way interaction between emotion × face part [F(3, 81) = 2.960, p < 0.05, ${\eta }_{\text{p}}^2=0.10$]. Anger and fear were more often used in the context of a niqāb than in the context of a cap and shawl niqāb [t(27) = 2.313, p = 0.01; t(27) = 2.280, p = 0.01]. Happiness was more often chosen for women wearing a cap and shawl than a niqāb [t(27) = 2.725, p = 0.005]. See Figure 5C. To test if these response biases were valid for all emotions, we compared the proportion of angry, happy, sad, and fearful responses given to angry, happy, sad, and fearful faces combined with a niqāb vs. a cap and a shawl³. Clearly, participants did not always choose the label fear or anger when being confronted with a niqāb and happiness when seeing a women wearing a cap and a shawl. It was dependent of the specific emotion which speaks against a general response bias. See Figure 5D.

Individual differences in negative attitudes toward the Islam

The consistent pattern that fear was best recognized from women wearing a niqāb did, in contrast with expectations, not correlate with negative attitudes toward the Islam or with trait anxiety scores. However, the number of times participants pressed a button with a threatening label, was positively related to both the STAI and the Negative Attitudes questionnaire, significantly for the label anger (STAI: R = 0.391, p < 0.05; Negative Attitudes: R = 0.415, p < 0.05), especially when the face expressed sadness (i.e., where performance was lowest; STAI: R = 0.420, p < 0.05; Negative Attitudes: R = 0.574, p < 0.001).

Discussion

Happiness was recognized better in women wearing a cap and scarf vs. a niqāb. Fear was recognized best from niqābis. The improved recognition of fear in women wearing a niqāb and of happiness in women wearing a cap and a scarf was not related to “attitudes toward the Islam” or general anxiety levels.

General Discussion

Everyday items like sunglasses, scarves, veils, caps, hats, helmets, medical masks, etc., lead to partial occlusions of the face, hampering identity, and emotion recognition. In the West and the Islamic world alike, the headscarf is the subject of intense discussions. Our aim was to investigate to what extent headdresses influence emotion recognition and whether a specific type of headdress has a stronger influence on one or another emotion.

Experiment 1 explored the importance of the eye region in recognizing emotions from women wearing different headdresses (hijāb, niqāb, burqa). Expressions could be recognized in the niqāb condition, but recognition in the burqa conditions was more impaired. While sad expressions were most difficult to recognize, performance in the fear burqa conditions was well above chance. Our results are consistent with the literature that stresses the importance of the eye region in emotional communication, especially when it comes to the expression of fear (Vuilleumier, 2005).

In Experiment 2 and 3, the importance of seeing the whole face vs. the eye region was further investigated. The effects of a hijāb and niqāb on emotion recognition performance were directly compared with the effects of a cap and cap and scarf covering the exact same face area. The results show that fear was recognized best in the niqāb condition. A cap and scarf were more often associated with happiness than the niqāb. Moreover, happiness but also sadness were recognized better in women wearing a cap and scarf than a niqāb. A smile is a strong cue in social communication and we routinely smile a lot to the people in our direct environment (Hess and Bourgeois, 2010). It is therefore not surprising that this emotion was better recognized from women seen as belonging to our in-group. Like happy faces, the display of sad expressions has long been linked to the inhibition of aggression and the elicitation of pro-social behavior which may be particularly strong among members of the same social group (Miller and Eisenberg, 1988).

We would have predicted a relationship between negative attitudes toward the Islam and recognition performance of participants when observing fearful out-group minus in-group members but did not see this in the results. However, asking participants directly about their attitudes on a sensitive topic such as Islamic headscarves might not have been the best way to test attitudes. Indirect questioning, such as asking participants to answer structured questions from the perspective of another person or group, might have given more variance. Moreover, brain activity measurements could have given us additional insight. A negative out-group bias can modulate activity in areas related to face perception (Golby et al., 2001), and in areas related to more complex aspects of social behavior such as empathic reactivity (Xu et al., 2009; Avenanti et al., 2010).

There are some limitations. In a naturalistic situation where one is confronted with a woman wearing a headscarf, one is often able to observe her for a much longer duration than 40–100 ms and is not forced to choose whether she is angry, happy, fearful, or sad. The forced-choice approach had the advantage of evoking quick and intuitive responses, but results from this research should be interpreted with caution. Moreover, the drawn conclusions only apply in the Western European culture and most probably not in a culture where women are forced to cover their hair and/or face. All participants were psychology students. Since students tend to have quite left-sided political views, it would be interesting to investigate the out-group bias toward fear further in politically right-oriented people. We expect to find even stronger effects in such a group, especially when compared with a group of Muslim participants.

Another limitation of the study is that only two representative types of headdresses were included, whereas many different types exist with different texture and colors and differences in the way it is draped around the head (the Turkish style giving more shade on the eye region than the Moroccan style for example).

To summarize, we show that a cultural context (i.e., headdress) can influence emotion recognition from the eyes. In line with previous studies, we show that emotional expressions can be quickly recognized from the eyes but in addition demonstrate that performance is dependent on the specific context in which the eyes are seen. Most importantly, fear was better recognized in the niqāb condition than in any other condition and this may reflect an implicit negative bias toward the out-group.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Research was supported by Nederlandse Organisatie voor Wetenschappelijk Onderzoek (400.04081) and European Commission (COBOL FP6-NEST-043403 and FP7 TANGO) grants. We thank A. Fischer and H. Meeren for comments on a previous version. We thank A. Tanaka for using his Matlab script to add noise to the eye region in Experiment 1. Development of the MacBrain Face Stimulus Set was overseen by N. Tottenham and supported by the J. D. and C. T. MacArthur Foundation Research Network on Early Experience and Brain Development.

Footnotes

1. ^A niqāb is a veil that covers the face, worn by some Muslim women as a part of a sartorial hijāb. We here use the term hijāb for a headscarf that leaves the full face visible but covers the hair. With a niqāb, we refer to a veil that covers hair, nose, and mouth and with a burqa to a veil which covers the whole face, including the eyes.
2. ^We ran an earlier version of this experiment in 33 other participants. The three-way interaction between emotion × culture × face part was highly significant for the IES [F(3, 84) = 8.84, p < 0.001, ${\eta }_{\text{p}}^2=0.24$] and the conclusions that could be drawn were similar to Experiments 2 and 3. However, we decided to consider this study as a pilot for Experiment 2 since we discovered a small difference between the hijāb and cap stimuli. In the former, more forehead was visible which of course is a confound. In Experiments 2 and 3 this was carefully controlled for. The pilot study helped us to refine the questionnaire and also our predictions in Experiments 2 and 3.
3. ^Number of fear responses given to a sad face with a niqāb/(number of fear responses given to a sad face with a niqāb + number of fear responses given to a sad face with a cap and scarf).

References

Amir, Y. (1969). Contact hypothesis in ethnic relations. Psychol. Bull. 71, 319–342.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Avenanti, A., Sirigu, A., and Aglioti, S. M. (2010). Racial bias reduces empathic sensorimotor resonance with other-race pain. Curr. Biol. 20, 1018–1022.

CrossRef Full Text

Aviezer, H., Bentin, S., Dudarev, V., and Hassin, R. R. (2011). The automaticity of emotional face-context integration. Emotion 11, 1406–1414.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Aviezer, H., Bentin, S., Hassin, R. R., Meschino, W. S., Kennedy, J., Grewal, S., and Moscovitch, M. (2009). Not on the face alone: perception of contextualized face expressions in Huntington’s disease. Brain 132, 1633–1644.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Aviezer, H., Hassin, R. R., Ryan, J., Grady, C., Susskind, J., Anderson, A., and Bentin, S. (2008). Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychol. Sci. 19, 724–732.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Baron-Cohen, S., Wheelwright, S., and Jolliffe, T. (1997). Is there a “language of the eyes?” Evidence from normal adults and adults with autism or Asperger syndrome. Vis. Cogn. 4, 311–331.

CrossRef Full Text

Caldara, R., Schyns, P., Mayer, E., Smith, M. L., Gosselin, F., and Rossion, B. (2005). Does prosopagnosia take the eyes out of face representations? Evidence for a defect in representing diagnostic facial information following brain damage. J. Cogn. Neurosci. 17, 1652–1666.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

de Bonis, M., De Boeck, P., Perez-Diaz, F., and Nahas, M. (1999). A two-process theory of facial perception of emotions. C. R. Acad. Sci. III Sci. Vie 322, 669–675.

de Gelder, B., Teunisse, J. P., and Benson, P. (1997). Categorical perception of facial expressions: categories and their internal structure. Cogn. Emot. 11, 1–23.

CrossRef Full Text

Eisenbarth, H., and Alpers, G. W. (2011). Happy mouth and sad eyes: scanning emotional facial expressions. Emotion 11, 860–865.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ekehammar, B., Akrami, N., and Araya, T. (2000). Development and validation of Swedish classical and modern sexism scales. Scand. J. Psychol. 41, 307–314.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Elfenbein, H. A., and Ambady, N. (2002). Is there an in-group advantage in emotion recognition? Psychol. Bull. 128, 243–249.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Emery, N. J. (2000). The eyes have it: the neuroethology, function and evolution of social gaze. Neurosci. Biobehav. Rev. 24, 581–604.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fischer, A. H., Gillebaart, M., Rotteveel, M., Becker, D., and Vliek, M. (2011). Veiled emotions: the effect of covered faces on emotion perception and attitudes. Soc. Psychol. Person. Sci. doi:10.1177/1948550611418534

CrossRef Full Text

Fiske, S. T. (2002). Bias and intergroup conflict, the problem of the century. Curr. Dir. Psychol. Sci. 11, 123.

CrossRef Full Text

Golby, A. J., Gabrieli, J. D., Chiao, J. Y., and Eberhardt, J. L. (2001). Differential responses in the fusiform region to same-race and other-race faces. Nat. Neurosci. 4, 845–850.

CrossRef Full Text

Gosselin, F., and Schyns, P. G. (2001). Bubbles: a technique to reveal the use of information in recognition tasks. Vision Res. 41, 2261–2271.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hechter, M., and Horne, C. (2003). Theories of Social Order. A Reader. Stanford: Stanford University Press.

Hess, U., and Bourgeois, P. (2010). You smile-I smile: emotion expression in social interaction. Biol. Psychol. 84, 514–520.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hugenberg, K., and Bodenhausen, G. V. (2003). Facing prejudice: implicit prejudice and the perception of facial threat. Psychol. Sci. 14, 640–643.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hugenberg, K., and Bodenhausen, G. V. (2004). Ambiguity in social categorization: the role of prejudice and facial affect in race categorization. Psychol. Sci. 15, 342–345.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Koji, S., and Fernandes, M. (2010). Does it matter where we meet? The role of emotional context in evaluative first impressions. Can. J. Exp. Psychol. 64, 107–116.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kret, M. E., and de Gelder, B. (2010). Social context influences recognition of bodily expressions. Exp. Brain Res. 203, 169–180.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Langner, O., Becker, E. S., and Rinck, M. (2009). Social anxiety and anger identification: bubbles reveal differential use of facial information with low spatial frequencies. Psychol. Sci. 20, 666–670.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meeren, H. K., van Heijnsbergen, C. C., and de Gelder, B. (2005). Rapid perceptual integration of facial expression and emotional body language. Proc. Natl. Acad. Sci. U.S.A. 102, 16518–16523.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Miller, P. A., and Eisenberg, N. (1988). The relation of empathy to aggressive and externalizing/antisocial behavior. Psychol. Bull. 103, 324–344.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Moore, D. S., McCabe, G. P., Duckworth, W. M., and Sclove, S. L. (2004). Practice of Business Statistics, Part IV, Chaps. 12–18. Palgrave Macmillan.

Moors, A., and Salih, R. (2009). “Muslim women” in Europe: secular normativities, bodily performances and multiple publics. Anthropol. Soc. 17, 375–378.

CrossRef Full Text

Morris, J. S., deBonis, M., and Dolan, R. J. (2002). Human amygdala responses to fearful eyes. Neuroimage 17, 214–222.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nahm, F. K. D., Perret, A., Amaral, D. G., and Albright, T. D. (1997). How do monkeys look at faces? J. Cogn. Neurosci. 5, 611–623.

CrossRef Full Text

Righart, R., and de Gelder, B. (2006). Context influences early perceptual analysis of faces – an electrophysiological study. Cereb. Cortex 16, 1249–1257.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Righart, R., and de Gelder, B. (2008). Recognition of facial expressions is influenced by emotional scene gist. Cogn. Affect. Behav. Neurosci. 8, 264–272.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Spezio, M. L., Adolphs, R., Hurley, R. S., and Piven, J. (2007). Analysis of face gaze in autism using “Bubbles.” Neuropsychologia 45, 144–151.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Spielberger, C. D. (1983). Manual for the State-Trait Anxiety Inventory (STAI). PaloAlto, CA: Consulting Psychologists Press.

Stephan, W. G., Boniecki, K. A., Ybarra, O., Bettencourt, A., Ervin, K. S., and Jackson, L. A. (2002). The role of threats in the racial attitudes of Blacks and Whites. Pers. Soc. Psychol. Bull. 28, 1242–1254.

CrossRef Full Text

Stephan, W. G., and Stephan, C. W. (1989). Antecedents of inter-group anxiety in Asian-Americans and Hispanic-Americans. Int. J. Intercult. Relat. 13, 203–219.

CrossRef Full Text

Tamietto, M., Latini Corazzini, L., de Gelder, B., and Geminiani, G. (2006). Functional asymmetry and interhemispheric cooperation in the perception of emotions from facial expressions. Exp. Brain Res. 171, 389–404.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Townsend, J. T., and Ashby, F. G. (1983). Stochastic Modeling of Elementary Psychological Processes. Cambridge: Cambridge University Press.

Van den Stock, J., Righart, R., and de Gelder, B. (2007). Body expressions influence recognition of emotions in the face and voice. Emotion 7, 487–494.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

van der Schalk, J., Fischer, A., Doosje, B., Wigboldus, D., Hawk, S., Rotteveel, M., and Hess, U. (2011). Convergent and divergent responses to emotional displays of ingroup and outgroup. Emotion 11, 286–298.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Van Zomeren, M., Fischer, A. H., and Spears, R. (2007). Testing the limits of tolerance: how intergroup anxiety amplifies negative and offensive responses to out-group-initiated contact. Pers. Soc. Psychol. Bull. 33, 1686–1699.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Vuilleumier, P. (2005). Cognitive science: staring fear in the face. Nature 433, 22–23.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Weisbuch, M., and Ambady, N. (2008). Affective divergence: automatic responses to others’ emotions depend on group membership. J. Pers. Soc. Psychol. 95, 1063–1079.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Whalen, P. J., Kagan, J., Cook, R. G., Davis, F. C., Kim, H., Polis, S., and Johnstone, T. (2004). Human amygdala responsivity to masked fearful eye whites. Science 306, 2061.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

White, M. (1999). Representation of facial expressions of emotion. Am. J. Psychol. 112, 371–381.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Xu, X., Zuo, X., Wang, X., and Han, S. (2009). Do you feel my pain? Racial group membership modulates empathic neural responses. J. Neurosci. 29, 8525–8529.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Appendix

The means and SD of the students who participated in Experiment 3 are written in italics.

1. Are you male or female?

2. Date of birth

4-11-1988

3. What is your nationality?

Dutch

Different, namely………‥       All Dutch

4. What is your religious belief?

5. Are you left or right-handed?

6. Do you carry glasses/contact lenses?

7. Do you have complaints about your concentration or your memory? (e.g., difficulty in following the news or reading a book)

8. How many children with an Islamic background did you have in class at elementary school?

Please only fill in the number below.

2.36 (3.78)

9. How many children with an Islamic background you had in class in high school?

Please only fill in the number below.

2.00 (2.23)

10. How was your experience with these students?

11. To what degree do you currently sort with people with an Islamic background?

12. Below are some statements that are used by people to describe themselves. Read through each statement and color the dot to indicate how you generally feel. There are no right or wrong answers. Do not think too long, it is about your first impression. It is important to indicate how you generally feel.

My attitude toward women who wear Islamic clothing (headdresses, niqāb) is one of

13. In my opinion, women wearing Islamic clothing (headscarf, niqāb) are generally

14. Women wearing Islamic clothing (headscarf, niqāb) have too much………… on society

15. I see women who wear Islamic clothing (headscarf, niqāb) as a threat to society

16. I think that women who wear Islamic clothing (headscarf, niqāb) do not appreciate the values of women who do not wear Islamic clothing.

17. If I see a woman who wears Islamic clothing (headscarf, niqāb) on the street, I generally feel

18. To what extent are you in the past offended by a woman who was wearing Islamic clothing (headscarf, niqāb)

19. There are plenty of programs designed to create jobs for people with Islamic faith

20. The demand of people with Islamic faith for equal rights is easy to understand

21. People with an Islamic belief are given too little attention in the media

22. People with Islamic faith are increasingly demanding in their fight for equal rights

23. It is good to strive for a multicultural society in the Netherlands