A commentary has been posted on this article:
Why Some Faces will Not be Remembered: Current ERP Evidence on Memory Encoding for Other-Race and Other-Age Faces
- Department of Psychology, Northwestern University, Evanston, IL, USA
Memory is often less accurate for faces from another racial group than for faces from one’s own racial group. The mechanisms underlying this phenomenon are a topic of active debate. Contemporary theories invoke factors such as inferior expertise with faces from other racial groups and an encoding emphasis on race-specifying information. We investigated neural mechanisms of this memory bias by recording event-related potentials while participants attempted to memorize same-race (SR) and other-race (OR) faces. Brain potentials at encoding were compared as a function of successful versus unsuccessful recognition on a subsequent-memory test. Late positive amplitudes predicted subsequent memory for SR faces and, to a lesser extent, for OR faces. By contrast, the amplitudes of earlier frontocentral N200 potentials and occipito-temporal P2 potentials were larger for later-remembered relative to later-forgotten OR faces. Furthermore, N200 and P2 amplitudes were larger for OR faces with features considered atypical of that race relative to faces that were race-stereotypical (according to a consensus from a large group of other participants). In keeping with previous reports, we infer that these earlier potentials index the processing of unique or individuating facial information, which is key to remembering a face. Individuation may tend to be uniformly high for SR faces but lower and less reliable for OR faces. Individuation may also be more readily applied for OR faces that appear less stereotypical. These electrophysiological measures thus provide novel evidence that poorer memory for OR faces stems from encoding that is inadequate because it fails to emphasize individuating information.
Introduction
The ability to recognize the faces of others is an important skill in human society. However, this skill does not benefit all faces equally. The other-race effect refers to the robust phenomenon wherein recognition memory is less accurate for other-race faces than for same-race faces (henceforth referred to as OR faces and SR faces, respectively). There has been much debate over what causes poorer memory for OR faces. Several perceptual and social factors that may contribute to the other-race effect have been identified in prior investigations.
One potential factor is perceptual expertise. According to perceptual expertise accounts of the other-race effect, a perceiver’s greater experience with SR relative to OR faces can lead to difficulty encoding the physical dimensions along which OR faces tend to differ from one another (Valentine, 1991; Chiroro and Valentine, 1995; Furl et al., 2002; Walker and Hewstone, 2006). This asymmetric expertise results in encoding for OR faces that places an insufficient emphasis on individuating physiognomic information, or facial attributes that distinguish a given face from others faces of the same social group1. Training in perceptual individuation (i.e., learning to identify and encode such person-specific attributes of a face) has been shown to alleviate the other-race effect, lending credence to this account (Tanaka and Pierce, 2009).
Other research suggests that the other-race effect may stem from different types of categorization elicited by SR versus OR faces due to their respective status as ingroup and outgroup members (Levin, 1996, 2000; Hugenberg and Sacco, 2008) or majority and minority groups (Chiao et al., 2006). Outgroup members are generally perceived to have more in common with each other and to possess fewer unique personality attributes relative to ingroup members, a phenomenon known as the outgroup homogeneity effect (Park and Rothbart, 1982; Judd and Park, 1988; Mullen and Hu, 1989; Ostrom and Sedikides, 1992). As a result, perceivers are more likely to make a variety of distinctive social attributions for ingroup individuals, an elaborative encoding strategy that typically aids face memory (Craik and Lockhart, 1972; Bower and Karlin, 1974; Chance and Goldstein, 1981). In other words, OR face encoding can emphasize not only race-specifying physiognomic attributes, but also stereotype-congruent social or personality attributes, which are not helpful for discriminating among old and new OR faces later. In support of such social-cognitive accounts of the other-race effect, several studies have found better memory for ingroup relative to outgroup faces in situations in which perceptual expertise did not differ between groups (MacLin and Malpass, 2001; Bernstein et al., 2007; Shriver et al., 2008).
Disambiguating contributions of perceptual expertise to the other-race effect from those of outgroup categorization has proven difficult for multiple reasons. One key reason is that these contributing factors may not operate independently from one another. For example, perceptual expertise may play a causal role in determining the level of abstraction at which a face is categorized, such as whether the face is viewed primarily as an individual or primarily as a member of a certain racial group. In this way, perceptual and social factors may create and maintain the other-race effect through a cycle wherein perceptual homogenization encourages social homogenization. Indeed, perceptual training in OR face recognition was recently found to reduce implicit social stereotyping to a degree that correlated across-subjects with levels of reduction in the other-race effect (Lebrecht et al., 2009). Thus, differences in face recognition that appear to be a direct result of perceptual expertise may partially or entirely reflect downstream effects of social-cognitive processes.
Our ability to explain the other-race effect has also been limited by a reliance on behavioral measures. As described above, candidate contributing factors have been identified by examining situations that lead to improvements in OR face recognition (e.g., expertise training), or by examining situations that impair memory by co-opting aspects of the SR/OR distinction (e.g., using SR faces that belong to an outgroup). From the extant findings, it is unclear whether these manipulations influence face recognition via the same mechanisms operative during the other-race effect. In one study, for example, college students showed superior memory for faces labeled as students from their own university relative to faces that were labeled as students from another university (Bernstein et al., 2007). However, this “other-university effect” and similar findings may or may not share underlying mechanisms with typical instances of the other-race effect.
Neuroimaging techniques hold promise for addressing the above concerns because neural measures can potentially implicate specific mechanisms of face encoding. The present study advances this goal using the subsequent-memory technique, in which brain activity during encoding is compared for items subsequently remembered relative to items subsequently forgotten (Paller and Wagner, 2002). This technique makes it possible to identify neural correlates of encoding activity known as differences due to subsequent-memory performance, or Dm, which index pivotal encoding operations that determine whether or not memory formation will ultimately be successful. In the present study, we examined Dm for faces using event-related potentials (ERPs) extracted from the electroencephalogram. Dm for OR faces, in particular, can reveal which aspects of encoding go awry, and to our knowledge has not been investigated previously. In addition to providing insights into neural mechanisms of the other-race effect, such findings can allow future research to examine the extent to which these mechanisms are the same as those responsible for other memory biases such as minimal outgroup memory impairments (e.g., Bernstein et al., 2007; Shriver et al., 2008). In this way, identifying Dm for OR faces may constitute an initial step toward disentangling perceptual and social contributions to the other-race effect.
In previous ERP studies, subsequent-memory effects have been observed for a variety of stimuli, including faces, words, and objects (e.g., Sommer et al., 1991, 1995, 1997; for reviews, see Wagner et al., 1999; Paller and Wagner, 2002). In most cases, Dm takes the form of a widespread positivity on the scalp approximately 400–600 ms after stimulus onset, such that ERPs are relatively more positive for subsequently remembered than for subsequently forgotten items2. These effects have been ascribed to semantic elaboration, a process by which information extracted from the current stimulus becomes meaningfully integrated with other knowledge (Paller et al., 1987; Paller and Kutas, 1992; Wagner et al., 1999). Indeed, this late positive Dm has been absent in studies using semantically deprived stimuli such as pseudowords and abstract drawings (Van Petten and Senkfor, 1996; Otten et al., 2007). The Dm literature thus suggests that trial-to-trial variations in semantic elaboration during the encoding of meaningful stimuli influence which stimuli will later be remembered, consistent with long-standing evidence that deeper processing leads to superior memory. However, OR faces represent a stimulus category for which depth-of-processing at encoding does not always benefit memory. Memory for OR faces has been found to be similar or worse under deep relative to shallow encoding conditions (Rhodes et al., 2009; Stahl et al., 2010), though only in White perceivers (Anthony et al., 1992). An unexplored possibility, then, is that Dm may differ for SR and OR faces. For example, if OR faces do not always receive sufficient individuation due to perceivers’ lack of expertise, then encoding may sometimes fail before elaborative encoding can be attempted.
Previous findings provide information about electrophysiological correlates of candidate perceptual and social-cognitive factors that may be relevant to the other-race effect. To examine ERP correlates of perceptual expertise, Tanaka and Pierce (2009) recorded ERPs to OR faces before and after expertise training. Increases in occipito-temporal N250 potentials after training correlated across-subjects with improvements in OR face memory. Similar potentials appear to relate to expertise with non-face categories as well (Scott et al., 2006). N250 potentials are also greater for familiar relative to unfamiliar faces, both when that familiarity is acquired in the experimental setting and for faces depicting pre-experimentally known individuals (Schweinberger et al., 1995, 2002; Herzmann et al., 2004; Tanaka et al., 2006; Kaufmann et al., 2009). Together, these findings suggest that N250 potentials reflect expertise with processing faces in an individuating manner. N170 potentials have also been linked to expertise (e.g., Tanaka and Curran, 2001; Gauthier et al., 2003), though some evidence suggests that they may reflect expertise with category level rather than individual-level processing (Scott et al., 2006; but see Walker et al., 2008). Yet other studies have found greater occipito-temporal P2 potentials for SR relative to OR faces (Stahl et al., 2008, 2010), which may reflect greater or more effective extraction of configural (Boutsen et al., 2006; Latinus and Taylor, 2006; Mercure et al., 2008) and sometimes featural information (Mercure et al., 2008) from SR faces. However, it is unclear whether these potentials correlate with subsequent memory for OR faces.
Previous research has also identified ERPs that may index race-based attentional biases that influence the extent to which OR faces are individuated (Ito and Urland, 2003, 2005; Dickter and Bartholow, 2007; Kubota and Ito, 2007; Willadsen-Jensen and Ito, 2008; He et al., 2009). For example, Kubota and Ito (2007) found that N200 potentials were greater in amplitude for SR compared to OR faces in White participants performing a race categorization task. Furthermore, regression analyses revealed that the greater the magnitude of the SR N200 enhancement across participants, the slower race was categorized for SR relative to OR faces. N200 may thus reflect the processing of individuating attributes that interfere with race categorization. However, the relationship between these ERPs and subsequent memory for OR faces remains to be directly explored. In addition, a possibility that has been underappreciated thus far is that frontocentral N200 potentials may partially or entirely reflect the same processes underlying occipito-temporal P2 potentials as described above. P2 potentials are thought to be particularly sensitive to the extraction of configural information from faces, which is widely believed to be important for face individuation. Interestingly, studies reporting race-related modulations in N200 potentials have generally used a mastoid reference, while studies reporting P2 differences by race have used an average reference. Given that a mastoid reference tends to project focal occipito-temporal activity to frontocentral midline, choice of reference electrode may determine the extent to which neural activity manifests primarily as a frontocentral negativity (N200) or an occipito-temporal positivity (P2). Examination of the same data using both mastoid and average references would therefore be informative.
The present study included analyses of ERPs that have previously been shown to be race-sensitive in conjunction with subsequent-memory analyses in order to better understand their mnemonic significance. The study was conducted with Caucasian–Americans, as this group has consistently produced the other-race effect in previous research (for review, see Meissner and Brigham, 2001). These participants completed study-test blocks with three different conditions. In the SR condition, all faces were White; in the OR1 condition, all faces were Black; and in the OR5 condition, faces were drawn from five racial groups. The OR1 and OR5 conditions were included as an initial step toward examining the generalizability of observed ERP modulations by target race, and to allow us to examine effects of race heterogeneity. Data analyses included both mastoid-reference and average-reference ERPs. A validation experiment was included to show that the specific face stimuli used in the OR1 condition do not yield poorer recognition memory simply because of some other bias in stimulus factors across SR and OR conditions; rather, comparable other-race effects were found whether testing was conducted with African–American or Caucasian–American participants.
Experiment 1
Methods
Participants
A total of 18 adult females (undergraduate students 19–22 years of age, M = 20.61, SD = 0.85) participated in the experiment and received monetary compensation. All identified their race as White or Caucasian. Data from an additional four participants were collected but were excluded from analyses due to excessive ocular artifacts (n = 3) and data corruption (n = 1).
Stimuli
A set of 360 color photographs of adult male faces was complied from multiple sources (Phillips et al., 1998, 2000; Golby et al., 2001; Minear and Park, 2004). Photographs were cropped to display the face, neck, and hair, with background and clothing cues removed. There were 144 White faces, 144 Black faces, 24 East Asian faces, 24 South Asian faces, and 24 Hispanic faces. Subsets of 120 of the White and Black faces were randomly chosen for SR (all White) and OR1 (all Black) conditions. The remaining 120 faces comprised the OR5 condition (24 faces from each racial group). Examples of faces in the SR, OR1, and OR5 conditions are depicted in Figure 1. For half of the subjects, random subsets of 24 White and Black faces were exchanged between the OR5 and SR/OR1 conditions. An additional 30 faces were used as buffers. All images were presented on a computer monitor against a gray background. A fixation cross appeared in the center of the screen during each interstimulus interval (ISI).
FIGURE 1
Figure 1. Sample stimuli from the SR (top), OR1 (middle), and OR5 (bottom) conditions.
We validated these stimuli for producing the other-race effect using behavioral data collected from six Caucasian–American females and six African–American females. Procedures for testing face recognition were the same as described below. As shown in Figure 2, the other-race effect was evident in both groups when comparing results for White faces versus Black faces [t(5) = 3.37, p = 0.02 for Caucasian–American participants; t(5) = −2.96, p = 0.03 for African–American participants, two-tailed pairwise t-tests]. These data demonstrate that the other-race effect for Caucasian–American participants cannot be attributed to the use of OR stimuli that differed on other stimulus dimensions that made them generally more difficult to recognize, because the same faces were recognized better than White faces by the African–American participants.
FIGURE 2
Figure 2. Recognition memory data demonstrating the other-race effect. Both Caucasian–American and African–American groups showed superior recognition memory for same-race faces compared to other-race faces.
Procedure
The experiment consisted of six study-test blocks. One block from each condition (SR, OR1, and OR5) was presented in each half of the experiment. The order of block presentation was counterbalanced across participants. In each study phase, 30 faces were presented in a random order bounded by two primacy buffers and three recency buffers. In each test phase, 30 studied and 30 unstudied faces were presented in random order. In OR5 blocks, there were six studied and unstudied faces from each racial group, with no more than two in a row from the same group. Study and test phases were separated by a 30-s distraction task in which participants counted backward by 3’s from a random number between 100 and 1000.
In the study phase, participants were instructed to pay attention to the faces and to try to remember them for an upcoming recognition test while maintaining central fixation. Participants were also asked to rate how likely they would be able to remember each face. These judgments-of-learning were assigned using a four-point scale, with button 1 indicating that the face was highly likely to be remembered, button 2 indicating the face was somewhat likely to be remembered, button 3 indicating that the face was somewhat unlikely to be remembered, and button 4 indicating that the face was highly unlikely to be remembered. Each face was displayed for 2500 ms with a 1000-ms ISI.
In the test phase, participants were informed that they would be shown faces from the most recent study phase and unstudied faces that had not been presented at all in the experiment. Participants used four buttons to categorize each stimulus as old or new using response categories based on the “Remember/Know” paradigm (Gardiner and Rosalind, 1993). Button 1 corresponded to a “Remember” response, indicating that the participant remembered at least one specific contextual detail from the study-phase episode with that face. Buttons 2 and 3 corresponded to high- and low-confidence “Know” judgments, respectively, indicating recognition of the face in the absence of any corresponding contextual details. Button 4 corresponded to a “New” response, indicating that the face was unstudied. Stimulus presentation time and ISI were the same as in the study phase.
Event-related potentials were extracted from scalp electroencephalographic recordings from 59 tin electrodes embedded in an elastic cap. Locations approximated standard 10–20 system locations, some labeled with an additional letter when slightly shifted (a for anterior, p for posterior, i for inferior, or s for superior). Voltage was referenced to a right mastoid electrode and re-referenced offline to averaged mastoids for primary analyses. For analyses of occipito-temporal ERPs (N170, P2, and N250), voltage was re-referenced to the average of the entire set of recordings. The electrooculogram was recorded from four additional channels using electrodes below the center of each eye and on each outer canthus. Electrode impedance was below 5 kΩ. Signals were recorded with a band pass of 0.05–200 Hz, and sampled at a rate of 1000 Hz (Neuroscan synamps). Each 1100-ms averaging epoch began 200 ms prior to stimulus onset. Mean prestimulus amplitudes were subtracted to correct for baseline variability. Epochs containing electroocular or other artifacts were excluded from ERP analyses (7.8%, SE = 0.75). Statistical comparisons were performed using repeated-measures ANOVA (criterion p = 0.05) with Geisser–Greenhouse correction for non-sphericity where appropriate.
Results
Recognition memory
Table 1 depicts memory performance for each of the three conditions and each possible response in the recognition test. Because 20% of the faces in the heterogeneous OR5 condition were White (SR) faces, all analyses for this condition were conducted after excluding data from these faces.
TABLE 1
Table 1. Recognition memory performance for each experimental condition.
For the high-confidence recognition responses that participants made (Remember and High-Confidence Know), the proportion of old faces endorsed as old (mean hit rate = 0.58) differed substantially from the proportion of new faces endorsed as old (mean false-alarm rate = 0.14). The hit rate was greater than the false-alarm rate in every condition for these two responses [t(17)’s > 5.2, p’s < 0.001]. In general, strong recognition memory can thus be inferred when old faces were endorsed in this way in the recognition test. On the other hand, there was no evidence for veridical recognition for Low-Confidence Know, as the hit rate was not significantly different from the false-alarm rate in two conditions [t(17) = −1.27, p = 0.22 for SR; t(17) = −1.71, p = 0.11 for OR1] and was lower than the false-alarm rate in the third [t(17) = −2.16, p = 0.045 for OR5].
Based on these findings, old faces recognized using the Remember and High-Confidence-Know response categories were combined to form the Hit category (Remembered) for subsequent analyses. Similarly, old faces endorsed with Low-Confidence Know or New were classified as Misses (Forgotten), and new faces endorsed as old using Remember or High-Confidence-Know responses were classified as False Alarms. A Condition (SR, OR1, OR5) × Response Type (Remember, High-Confidence Know) ANOVA revealed that the proportions of Remember and High-Confidence Know Hits did not did not differ across conditions [F(2,16) = 1.65, p = 0.22 for the Condition × Response interaction], indicating that patterns of neural activity that predicted memory for SR and OR faces were not confounded with neural activity that predicted later-recollection versus later-familiarity.
Discrimination accuracy (as calculated by d′) was compared among the three conditions using Hits and False Alarms defined in this way. Corresponding d′ values (with SE) were 1.63 (0.09), 1.24 (0.11), and 1.27 (0.13) for SR, OR1, and OR5 conditions, respectively. Accuracy was significantly higher for the SR condition than for the OR1 condition [t(17) = 4.5, p < 0.001] or the OR5 condition [t(17) = 3.46, p = 0.003]. Accuracy did not differ between OR1 and OR5 conditions [t(17) = −0.19, p = 0.85].
In summary, memory results showed the expected pattern. Recognition accuracy was superior for SR faces compared to OR faces. Furthermore, recognition accuracy was similar when OR faces were presented in a homogenous versus a heterogeneous manner.
Electrophysiology
To isolate subsequent-memory effects within each condition, encoding trials were sorted into sets of subsequent-hits and subsequent-misses. To avoid an undue influence of a small number of trials in some subjects, an inclusion criterion was established of 15 artifact-free trials per condition per subject. Dm analyses thus included data from less than the full sample. Sixteen subjects were included in the SR analysis (mean trial count for SR subsequent-hits = 33, range = 17–42; mean for subsequent-misses = 22, range = 15–33), 16 in the OR1 analysis (mean trial count for OR1 subsequent-hits = 30, range = 19–44; mean for subsequent-misses = 25, range = 15–36) and 9 in the OR5 analysis (mean trial count for OR5 subsequent-hits = 22, range = 16–28; mean for subsequent-misses = 21, range = 16–25). Results from the OR5 condition should be interpreted with caution because of the small sample; our interpretations instead emphasize the SR and OR1 conditions. In addition, a further statistical analysis of subsequent-memory effects for OR faces was conducted by combining OR1 and OR5 faces, for which we enforced a more conservative inclusion criterion of 25 artifact-free trials. Seventeen participants were included in this contrast, for which the mean trial count for OR subsequent-hits was 55 (range = 33–72) and the mean for subsequent-misses was 43 (range = 28–65).
Previous studies of Dm for faces have yielded topographically widespread late positive ERPs (or, with an average reference, both late frontocentral positive and late occipito-temporal negative ERPs), with larger amplitudes for later-remembered relative to later-forgotten faces (Sommer et al., 1991, 1995, 1997; Yovel and Paller, 2004; Guo et al., 2005). We expected to obtain a late-onsetting Dm for SR faces, but predicted that earlier potentials related to perceptual expertise and/or social aspects of processing race might also index individuation and thus be correlated with later memory uniquely for OR faces. Based on findings from previous studies of ERPs sensitive to race or expertise, we focused Dm analyses on midline and occipito-temporal recordings.
Late positive Dm. Midline ERPs from SR, OR1, and OR5 conditions are shown in Figure 3. In the SR condition, later-remembered faces elicited larger positive ERPs relative to later-forgotten faces beginning approximately 200 ms after stimulus onset, maximum at centroparietal electrodes. Similar but visibly smaller differences were present for OR faces. We conducted separate 2 × 2 ANOVAs examining the mean amplitude of consecutive 200-ms intervals from 200 to 800 ms with factors Dm (later-hit/later-miss) and Condition (SR/OR) at centroparietal electrode Pzs (a location slightly superior to Pz). Data were included from the 16 subjects who contributed ERPs for both the SR condition and the combined OR condition. Dm was marginal at 200–400 ms [F(1,15) = 3.86, p = 0.07], and significant over subsequent intervals [F’s(1,15) > 10, p’s < 0.006], indicating more positive amplitudes for later-hits relative to later-misses. A Condition × Dm interaction at 200–400 ms [F(1,15) = 5.04, p = 0.04] indicated that Dm was larger for SR relative to OR faces. A similar trend was non-significant at 400–600 ms [F(1,15) = 1.29, p = 0.27] and marginal at 600–800 ms [F(1,15) = 3.51, p = 0.08].
FIGURE 3
Figure 3. Subsequent-memory effects. (A) ERPs at encoding contrasted for subsequently remembered and subsequently forgotten faces for SR faces (left), OR1 faces (center), and OR5 faces (right, based on a subset of nine participants). Waveforms shown for five midline electrode locations (Fzp, Cza, Cz, Pzs, and Pzi). (B) Topography of corresponding subsequent-memory effects.
To further explore the timecourse of effects, we analyzed Dm for each interval and for each of the three conditions. For SR faces, Dm was significant for every interval [F’s(1,15) > 7.44, p’s < 0.02]. By contrast, Dm for OR1 faces was significant only at 600–800 ms [F(1,15) = 5.64, p = 0.03; other p’s > 0.17]. For OR5 faces, Dm was not significant for any interval [F’s(1,8) < 2.17, p’s > 0.18], although statistical power was limited by the small sample size. To summarize, a late positive Dm was observed at 200–800 ms and was particularly robust for SR faces.
Early frontocentral Dm (N200). Inspection of midline waveforms revealed subsequent-memory effects in the neighborhood of a negative peak around 200 ms, referred to as N200 (Figure 3). The direction of these effects differed systematically across conditions. N200 potentials were larger (more negative) for later-remembered relative to later-forgotten OR faces, whereas the opposite pattern was apparent for SR faces. N200 Dm effects were maximal at frontocentral electrode Cza, and N200 potentials were quantified as the mean amplitude at Cza from 200 to 240 ms. A Condition × Dm ANOVA comparing the SR and combined OR conditions confirmed a significant Condition × Dm interaction [F(1,15) = 10.46, p = 0.006]. Within-condition comparisons revealed that N200 amplitudes were more positive for subsequently remembered relative to subsequently forgotten SR faces [F(1,15) = 5.68, p = 0.03]. By contrast, an opposite-polarity Dm was present for the combined OR condition, with more negative amplitudes for subsequent-hits than subsequent-misses [F(1,16) = 4.61, p = 0.047, Figure A1 in Appendix]. This difference was also significant for OR1 faces considered separately [F(1,15) = 4.87, p = 0.043]. For OR5 faces, Dm was apparent slightly later, and mean amplitude measurements from 235 to 265 ms revealed a marginal Dm of the same polarity as was observed for OR1 faces [F(1,8) = 3.45, p = 0.10].
Given that similar frontocentral N200 potentials have previously been found to be larger in response to SR relative to OR faces (Ito and Urland, 2003, 2005; Dickter and Bartholow, 2007; Kubota and Ito, 2007), we sought to determine whether our data replicated this pattern. ERPs were computed for each condition regardless of subsequent memory, as depicted in Figures 4A,C. Mean amplitudes from 200 to 240 ms at Cza differed across the three conditions [F(1.95,33.19) = 15.43 p > 0.001]. Pairwise comparisons revealed that N200 amplitudes were larger for SR > OR1 [F(1,17) = 26.22, p < 0.001], SR > OR5 [F(1,17) = 4.69, p = 0.045], and OR5 > OR1 [F(1,17) = 12.82, p = 0.002]. However, interpretations of the OR5 > OR1 difference are complicated by overlap from a prior positivity peaking between 150 and 200 ms, which is also more positive for OR1 > OR5 [F(1,17) = 7.65, p = 0.01]. Thus, these data are equivocal with respect to whether N200 potentials per se differentiated between the two OR conditions. Regardless, these analyses suggest that N200 potentials during face encoding reflected a process engaged to a greater extent for SR relative to OR faces in addition to exhibiting an OR-specific relationship to subsequent memory.
FIGURE 4
Figure 4. Event-related potentials at encoding for SR, OR1, and OR5 faces, independent of subsequent memory. (A) Waveforms at frontocentral electrode Cza computed using a mastoid reference. (B) Waveforms at right-occipito-temporal electrode T2i computed using an average reference. (C) Topography of differences between conditions from 200 to 240 ms computed using a mastoid reference. (D) Topography of differences between conditions from 200 to 240 ms computed using an average reference.
In addition to larger frontocentral N200s for SR relative to OR faces, similar findings have been reported for positive-going occipito-temporal ERPs occurring around 200 ms (P2 potentials; Stahl et al., 2008, 2010). Thus, at the same latency in which SR faces elicit greater frontocentral negativities relative to OR faces, SR faces also elicit greater occipito-temporal positivities relative to OR faces, particularly when an average across all scalp locations is used as the reference. Given the proximity of right and left mastoids to occipito-temporal locations, using a mastoid reference tends to project activity from occipito-temporal sites to frontocentral sites with inverted polarity. Thus, these frontocentral N200s may partially or entirely measure the same underlying processes reflected in occipito-temporal P2s. We thus examined ERPs re-referencing to the average of all scalp locations as follows.
Visually pronounced differences in the resulting waveforms occurred in positive-going potentials around 200 ms, referred to as P2, which were maximal at right-occipito-temporal electrode T2i. As shown in Figures 4B,D, mean amplitudes from 200 to 240 ms at this electrode differed across the three conditions [F(1.94,33.03) = 19.87, p < 0.001]. Pairwise comparisons revealed that P2 amplitudes were larger for SR > OR1 [F(1,17) = 34.04, p < 0.001], SR > OR5 [F(1,17) = 7.73, p = 0.01], and OR5 > OR1 [F(1,17) = 13.85, p = 0.002]. Thus, occipito-temporal P2 potentials showed effects at encoding that mirrored those of frontocentral N200s. Parallels between N200 and P2 potentials are also evident in additional Dm analyses described below.
Early occipito-temporal Dm. In order to examine several possible subsequent-memory effects in occipito-temporal potentials, we analyzed these ERPs re-referenced to the average of all scalp locations. Visual inspection of waveforms from the SR and combined OR conditions revealed occipito-temporal subsequent-memory effects within the first 300 ms that were largest at electrodes T2i in the right hemisphere and T1i in the left hemisphere. As depicted in Figure 5, Dm was apparent in differences encompassing a negative peak around 170 ms (referred to as N170), a positive peak around 200 ms (referred to as P2), and a negative peak around 270 ms (referred to as N250). Statistical analyses were conducted using repeated-measures ANOVAs with factors Condition (SR/Combined OR), Dm (Later-Hit/Later-Miss), and Hemisphere (Right/Left) for each potential, followed by Dm × Hemisphere ANOVAs within each condition.
FIGURE 5
Figure 5. Subsequent-memory effects at occipito-temporal locations. ERPs at encoding are contrasted as a function of later memory for SR (top), OR1 (center), and OR5 faces (bottom, based on a subset of nine participants). Whereas waveforms in other figures were computed using a mastoid reference, these waveforms were computed using an average reference (see text).
When N170 potentials were measured as a function of subsequent memory, amplitudes were slightly more negative for later-remembered relative to later-forgotten faces. However, formal comparisons from 150 to 200 ms at T1i and T2i revealed no main effects or interactions involving Dm [p’s > 0.26]. Within-condition analyses concerned mean amplitudes over a 20-ms interval centered on the peak difference measured between 150 and 200 ms at each electrode in each condition. No significant main effects of Dm [F’s < 2.5, p’s > 0.14] or Dm × Hemisphere interactions [F’s < 0.33, p’s > 0.58] were obtained.
We were particularly interested in the extent to which P2 potentials exhibited subsequent-memory effects that mirrored those of frontocentral N200 potentials in the prior analyses. Formal comparisons from 200 to 240 ms revealed a significant Dm × Condition interaction [F(1,15) = 7.42, p = 0.02]. Other main effects and interactions involving Dm were non-significant [p’s > 0.07]. Within-condition analyses revealed a significant main effect of Dm for SR faces [F(1,15) = 5.41, p = 0.03], such that later-remembered SR faces exhibited smaller P2 potentials than did later-forgotten SR faces. By contrast, the combined group of OR faces showed the opposite pattern, by which later-remembered faces exhibited larger P2 potentials relative to later-forgotten faces [F(1,16) = 4.66 p = 0.046, see Figure A1 in Appendix]. Similar trends were non-significant for OR1 faces considered separately [F(1,15) = 2.57, p = 0.13], and marginal for OR5 faces considered separately [F(1,8) = 5.11, p = 0.054, measured from 235 to 265 in ms to reflect a slightly later P2 peak in this comparison]. No Dm × Hemisphere interactions emerged in any condition [all p’s > 0.17]. Thus, Dm effects in occipito-temporal P2 potentials assessed using averaged-referenced data were similar to those found in frontocentral N200 potentials using mastoid-referenced data. Whereas N200 potentials were more negative for later-remembered related to later-forgotten OR faces, P2 potentials were more positive for later-remembered relative to later-forgotten OR faces.
N250 subsequent-memory effects were visually apparent within the SR condition only. Formal comparisons from 275 to 350 ms revealed no significant main effects or interactions involving Dm except a Dm × Hemisphere × Condition interaction [F(1,15) = 4.54, p = 0.050; other p’s > 0.12]. Within-condition analyses concerned mean amplitudes over a 40-ms interval centered on the peak of N250 potentials between 275 and 350 ms in each condition. A marginal main effect of Dm was present for SR faces [312–352 ms at T1i, 309–349 ms at T2i, F(1,15) = 3.68, p = 0.07], as N250 amplitudes were greater for later-remembered relative to later-forgotten faces. Dm was not significant for the combined group of OR faces [F(1,16) = 0.41, p = 0.53, Figure A1 in Appendix], nor for OR1 or OR5 faces considered separately [p’s > 0.19]. No significant Dm × Hemisphere interactions emerged in any condition [all p’s > 0.15].
Discussion
Event-related potentials recorded during the encoding phase of a recognition paradigm provided evidence that the neural loci of memory failures and successes for OR faces are partially distinct from those for SR faces. Dm effects for OR and SR faces differed within two different time intervals, presumably reflecting two different stages of face encoding. Face encoding may be deficient for OR faces both with respect to individuation, as reflected by effects on frontocentral N200 and occipito-temporal P2 potentials, and with respect to semantic elaboration, as indicated by the late positive Dm findings.
A late positive Dm was present for both SR and OR faces, but was larger for SR faces. Within-condition Dm analyses revealed that the onset of this Dm was later for OR relative to SR faces. We interpret this Dm as a reflection of differential elaborative encoding, consistent with similar ERP findings with non-face stimuli as well as with behavioral evidence that semantic elaboration is a less-effective encoding strategy for OR than for SR faces (Rhodes et al., 2009; Stahl et al., 2010). Semantic elaboration may generally transpire during encoding over the interval from approximately 200–800 ms after face onset, in keeping with the robust subsequent-memory effects found for SR faces in this interval.
An earlier, temporally restricted, and OR-selective Dm was found in frontocentral N200 potentials, which were more negative for later-remembered relative to later-forgotten OR faces. In addition, occipito-temporal P2 potentials showed a parallel pattern when average-referenced data were examined, such that P2 potentials were larger for later-remembered relative to later-forgotten OR faces. These findings concur with other parallels between N200 and P2 potentials evident in the literature on race and face perception. Indeed, during face perception tasks, both N200 potentials (assessed using mastoid references) and P2 potentials (assessed using average references) have been larger for SR relative to OR faces. N200 differences have been interpreted to reflect perceivers’ greater abilities and/or tendencies to orient toward unique aspects of SR faces (Ito and Urland, 2003, 2005; Kubota and Ito, 2007). P2 potentials have been specifically attributed to the extraction of physiognomic information, especially configural information (Latinus and Taylor, 2006; Stahl et al., 2008, 2010), which is thought to be a primary means by which faces are individuated. The present findings support these interpretations by showing that both N200 and P2 potentials reflect encoding processes consistently engaged for SR faces but variable for OR faces, such that greater amplitudes predict subsequent memory only for the latter. Poor individuation at this early latency could also have diminished the effectiveness of later elaborative encoding, thus contributing to the pattern by which less robust late positive Dm effects were observed for OR relative to SR faces.
Given the many parallels exhibited here and elsewhere between frontocentral N200 potentials and occipito-temporal P2 potentials, it is tempting to conclude that these ERPs are functionally the same. However, further research is needed to substantiate this possibility given that N200s and P2s have generally been studied in separate experiments. Indeed, the functional properties of N200s and P2s described in the literature are not identical. N200 potentials are enhanced by the presence of perceptually unique stimulus features in non-face items (Folstein and van Petten, 2008; Folstein et al., 2008). For example, Folstein et al. (2008) examined N200 potentials during classification tasks involving artificial creatures, and found larger N200 potentials for creatures with relatively unique perceptual features when those features represented task-relevant dimensions. Although occipito-temporal P2s have not been extensively examined using non-face stimuli, P2 amplitudes have been found to decrease when faces are transformed along featural or configural dimensions, such that they are smaller for faces that appear more unique (although this pattern may be specific to within-identity transformations and has not been examined in OR faces; Halit et al., 2000; Burkhardt et al., 2010). On the basis these reports, N200 and P2 potentials appear to have opposing relationships to perceptual uniqueness. One possible explanation for this pattern is that greater prototypicality for SR faces, but not for non-faces, allows for more fine-grained perceptual processing, given the extensive experience perceivers have individuating prototypical SR faces. Indeed, a smaller perceptual difference is needed to distinguish two typical faces relative to two atypical faces (Tanaka et al., 1998; Tanaka and Corneille, 2007). Thus, paradoxically, SR faces with less unique appearances may be attended and processed on a more individuating level. However, future research is needed to substantiate this speculation and, more generally, to elucidate the relationship between N200s and P2s.
An unexpected finding was that both N200 and P2 potentials showed an opposite-polarity Dm for SR versus OR faces. For example, N200 potentials were smaller (more positive) for later-remembered relative to later-forgotten SR faces. The findings for SR faces, however, may reflect the early onset of the late positive Dm for SR faces, which was significant over 200–400 ms and so overlapped with N200 and P2 potentials. In other words, the apparent relationship between N200 amplitude and later memory for SR faces may reflect the same phenomenon described as the late positive Dm.
Among all the potentials examined, frontocentral N200 potentials and occipito-temporal P2 potentials were particularly informative because they yielded OR-specific Dm findings. We thus propose that these potentials indexed face individuation that tended to be uniformly high for SR faces but lower and more variable for OR faces. Findings here and in prior reports of larger N200 and P2 potentials for SR relative to OR faces could likewise signal deficient individuation for OR faces. And yet, OR face encoding is not uniformly deficient. Despite the multiple perceptual and social factors working against the successful encoding of OR faces, some of these faces are nonetheless successfully memorized. We hypothesize that the larger N200 and P2 potentials exhibited for remembered OR faces in the present study indexed greater processing of perceptually unique facial information, which afforded more individuated encoding (as routinely applied to SR faces). This account is premised on the idea that there is functionally meaningful variability in N200 and P2 amplitudes when OR faces are encoded, but it does not address the specific causes of this variability. Thus, the question remains – what determines the extent to which OR face individuation occurs?
Given the dependence of N200 novelty effects on attention to novel features, it is possible that attentional fluctuations across the course of an encoding session are the primary source of this variability. P2 potentials to OR faces have also been found to be influenced by attentional manipulations (Stahl et al., 2010). However, it is plausible that specific OR faces were particularly prone to individuation, and therefore that stimulus-based ERP effects may have been present. For example, highly distinctive OR faces may not require such fine-grained analysis in order to be adequately encoded (Tanaka et al., 1998). Furthermore, findings from Sommer et al. (1995) suggest that Dm for faces can be intimately related to facial distinctiveness. Thus, N200 and/or P2 enhancements to later-remembered OR faces may have stemmed from the presence of particularly distinctive facial information. In a like manner, variations in race-atypical features, rather than distinctiveness per se, may account for these variations. OR faces that appear race-atypical may be encoded more effectively, with greater emphasis on individuating information, larger N200 and P2 amplitudes, and superior subsequent recognition. In Experiment 2 we investigated these possibilities by collecting norms for distinctiveness and race-stereotypicality for the same set of facial images, and by applying these norms in further analyses of data from Experiment 1.
Experiment 2
We collected face ratings along four dimensions. In addition to measuring distinctiveness and race-stereotypicality, we also examined approachability and facial affect. These last two factors might be expected to influence N200 and/or P2 amplitudes given some social-cognitive accounts of the other-race effect that posit a strong role of motivation in determining whether perceivers allocate resources toward processing individuating features of outgroup members (e.g., Rodin, 1987; Ackerman et al., 2006).Ackerman et al. (2006) hypothesized that motivation to individuate outgroup members is strongest when those outgroup members appear to be threatening. In support of this hypothesis, these authors found that the other-race effect was eliminated when OR faces displayed angry expressions. Affective factors such as expression or perceived approachability could thus contribute to the likelihood of individuation.
We used these ratings to re-analyze ERP data from Experiment 1, grouping encoding trials from each condition according to consensus characteristics. This strategy allowed us to examine effects of these stimulus characteristics on ERPs elicited during encoding without complicating interpretations by introducing additional task requirements. In addition, we examined relationships between the characteristics assessed in Experiment 2 and behavioral measures of memorability assessed in Experiment 1.
Methods
Participants
A total of 96 adult females (undergraduate students 18–21 years of age) participated in the experiment and received course credit. All identified their race as White or Caucasian. Half rated faces for distinctiveness and approachability, in counterbalanced order. The remaining half rated faces for race-stereotypicality and facial affect, also in counterbalanced order.
Procedure
Participants saw the six blocks of faces from Experiment 1. Stimulus presentation was identical to that in Experiment 1 except that the test phases were omitted. After receiving instructions and rating each face once, participants received new instructions and then rated all faces a second time.
Participants rated each face according to a different five-point scale for each task. For distinctiveness, Button 1 corresponded to “not at all distinctive” while Button 5 corresponded to “very distinctive.” For approachability, Button 1 corresponded to “not at all approachable” while Button 5 corresponded to “most approachable.” For race-stereotypicality (following the procedures of Eberhardt et al., 2004), participants were instructed to rate stereotypicality using physical features that most people would associate with individuals of that race. They were instructed to use any number of physical features (e.g., lips, nose, hair texture, skin tone) to make these ratings. Button 1 corresponded to “not at all stereotypical” while Button 5 corresponded to “very stereotypical.” For facial affect, Button 1 indicated “a very negative emotional expression” while Button 5 indicated “a very positive emotional expression.”
Results
Overall ratings by condition
Table 2 shows results for each characteristic on which faces were rated. Statistical analyses revealed no significant differences among the three conditions in average ratings for distinctiveness, approachability, or facial affect (p’s > 0.32). By contrast, ratings of race-stereotypicality varied by condition [F(2,201) = 5.54, p = 0.005], as race-stereotypicality was higher for OR1 relative to SR faces [t(142) = −3.15, p = 0.002], and higher for OR5 relative to SR faces [t(130) = −2.74, p = 0.007]. Race-stereotypicality did not differ between OR1 and OR5 faces [t(130) = 0.28, p = 0.78].
TABLE 2
Table 2. Average rating of distinctiveness, approachability, race-stereotypicality, and facial affect assigned to faces within each condition.
Intercorrelations among these ratings were also examined. Affect and approachability were positively correlated for all three conditions (r’s < 0.90, p’s < 0.001), indicating that faces displaying more positive affect were judged to be more approachable. Affect and race-stereotypicality were positively correlated for SR [r(70) = 0.52, p < 0.001] and OR5 faces [r(58) = 0.36, p = 0.005], but not for OR1 faces [r(70) = 0.05, p = 0.67], indicating that SR and OR5 faces displaying a more positive expression were judged to be more stereotypical. Likewise, approachability and race-stereotypicality were also positively correlated for SR [r(70) = 0.53, p < 0.001] and OR5 faces [r(58) = 0.35, p = 0.007], but not for OR1 faces [r(70) = 0.14, p = 0.24], indicating that SR and OR5 faces that were judged to be more approachable were also judged to be more stereotypical. Finally, for OR1 faces only, distinctiveness was negatively correlated with race-stereotypicality [r(70) = −0.48, p < 0.001; r’s for SR and OR5 faces = −0.10 and −0.12, respectively; p’s > 0.34]. Thus, OR1 faces that were perceived as more distinctive in appearance were also perceived to be less stereotypical.
Ratings and face memorability
To determine whether across-face variations in these consensus characteristics related differentially to recognition accuracy in Experiment 1, we conducted a correlational analysis for each rating. For this purpose, we used the average memory judgment across all participants in Experiment 1 as a measure of face memorability for each face, wherein a lower value indicated a better-remembered face (1 for remember, 2 for high-confidence know, 3 for low-confidence know, and 4 for new). Each face thus had a memorability index along with a rating of consensus distinctiveness, race-stereotypicality, approachability, and facial affect. The only significant correlation between any rating and face memorability was a negative correlation between distinctiveness and memorability [r’s = −0.62, −0.60, and −0.39 for SR, OR1, and OR5 faces, respectively, p’s < 0.002]. These findings are in line with previous evidence that distinctiveness benefits face memory (Bartlett et al., 1984; Vokey and Read, 1992; Sommer et al., 1995), although this relationship was no stronger for OR than for SR faces. Correlations between memorability and approachability, race-stereotypicality, or facial affect were non-significant (r’s < 0.15, p’s > 0.13).
As previously mentioned, our a priori hypothesis was that race-stereotypicality would influence memory for OR faces such that less stereotypical faces would be better remembered. To examine this relationship, we divided OR1 faces into four quartiles based on race-stereotypicality (Table 3). The main effect of quartile on memory was marginal [F(3,68) = 2.72, p = 0.051]. Pairwise comparisons revealed that this effect was driven by better memory for faces in the least stereotypical quartile relative to the others [F(1,34) = 4.71, p = 0.04 for Q1 relative to Q2; F(1,34) = 6.81, p = 0.01 for Q1 relative to Q3; F(1,34) = 4.95, p = 0.03 for Q1 relative to Q4]. Thus, consensus race-stereotypicality influenced memory only for the least stereotypical faces, which were recognized best. Similar analyses by quartiles did not reveal significant effects of race-stereotypicality on memory for OR5 and SR faces [F(3,56) = 2, p = 0.13 and F(3,68) = 1.38, p = 0.26, respectively].
TABLE 3
Table 3. Recognition accuracy for faces in Experiment 1 as a function of consensus ratings of race-typicality in Experiment 2.
Ratings and ERPs
To examine the possible relevance of race-stereotypicality to the OR-specific N200 and P2 Dm effects in Experiment 1, we asked whether these ERPs also tracked variations in consensus race-stereotypicality. ERPs obtained at encoding during Experiment 1 were calculated by sorting trials within each condition according to median splits by rating (i.e., atypical versus stereotypical). N200 comparisons were performed using mastoid-referenced data for mean amplitudes measured at electrode Cza from 200 to 240 ms within each condition. P2 comparisons were performed using average-referenced data by measuring mean amplitudes at T2i and T1i from 200 to 240 ms in each condition.
As shown in Figure 6, OR faces elicited a pronounced, topographically widespread negativity for atypical relative to stereotypical faces. Whereas frontocentral N200 potentials to SR faces were not influenced by race-stereotypicality [F(1,17) = 0.11, p = 0.74], N200 potentials to OR1 faces showed a marginal effect in that they tended to be greater for atypical relative to stereotypical faces [F(1,17) = 3.33, p = 0.086]. This negative shift continued for several hundred milliseconds. N200s to OR5 faces were significantly greater for atypical relative to stereotypical faces [F(1,17) = 8.81, p = 0.009]. In short, the OR-selective sensitivity of frontocentral N200 potentials to subsequent memory (more negative for later-remembered than later-forgotten faces) was paralleled by the sensitivity of these potentials to the degree to which OR faces possessed features that were consistently rated by others as race-atypical. The same pattern occurred in occipito-temporal P2 potentials, which showed a main effect of stereotypicality for OR1 [F(1,17) = 6.01, p = 0.025] and OR5 faces [F(1,17) = 5.91, p = 0.026], but not for SR faces [F(1,17) = 1.63, p = 0.22]. No significant Stereotypicality × Hemisphere interactions were present in any condition [p’s > 0.27].
FIGURE 6
Figure 6. Consensus race-typicality effects on midline ERPs. Trials were divided via a median split into high stereotypical faces and low stereotypical faces for SR, OR1, and OR5 conditions (left, center, and right, respectively). (A) Waveforms for the same five midline electrode locations depicted in Figure 3. (B) Waveforms from right-occipito-temporal electrode T2i, computed using averaged-reference data.
A further analysis focused on effects of consensus distinctiveness. Corresponding waveforms are shown in Figure 7. For OR1 faces, a negative deflection between 200 and 240 ms was more negative for distinctive relative to non-distinctive faces. Unlike the frontocentral Dm and race-stereotypicality effect found in this latency, the distinctiveness effect had a posterior focus, largest at electrode Pzs [F(1,17) = 7.2, p = 0.02 for the distinctive > non-distinctive comparison at this electrode]. No effect of distinctiveness was found in the N200 range for SR or OR5 faces at either Pzs [F(1,17) = 0.04, p = 0.84 for SR faces; F(1,17) = 0.01, p = 0.93 for OR5 faces] or Cza [F(1,17) = 0.34, p = 0.57 for SR faces; F(1,17) < 0.001, p > 0.99 for OR5 faces]. Analyses of P2 potentials revealed no significant effects of distinctiveness or distinctiveness × hemisphere interactions in any condition (all p’s > 0.14). Thus, although distinctiveness correlated significantly with memorability for faces in all three conditions, distinctiveness ratings related to N200 potentials only for OR1 faces, and did not relate to P2 potentials in any condition. These findings suggest that the relationship between N200 and P2 potentials and subsequent memory for OR faces does not arise by virtue of an association between these potentials and the detection of distinctiveness. Distinctiveness and race-stereotypicality were positively correlated for OR1 faces only, suggesting that N200 potentials for OR1 faces may be present in the distinctiveness contrast indirectly because of their association with race-stereotypicality.
FIGURE 7
Figure 7. Consensus distinctiveness effects on midline ERPs. Trials were divided via a median split into high-distinctiveness and low-distinctiveness categories for SR, OR1, and OR5 conditions (left, center, and right, respectively). (A) Waveforms for the same five midline electrode locations depicted in Figures 3 and 6. (B) Waveforms from right-occipito-temporal electrode T2i, computed using averaged-reference data.
General Discussion
Previous studies have examined electrophysiological correlates of memory formation with faces (Sommer et al., 1991, 1995, 1997; Yovel and Paller, 2004; Guo et al., 2005) but the present study is the first to contrast results for SR versus OR faces. Robust late positive Dm effects believed to reflect elaborative encoding occurred for SR faces, as previously observed for various stimuli, including faces, words, and common objects. Late positive Dm effects for OR faces were less robust and less extended in time, consistent with behavioral findings that elaborative encoding strategies are less effective for OR faces (Rhodes et al., 2009; Stahl et al., 2010). Most intriguingly, Dm effects for OR faces included frontocentral N200 potentials previously linked to attention-dependent individuation. When data were analyzed using an average reference instead of a mastoid reference, Dm effects were evident in occipito-temporal P2 potentials, which have also been linked to face individuation. Whereas N200 and P2 amplitudes were greater for SR relative to OR faces overall, these potentials were larger for later-remembered relative to later-forgotten OR faces only. These findings can be understood in light of the proposal that individuating information was encoded dependably for SR faces, but in a weaker and more variable manner for OR faces.
Consensus ratings of race-stereotypicality from a separate group of participants shed additional light on the key ERP results that frontocentral N200 potentials and occipito-temporal P2 potentials predicted later memory for OR faces. The OR faces rated as least stereotypical by broad consensus also yielded larger N200 and P2 amplitudes. Moreover, race-atypical OR1 faces were better remembered than were stereotypical OR1 faces. These findings suggest that, to the extent that a face resembles the stereotypical appearance of an OR individual, N200 and P2 potentials at encoding will be smaller and subsequent recognition less likely. This pattern does not reflect an overall sensitivity of these potentials to facial distinctiveness per se, because the ERP contrast for consensus distinctiveness did not yield differences in these potentials. Rather, these potentials correlated specifically with the extent to which an OR face did not possess physical features considered to be race-stereotypical. These findings directly support previous claims that both P2 and N200 enhancements to SR relative to OR faces reflect greater abilities and/or tendencies to process SR faces beyond the social-category level, orienting instead toward individuating information that enhances subsequent recognition. The other-race effect can thus be explained as a consequence of this failure to detect or orient toward individuating information in OR faces.
As previously mentioned, both perceptual-expertise and social-cognitive theories of the other-race effect have been proposed. How do the present findings fit with these accounts? A prominent perceptual-expertise model is the multidimensional face-space model (Valentine, 1991), which suggests that faces are coded along multiple perceptual dimensions that evolve in accordance with lifelong experience to optimally discriminate among faces. When people’s exposure to faces is predominantly limited to their own race, their “face space” evolves to emphasize the dimensions that best discriminate among SR faces. Consequently, these coding dimensions when applied to OR faces could correspond to race-specifying rather than individuating features. With respect to the present results, a set of dimensions optimized (in accordance with this model) for SR faces would be less reliably able to capture important variations among OR faces that would allow each face be encoded in a way that discriminates it from other faces of the same racial group. It is precisely the detection of this unique individuating information that we hypothesize to be reflected in N200 and P2 potentials. Furthermore, because race-atypical OR faces may be more likely to vary along dimensions that other OR faces have in common, they would be expected to elicit greater N200 and P2 potentials and to be better remembered than stereotypical OR faces.
Despite the ease with which perceptual-expertise models can accommodate these results, social-cognitive explanations also remain plausible. Importantly, N200 potentials are only sensitive to visual novelty when it occurs along task-relevant or attended dimensions (Folstein and Van Petten, 2008). Prominent social-cognitive accounts of the other-race effect suggest that attention is directed away from individuating information within OR faces because perceivers rapidly and automatically categorize OR faces by race (e.g., Hugenberg and Sacco, 2008). There is evidence that category stereotypes are activated during encounters with OR faces, as stereotype-congruent concepts are processed more fluently following such encounters (Macrae and Bodenhausen, 2000; Eberhardt et al., 2004; Wheeler and Fiske, 2005). Furthermore, stereotype-congruent biases are found during and after relevant encounters in judgments regarding many attributes, including racial stereotypes (Corneille et al., 2004), facial affect (Hugenberg and Bodenhausen, 2003), and skin tone (Levin and Banaji, 2006). Strikingly, when faces are racially ambiguous, merely categorizing a face as belonging to one racial group or another may determine the directionality of these biases (MacLin and Malpass, 2003; but see Rhodes et al., 2010). Thus, social categorization alone could impair OR face encoding by emphasizing race-specifying features. Stereotype activation has been found to be strongest – and therefore most likely to disrupt individuation – in response to OR individuals who appear race-stereotypical (e.g., Blair et al., 2002, 2004; Eberhardt et al., 2004), which could account for the presence of better memory and greater N200 and P2 potentials for race-atypical OR faces in the present study. On the whole, facial individuation, as indexed by these early potentials, may be susceptible to both social-cognitive and perceptual biases. The present findings are not at odds with the notion that both perceptual and social-cognitive factors can contribute to the memory disadvantage for OR faces.
Indeed, the present electrophysiological evidence points to two aspects of encoding that differed for SR and OR faces – an early stage of face individuation and a later stage of elaboration – highlighting the principle that both perceptual and conceptual encoding are relevant for understanding successful memory in general. Furthermore, the N200/P2 measures described here pinpoint the timing of deficient individuation for OR faces and may provide a means to put various theories regarding the other-race effect to further test. For example, some studies have provided apparent support for social-cognitive accounts by demonstrating impaired memory for SR faces categorized as outgroup members relative to SR faces categorized as ingroup members (Bernstein et al., 2007; Shriver et al., 2008). However, it is currently unclear whether these manipulations influence face recognition via the same mechanisms typically operative in the other-race effect. N200 and P2 effects in the present study suggest that the mnemonic fate of OR faces hinged on an early individuation process that presented no obstacle to the encoding of SR faces. If outgroup categorization were shown to lead to reduced N200 and P2 potentials for SR faces, such a finding would bolster claims that outgroup categorization alone can account for poor face individuation. In addition, ERP evidence could also shed light on the attenuation of the other-race effect that has been achieved through manipulations such as expertise training (Tanaka and Pierce, 2009), positive mood induction (Johnson and Fredrickson, 2005), varying facial expressions (Ackerman et al., 2006), and instructing participants to individuate OR faces (Hugenberg et al., 2007; Rhodes et al., 2009). Future research could examine whether improvements in OR face memory engendered by these manipulations are mirrored by increased N200 or P2 potentials, and whether these manipulations alter processing at multiple stages of encoding, such as individuation and/or elaboration.
One limitation of this study is that all participants were Caucasian–American. Although other-race effects have been found in racial minorities when remembering faces and voices, such effects tend to be weaker and less consistent than in racial majorities (Golby et al., 2001; Meissner and Brigham, 2001; Chiao et al., 2006; Perrachione et al., 2010). Minority-group members may be more able and/or more likely to individuate OR faces than are majority-group members, or they may more readily switch encoding strategies to optimize them as needed for different racial categories. Comparisons between majority-White and minority-Non-White perceivers have shown that N200 enhancements to SR relative to OR faces are similar (Dickter and Bartholow, 2007; Willadsen-Jensen and Ito, 2008). Interestingly, however, Willadsen-Jensen and Ito (2008) reported flexible SR/OR effects in a group of Asian–Americans as a function of the context in which faces were viewed; N200s were greater for Asian compared to White faces during blocks that were 80% Asian and 20% White, but were greater for White compared to Asian faces when this ratio was reversed. Thus, these participants appeared to change how they attended to individuating information in order to optimize encoding. By contrast, similar contextual manipulations in other studies did not influence N200s in Caucasian–Americans (Ito and Urland, 2003, 2005). A fruitful direction for future research may thus be to examine ways in which N200 and/or P2 potentials relate to face individuation and subsequent memory in a variety of perceiver and target racial groups in order to paint a more complete picture of the many perceptual and social factors that influence face recognition.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We thank Steven Franconeri for helpful discussion. We also thank Ana Cunningham, Rachael Dory, and Ganesh Thippeswamy for assistance with data collection. A preliminary report on the present findings was made in March 2009 at the annual meeting of the Cognitive Neuroscience Society (Lucas et al., 2009). This research was supported by the National Science Foundation grant BCS-0818912. Portions of the research in this paper use the FERET database of facial images collected under the FERET program, sponsored by the DOD Counterdrug Technology Development Program Office.
Footnotes
- ^In the social-cognitive literature, the term individuation is sometimes used to refer to the retrieval of personalized biographical information and episodic traces from memory (i.e., Macrae and Bodenhausen, 2000). By contrast, here we use individuation to refer to the extraction of person-specific facial information, as is typical in research on face processing (i.e., Mason and Macrae, 2004; Scott et al., 2006; Hugenberg et al., 2007; Tanaka and Pierce, 2009). The exact number or type of dimensions used in the individuation process is unknown. In some studies, individuation has been linked specifically to the processing of configural information rather than isolated facial features (e.g., Mason and Macrae, 2004). However, both configural and featural processing contribute to face recognition, and other-race effects have been found for both types of information (Hayward et al., 2008). Thus, it seems likely that individuation can be accomplished via any physiognomic information – including featural information – that distinguishes a given face from others.
- ^While most studies examining Dm have used linked mastoids or earlobes as a reference, some have used the average of all electrodes across the scalp. In such cases, the same Dm can manifest as a bipolar pattern with ERPs that are more positive to later-remembered relative to later-forgotten stimuli at frontocentral sites, but more negative to later-remembered stimuli at occipito-temporal sites (i.e., Sommer et al., 1991, 1995, 1997).
References
Ackerman, J. M., Shapiro, J. R., Neuberg, S. L., Kenrick, D. T., Becker, D. V., Griskevicius, V., Maner, J. K., and Schaller, M. (2006). They all look the same to me (unless they’re angry). Psychol. Sci. 17, 836–840.
Anthony, T., Copper, C., and Mullen, B. (1992). Cross-racial facial identification: a social cognitive integration. Pers. Soc. Psychol. Bull. 18, 296–301.
Bartlett, J. C., Hurry, S., and Thorley, W. (1984). Typicality and familiarity of faces. Mem. Cognit. 12, 219–228.
Bernstein, M. J., Young, S. G., and Hugenberg, K. (2007). The cross-category effect. Psychol. Sci. 18, 706–712.
Blair, I. V., Judd, C. M., and Fallman, J. L. (2004). The automaticity of race and Afrocentric facial features in social judgments. J. Pers. Soc. Psychol. 87, 763–778.
Blair, I. V., Judd, C. M., Sadler, M. S., and Jenkins, C. (2002). The role of Afrocentric features in person perception: judging by features and categories. J. Pers. Soc. Psychol. 83, 5–25.
Boutsen, L., Humphreys, G. W., Praamstra, P., and Warbrick, T. (2006). Comparing neural correlates of configural processing in faces and objects: an ERP study of the Thatcher illusion. Neuroimage 32, 352–367.
Bower, G. H., and Karlin, M. B. (1974). Depth of processing pictures of faces and recognition memory. J. Exp. Psychol. 103, 751–757.
Burkhardt, A., Blaha, L. M., Jurs, B. S., Rhodes, G., Jeffery, L., Wyatte, D., DeLong, J., and Busey, T. (2010). Adaptation modulates the electrophysiological substrates of perceived facial distortion: support for opponent coding. Neuropsychologia 48, 3743–3756.
Chance, J. E., and Goldstein, A. G. (1981). Depth of processing in response to own- and other-race faces. Pers. Soc. Psychol. Bull. 7, 475–480.
Chiao, J. Y., Heck, H. E., Nakayama, K., and Ambady, N. (2006). Priming race in biracial observers affects visual search for Black and White faces. Psychol. Sci. 17, 387–392.
Chiroro, P., and Valentine, T. (1995). An investigation of the contact hypothesis of the own-race bias in face recognition. Q. J. Exp. Psychol. 48, 879–894.
Corneille, O., Huart, J., Becquart, E., and Brédart, S. (2004). When memory shifts toward more typical category exemplars: accentuation effects in the recollection of ethnically ambiguous faces. J. Pers. Soc. Psychol. 86, 236–250.
Craik, F. I., and Lockhart, R. S. (1972). Levels of processing: a framework for memory research. J. Verb. Learn. Verb. Behav. 11, 671–684.
Dickter, C. L., and Bartholow, B. D. (2007). Racial ingroup and outgroup attention biases revealed by event-related brain potentials. Soc. Cogn. Affect. Neurosci. 2, 189–198.
Eberhardt, J. L., Goff, P. A., Purdie, V. J., and Davies, P. G. (2004). Seeing black: race, crime, and visual processing. J. Pers. Soc. Psychol. 87, 876–893.
Folstein, J. R., and Van Petten, C. (2008). Influence of cognitive control and mismatch on the N2 component of the ERP: a review. Psychophysiology 45, 152–170.
Folstein, J. R., Van Petten, C., and Rose, S. A. (2008). Novelty and conflict in the categorization of complex stimuli. Psychophysiology 45, 467–479.
Furl, N., Phillips, P. J., and O’Toole, A. J. (2002). Face recognition algorithms and the other-race effect: computational mechanisms for a developmental contact hypothesis. Cogn. Sci. 26, 797–815.
Gardiner, J. M., and Rosalind, J. I. (1993). Recognition memory and awareness: an experiential approach. Eur. J. Cognit. Psychol. 5, 337–346.
Gauthier, I., Curran, T., Curby, K. M., and Collins, D. (2003). Perceptual interference supports a non-modular account of face processing. Nat. Neurosci. 6, 428–342.
Golby, A. J., Gabrieli, J. D. E., Chiao, J. Y., and Eberhardt, J. L. (2001). Differential responses in the fusiform region to same-race and other-race faces. Nat. Neurosci. 4, 845–850.
Guo, C., Voss, J. L., and Paller, K. A. (2005). Electrophysiological correlates of forming memories for faces, names, and face-name associations. Cognit. Brain Res. 22, 153–164.
Halit, H., de Haan, M., and Johnson, M. H. (2000). Modulation of event-related potentials by prototypical and atypical faces. Neuroreport 11, 1871–1875.
Hayward, W. G., Rhodes, G., and Schwaninger, A. (2008). An own-race advantage for components as well as configurations in face recognition. Cognition 106, 1017–1027.
He, Y., Johnson, M. K., Dovidio, J. F., and McCarthy, G. (2009). The relation between race-related implicit associations and scalp-recorded neural activity evoked by faces from different races. Soc. Neurosci. 4, 426–442.
Herzmann, G., Schweinberger, S. R., Sommer, W., and Jentzsch, I. (2004). What’s special about personally familiar faces? A multimodal approach. Psychophysiology 41, 688–701.
Hugenberg, K., and Bodenhausen, G. V. (2003). Facing prejudice: implicit prejudice and the perception of facial threat. Psychol. Sci. 14, 640–643.
Hugenberg, K., Miller J., and Claypool, H. M. (2007). Categorization and individuation in the cross-race recognition deficit: toward a solution to an insidious problem. J. Exp. Soc. Psychol. 43, 334–340.
Hugenberg, K., and Sacco, D. F. (2008). Social categorization and stereotyping: how social categorization biases person perception and face memory. Soc. Personal. Psychol. Compass 2, 1052–1072.
Ito, T. A., and Urland, G. R. (2003). Race and gender on the brain: electrocortical measures of attention to the race and gender of multiply categorizable individuals. J. Pers. Soc. Psychol. 85, 616–626.
Ito, T. A., and Urland, G. R. (2005). The influence of processing objectives on the perception of faces: an ERP study of race and gender perception. Cogn. Affect. Behav. Neurosci. 5, 21–36.
Johnson, K. J., and Fredrickson, B. L. (2005). “We all look the same to me”: positive emotions eliminate the own-race bias in face recognition. Psychol. Sci. 16, 875–881.
Judd, C. M., and Park, B. (1988). Out-group homogeneity: judgments of variability at the individual and group levels. J. Pers. Soc. Psychol. 54, 778–788.
Kaufmann, J. M., Schweinberger, S. R., and Burton, A. M. (2009). N250 ERP correlates of the acquisition of face representations across different images. J. Cogn. Neurosci. 21, 625–641.
Kubota, J. T., and Ito, T. A. (2007). Multiple cues in social perception: the time course of processing race and facial expression. J. Exp. Soc. Psychol. 43, 738–752.
Latinus, M., and Taylor, M. J. (2006). Face processing stages: impact of difficulty and the separation of effects. Brain Res. 1123, 179–187.
Lebrecht, S., Pierce, L. J., Tarr, M. J., and Tanaka, J. W. (2009). Perceptual other-race training reduces implicit racial bias. PLoS ONE 4, e4215. doi: 10.1371/journal.pone.0004215
Levin, D. T. (1996). Classifying faces by race: the structure of face categories. J. Exp. Psychol. Learn. Mem. Cogn. 22, 1364–1382.
Levin, D. T. (2000). Race as a visual feature: using visual search and perceptual discrimination tasks to understand face categories and the cross-race recognition deficit. J. Exp. Psychol. Gen. 129, 559–574.
Levin, D. T., and Banaji, M. R. (2006). Distortions in the perceived lightness of faces: the role of race categories. J. Exp. Psychol. Gen. 135, 501–512.
Lucas, H. D., Chiao, J. Y., & Paller, K. A. (2009). “Neural correlates of face memory as a function of race-based attention and categorization,” Poster Presented at the 16th Annual Meeting of the Cognitive Neuroscience Society, San Francisco, CA.
MacLin, O. H., and Malpass, R. S. (2001). Racial categorization of faces: the ambiguous race face effect. Psychol. Publ. Pol. Law 7, 98–118.
Macrae, C. N., and Bodenhausen, G. V. (2000). Social cognition: thinking categorically about others. Annu. Rev. Psychol. 51, 93–120.
Mason, M. F., and Macrae, C. N. (2004). Categorizing and individuating others: the neural substrates of person perception. J. Cogn. Neurosci. 16, 1785–1795.
Meissner, C. A., and Brigham, J. C. (2001). Thirty years of investigating the own-race bias in memory for faces: a meta-analytic review. Psychol. Public Policy Law 7, 3–35.
Mercure, E., Dick, F., and Johnson, M. H. (2008). Featural and configural face processing differentially modulate ERP components. Brain Res. 1239, 162–170.
Minear, M., and Park, D. C. (2004). A lifespan database of adult facial stimuli. Behav. Res. Methods Instrum. Comput.36, 630–633.
Mullen, B., and Hu, L. (1989). Perceptions of ingroup and outgroup variability: a meta-analytic integration. Basic Appl. Soc. Psychol. 10, 233–252.
Ostrom, T. M., and Sedikides, C. (1992). Out-group homogeneity effects in natural and minimal groups. Psychol. Bull. 112, 536–552.
Otten, L. J., Sveen, J., and Quayle, A. H. (2007). Distinct patterns of neural activity during memory formation of nonwords versus words. J. Cogn. Neurosci. 19, 1776–1789.
Paller, K. A., and Kutas, M. (1992). Brain potentials during memory retrieval provide neurophysiological support for the distinction between conscious recollection and priming. J. Cogn. Neurosci. 4, 375–392.
Paller, K. A., Kutas, M., and Mayes, A. R. (1987). Neural correlates of encoding in an incidental learning paradigm. Electroencephalogr. Clin. Neurophysiol. 67, 360–371.
Paller, K. A., and Wagner, A. D. (2002). Observing the transformation of experience into memory. Trends Cogn. Sci. 6, 93–102.
Park, B., and Rothbart, M. (1982). Perception of out-group homogeneity and levels of social categorization: memory for the subordinate attributes of in-group and out-group members. J. Pers. Soc. Psychol. 42, 1051–1068.
Perrachione, T. K., Chiao, J. Y., and Wong, P. C. (2010). Asymmetric cultural effects on perceptual expertise underlie an own-race bias for voices. Cognition 114, 42–55.
Phillips, P. J., Moon, H., Rizvi, S. A., and Rauss, P. J. (2000). The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104.
Phillips, P. J., Wechsler, H., Huang, J., and Rauss, P. (1998). The FERET database and evaluation procedure for face recognition algorithms. Image Vis. Comput. 16, 295–306.
Rhodes, G., Lie, H. C., Ewing, L., Evangelista, E., and Tanaka, J. W. (2010). Does perceived race affect discrimination and recognition of ambiguous-race faces? A test of the sociocognitive hypothesis. J. Exp. Psychol. Learn. Mem. Cogn. 36, 217–223.
Rhodes, G., Locke, V., Ewing, L., and Evangelista, E. (2009). Race coding and the other-race effect in face recognition. Perception 38, 232–241.
Rodin, M. J. (1987). Who is memorable to whom: a study of cognitive disregard. Soc. Cogn. 5, 144–165.
Schweinberger, S. R., Pfütze, E. M., and Sommer, W. (1995). Repetition priming and associative priming of face recognition: evidence from event-related potentials. J. Exp. Psychol. Learn. Mem. Cogn. 21, 722–736.
Schweinberger, S. R., Pickering, E. C., Jentzsch, I., Burton, A. M., and Kaufmann, J. M. (2002). Event-related brain potential evidence for a response of inferior temporal cortex to familiar face repetitions. Cognit. Brain Res. 14, 398–409.
Scott, L. S., Tanaka, J. W., Sheinberg, D. L., and Curran, T. (2006). A reevaluation of the electrophysiological correlates of expert object processing. J. Cogn. Neurosci. 18, 1453–1465.
Shriver, E. R., Young, S. G., Hugenberg, K., Bernstein, M. J., and Lanter, J. R. (2008). Class, race, and the face: social context modulates the cross-race effect in face recognition. Pers. Soc. Psychol. Bull. 34, 260–274.
Sommer, W., Heinz, A., Leuthold, H., Matt, J., and Schweinberger, S. R. (1995). Metamemory, distinctiveness, and event-related potentials in recognition memory for faces. Mem. Cogn. 23, 1–11.
Sommer, W., Komoss, E., and Schweinberger, S. R. (1997). Differential localization of brain systems subserving memory for names and faces with event-related potentials. Electroencephalogr. Clin. Neurophysiol. 102, 192–199.
Sommer, W., Schweinberger, S. R., and Matt, J. (1991). Human brain potential correlates of face encoding into memory. Electroencephalogr. Clin. Neurophysiol. 79, 457–463.
Stahl, J., Wiese, H., and Schweinberger, S. R. (2008). Expertise and own-race bias in face processing: an event-related potential study. Neuroreport 19, 583–587.
Stahl, J., Wiese, H., and Schweinberger, S. R. (2010). Learning task affects ERP-correlates of the own-race bias, but not recognition memory performance. Neuropsychologia 48, 2027–2040.
Tanaka, J. W., and Corneille, O. (2007). Typicality effects in face and object perception: further evidence for the attractor field model. Percept. Psychophys. 69, 619–627.
Tanaka, J. W., and Curran, T. (2001). A neural basis for expert object recognition. Psychol. Sci. 12, 43–47.
Tanaka, J. W., Curran, T., Porterfield, A. L., and Collins, D. (2006). Activation of preexisting and acquired face representations: the N250 event-related potential as an index of face familiarity. J. Cogn. Neurosci. 18, 1488–1497.
Tanaka, J. W., Giles, M., Kremen, S., and Simon, V. (1998). Mapping attractor fields in face space: the atypicality bias in face recognition. Cognition 68, 199–220.
Tanaka, J. W., and Pierce, L. J. (2009). The neural plasticity of other-race face recognition. Cogn. Affect. Behav. Neurosci. 9, 122–131.
Valentine, T. (1991). A unified account of the effects of distinctiveness, inversion, and race in face recognition. Q. J. Exp. Psychol. 43, 161–204.
Van Petten, C., and Senkfor, A. J. (1996). Memory for words and novel visual patterns: repetition, recognition, and encoding effects in the event-related brain potential. Psychophysiology 33, 491–506.
Vokey, J. R., and Read, J. D. (1992). Familiarity, memorability, and the effect of typicality on the recognition of faces. Mem. Cognit. 20, 291–302.
Wagner, A. D., Koutstaal, W., and Schacter, D. L. (1999). When encoding yields remembering: insights from event-related neuroimaging. Philos. Trans. R. Soc. Lond. B Biol. Sci. 354, 1307–1324.
Walker, P. M., and Hewstone, M. (2006). A perceptual discrimination investigation of the own-race effect and intergroup experience. Appl. Cogn. Psychol. 20, 461–475.
Walker, P. M., Silvert, L., Hewstone, M., and Nobre, A. C. (2008). Social contact and other-race face processing in the human brain. Soc. Cogn. Affect. Neurosci. 3, 16–25.
Wheeler, M. E., and Fiske, S. T. (2005). Controlling racial prejudice: social-cognitive goals affect amygdala and stereotype activation. Psychol. Sci. 16, 56–63.
Willadsen-Jensen, E. C., and Ito, T. A. (2008). A foot in both worlds: Asian Americans’ perceptions of Asian, White, and racially ambiguous faces. Group Process. Intergroup Relat. 11, 182–200.
Keywords: facial memory, recognition, social categorization, expertise, EEG, ERPs, other-race effect
Citation: Lucas HD, Chiao JY and Paller KA (2011) Why some faces won’t be remembered: brain potentials illuminate successful versus unsuccessful encoding for same-race and other-race faces. Front. Hum. Neurosci. 5:20. doi: 10.3389/fnhum.2011.00020
Received: 17 November 2010;
Accepted: 16 February 2011;
Published online: 08 March 2011.
Edited by:
Jennifer S. Beer, University of Texas at Austin, USAReviewed by:
Jennifer Kubota, New York University, USAStefan Schweinberger, Friedrich Schiller University, Germany
Copyright: © 2011 Lucas, Chiao and Paller. This is an open-access article subject to an exclusive license agreement between the authors and Frontiers Media SA, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence: Heather D. Lucas, Department of Psychology, Northwestern University, 2029 Sheridan Road, Evanston, IL 60201, USA. e-mail: hdlucas@gmail.com