Acute stress influences neural circuits of reward processing

People often make decisions under aversive conditions such as acute stress. Yet, less is known about the process in which acute stress can influence decision-making. A growing body of research has established that reward-related information associated with the outcomes of decisions exerts a powerful influence over the choices people make and that an extensive network of brain regions, prominently featuring the striatum, is involved in the processing of this reward-related information. Thus, an important step in research on the nature of acute stress’ influence over decision-making is to examine how it may modulate responses to rewards and punishments within reward processing neural circuitry. In the current experiment, we employed a simple reward processing paradigm – where participants received monetary rewards and punishments – known to evoke robust striatal responses. Immediately prior to performing each of two task runs, participants were exposed to acute stress (i.e., cold pressor) or a no stress control procedure in a between-subjects fashion. No stress group participants exhibited a pattern of activity within the dorsal striatum and orbitofrontal cortex (OFC) consistent with past research on outcome processing – specifically, differential responses for monetary rewards over punishments. In contrast, acute stress group participants’ dorsal striatum and OFC demonstrated decreased sensitivity to monetary outcomes and a lack of differential activity. These findings provide insight into how neural circuits may process rewards and punishments associated with simple decisions under acutely stressful conditions.

Introduction

Human decision-making often occurs under stressful conditions. The type of stress exposure may be intrinsic or inherent to the decision itself (e.g., choosing between two desirable, but costly options with important consequences) or extrinsic, a pre-existing state which influences decision-making (e.g., stress exposure leading a person to use drugs as a coping mechanism). Thus, understanding how stress exposure influences decision-making is a topic of great interest. Recent efforts suggest that acute stress can modulate risk-taking in decision-making (Preston et al., 2007; Mather et al., 2009; Porcelli and Delgado, 2009), conditioning (for review, see Shors, 2004), and reinforcement learning critical to guiding future decisions (Cavanagh et al., 2010; Petzold et al., 2010). However, less is known about the impact of stress exposure on the processing of affective outcomes, a critical aspect of decision-making. The goal of the current experiment was to examine the influence of exposure to acute stress on reward-related responses in neural circuitry during the delivery of monetary rewards and punishments.

A rich animal literature has delineated a network of regions involved in processing reward-related information, also used to inform decision-making in the human brain (for review, see Schultz, 2006; Balleine et al., 2007; Haber and Knutson, 2010). This reward-related corticostriatal circuitry consists of prefrontal cortex (PFC) regions such as medial PFC and orbitofrontal cortex (OFC) as well as subcortical limbic regions involved in motivation and affect, including the dorsal and ventral striatum. The multifaceted striatum is of particular importance in coding for the subjective value of reward-related information critical to evaluation of outcomes associated with decisions (for review, see O’Doherty et al., 2004; Delgado, 2007; Rangel et al., 2008). Notably, components of the same reward-related neural circuitry have been implicated as a target of the physiological and neurochemical changes associated with engagement of the stress response.

Two complementary biological systems activated by acute stress exposure may influence brain regions involved in reward processing: the sympatho-adrenomedullary axis (i.e., the sympathetic branch of the autonomic nervous system or ANS) and the hypothalamic-pituitary-adrenal axis (HPA; for review, see Ulrich-Lai and Herman, 2009). In response to stress-related homeostatic disruption, the sympathetic ANS quickly responds with the release of catecholamines (CA; e.g., noradrenaline) from the adrenal medulla and ascending CA neurons in communication with the brainstem. As CA release in the peripheral nervous system promotes rapid excitatory changes within the body that enable an organism to deal with the source of the disruption (i.e., the classic “fight-or-flight” response; Cannon, 1915), signals of homeostatic disruption from the brainstem contribute to activation of the HPA via projections to the paraventricular nucleus of the hypothalamus. Proceeding at a slower pace, HPA activation ultimately results in the release of glucocorticoids from the adrenal cortex (i.e., cortisol in humans, corticosterone in rodents; Lupien et al., 2007).

Overall, the influence of acute stress has been studied in the context of memory and other cognitive processes (Joels et al., 2006), but less is known about the impact of stress on processing of reward-related information. One prominent idea is that stress may promote a shift from goal-oriented decision-making toward habit-based decisions that are insensitive to one’s current environment, and can be maladaptive in some contexts (Schwabe and Wolf, 2011; Schwabe et al., 2012). This is supported by studies highlighting changes in structure and function of striatal regions involved in reward-related learning and habit-based decisions (e.g., Delgado, 2007; Tricomi et al., 2009; Balleine and O’Doherty, 2010). For example, rats exposed to chronic stress exhibit marked degradation of dorsomedial striatum and medial PFC with concurrent augmentation of the dorsolateral striatum associated with sustained habitual responses to stimuli even when altered decision outcomes devalue those responses (Dias-Ferreira et al., 2009). In humans, stress-related reductions in reward-related medial PFC responses have been observed in a task involving monetary rewards or neutral outcomes (Ossewaarde et al., 2011), while exposure to acute stress has been linked to reductions in dorsomedial striatal responses to a primary reward (i.e., food; Born et al., 2009).

The current literature suggests that acute stress may modulate neural systems involved in reward processing, particularly the striatum, but a direct test of this hypothesis in humans has not yet been made. The goal of the current study was to utilize a simple reward processing paradigm known to evoke robust striatal responses to examine the influence of exposure to acute stress on outcome evaluation. A potent secondary reinforcer was used: monetary rewards and punishments. A variant of a card guessing task was employed which involved asking participants to make a choice regarding a hidden number on a virtual “card” (Delgado et al., 2000). When participants guessed correctly, they received a monetary reward. When they guessed incorrectly, they received a monetary punishment. Furthermore, rewards and punishments varied in magnitude (high or low). In past research, performance on this task has been shown to evoke robust fMRI blood-oxygen-level-dependent (BOLD) responses in striatal regions. We hypothesized that the previously characterized differential response between rewards and punishments in the striatum would be reduced after exposure to acute stress.

Materials and Methods

Participants

Thirty-four individuals participated in the study. Two participants were excluded from final data analysis, one due to an MRI equipment failure and the other resulting from a request to withdraw from participation. Thus, final data analysis was performed on 32 participants (16 females, 16 males; mean age = 23.41 years, SD years = 4.07). Participants responded to IRB-approved advertisements describing the study. The advertisements also indicated that compensation would be offered for their time at a rate of $25 per hour. All participants gave informed consent according to the guidelines of the Institutional Review Boards of the University of Medicine and Dentistry of New Jersey and Rutgers University.

Procedure

Stress induction

Participants were exposed to acute stress in a between-subjects fashion using a variant on the traditional cold pressor task, which involves immersion of one’s hand into a container of ice-cold water. It is important to note that although water is not inherently incompatible with the MRI environment, if spilled it can be a threat to sensitive MRI equipment (such as the head coil). Additionally, even in the absence of damage due to a spill water can interfere with MRI signal due to its high proton density (Huettel et al., 2008). In the current experiment, we adapted the cold pressor test to fit the MRI environment. To administer cold pressor stress safely once participants were placed within the MRI, rather than prior to entry, an arm wrap was created from a combination of MRI-compatible dry gelpacs and maintained at a temperature of approximately 4°C. This “cold pressor arm wrap” was placed around the right hand and arm of participants assigned to the acute stress group for 2 min immediately prior to each of the two card guessing tasks. For participants assigned to the no stress group, a similar wrap created from towels (at room-temperature) was applied to control for tactile stimulation of the cold pressor arm wrap prior to each card guessing task. Hereafter, when making reference to the two groups collectively the term “experimental groups” will be used.

Card guessing task

In the card guessing task (adapted from Delgado et al., 2000; Delgado et al., 2003) participants were presented with a virtual “card” upon which a question mark was printed for 2 s, representing a number between 1 and 9 (Figure 1A). Their task was to make a button press during those 2 s indicating whether they believed the number on the card was higher or lower than the number 5 (choice phase). After making their response during the 2 s choice phase, the actual number appeared on the card for 2 s (outcome phase). If participants had made a correct guess, they received a monetary reward. If their guess was incorrect, they received a monetary punishment. Rewards and punishments could be of high or low magnitude (reward: +$5.00 or +$0.50; punishment: −$2.50 or −$0.25). Importantly, values were manipulated to account for increased sensitivity to monetary losses over gains (i.e., loss aversion), thus ensuring that variations in BOLD signal related to rewards were comparable to those associated with punishments (Tversky and Kahneman, 2004). The magnitude of a reward was concurrently presented during the 2 s outcome phase via presentation of five green check marks (high magnitude) or one green check mark (low magnitude) below the card’s indicated number. Similarly, the magnitude of monetary punishments was represented by five red “×” marks (high magnitude) or one red “×” mark (low magnitude). Participants were explicitly informed as to the monetary value associated with each stimulus prior to beginning the task, but actual dollar amounts were not presented during the task (only the check and × marks). A jittered inter-trial-interval followed the outcome phase during which participants viewed a fixation lasting between 10 and 12 s, followed by the next trial.

FIGURE 1

Figure 1. (A) Depiction of the card guessing task. Note, in the example above a correct choice would be “higher than five.” (B) Experimental timeline and cortisol sampling schedule (C = cortisol sample).

Participants engaged in two runs of the card guessing task and were informed that they would receive compensation consistent with their performance (i.e., the outcomes they were presented with) during the card guessing task. Each run involved 40 trials with a total run time of 10 min. Participants were unaware that the outcome of each trial was predetermined such that a balanced presentation of rewards and punishments, as well as high and low magnitudes, was maintained. Thus, of the 40 trials per run 20 were associated with rewards and 20 with punishments, 10 of high/low magnitude for each valence. After completion of the experiment, participants were debriefed as to the actual nature of the task. They then completed a post-experimental questionnaire where they rated subjective stress levels associated with the arm wrap on a seven point Likert scale, as well as how the wrap made them feel (good or bad).

Salivary cortisol measurements

Participants were instructed to avoid eating, drinking (anything other than water), or smoking for 2 h prior to the beginning of the experiment to ensure that saliva samples were untainted. To acquire salivary cortisol data, participants were asked to moisten a Salimetrics Oral Swab (SOS) in their mouths for about 1 min by placing the SOS underneath their tongue. Upon completion of this procedure, the subject withdrew the SOS and the experimenter immediately placed it in an individual centrifuge tube. Three samples were acquired for each participant interspersed throughout the scanning session in approximately 15 min intervals, with the first sample taken after anatomical MRI scans were completed (prior to the first card guessing task). Samples two and three were acquired after each of the two blocks of the card guessing task. Samples were frozen in cold storage at −10°C, packed with dry ice and sent to Salimetrics Laboratory (State College, PA, USA) for duplicate biochemical assay analysis. An experimental timeline and cortisol sampling schedule is presented in Figure 1B. Importantly, female participants were screened for use of oral contraceptives (OC) that might influence cortisol levels (though this information was not used as an exclusionary criterion per se). Although 5 of the 16 female participants did report use of OC, no significant differences in cortisol levels were observed between OC and non-OC participants as measured by repeated-measures ANOVA. Furthermore, when those five participants were excluded from the imaging analysis the significance and directionality of all reported effects remained unchanged.

fMRI Acquisition and Analysis

Imaging was performed on a 3T Siemens Allegra scanner equipped with a fast gradient system for echoplanar imaging. A standard radiofrequency head coil with foam padding was used to restrict participants’ head motion while minimizing discomfort. High-resolution axial images (T1-weighted MPRAGE: 256 × 256 matrix, FOV = 256 mm, 176 1 mm axial slices) were obtained from all subjects. Functional images (single-shot gradient echo EPI sequence; TR = 2000 ms; TE = 25 ms; FOV = 192 cm; flip angle = 80°; matrix = 64 × 64; slice thickness = 3 mm) were acquired during performance on the two card guessing task runs. Data were then preprocessed and analyzed using BrainVoyager QX software (version 2.2, Brain Innovation, Maastricht, Netherlands). Preprocessing involved motion correction (six-parameter, three-dimensional), spatial smoothing (4-mm FWHM), voxel-wise linear detrending, high-pass filtering of frequencies (three cycles per time course) and normalization to Talairach stereotaxic space (Talairach and Tournoux, 1988).

General linear models (GLM) were defined at the single-subject level in which predictors were regressed onto the dependent variable of BOLD changes within the brain. Two separate models were generated. In model 1 (outcome valence only), two predictors modeled the outcome phase of the card guessing task based on whether participants had received a rewarding outcome (gain of money) or punishing outcome (loss of money) after their choice. For model 2 (outcome valence and outcome magnitude), the magnitude of rewards and punishments were included, resulting in a model comprised of four predictors: high magnitude reward, low magnitude reward, high magnitude punishment, and low magnitude punishment. In both models, motion parameters generated during fMRI data preprocessing were included as covariates of no-interest (to control for head motion), as was a missed-trial predictor. Two second-level random effects GLMs were then performed.

Based on the random effects GLMs whole-brain statistical parametric maps were generated. Given a priori patterns of BOLD signal defined by a similar contrasts in past work (for review, see Delgado, 2007) it was thought that a Reward – Punishment contrast would best highlight task-related alterations in BOLD signal in regions of the brain known to be involved in processing reward-related information. Using model 1 (outcome valence only) a whole-brain two-tailed contrast was performed on outcome phase BOLD in which rewards and punishments were received (Reward – Punishment), and the difference in BOLD associated with this contrast was contrasted along the between-subjects factor of experimental group (No Stress vs. Acute Stress). Thus, this analysis highlighted brain regions responsive to outcome valence that significantly differed between experimental groups. In a similar whole-brain analysis using model 2, a contrast of high and low magnitude outcomes across outcome valence was performed ([High Reward + High Punishment] – [Low Reward + Low Punishment]) and the difference in BOLD associated with this contrast was computed along the between-subjects factor of experimental group (No Stress vs. Acute Stress). Therefore, this analysis examined brain regions responsive to the magnitude of monetary outcomes that significantly differed between experimental groups.

The resultant contrast maps were then examined to identify statistically significant clusters of activation at a threshold of p < 0.005, with a contiguity threshold of 5³ mm voxels. Correction for multiple comparisons was verified through the use of cluster-size thresholding (Forman et al., 1995; Goebel et al., 2006). Thus, only clusters of a sufficient extent so as to be associated with a cluster-level false-positive rate of α = 0.05 remained in the analysis. Additionally, an exploratory analysis of the possible role of participants’ sex was performed in a priori regions of interest given previous sex-related effects observed in the literature (e.g., Lighthall et al., 2011). Specifically, parameter estimates were extracted from significant clusters resultant from both contrasts and examined for potential interactions with sex. Importantly, all post hoc tests within each family of analyses were corrected for multiple comparisons via sequential Bonferroni correction (Holm, 1979).

Results

Reaction Time Data

A two-tailed independent t-test was performed to examine differences in reaction time in the card guessing task between experimental groups. No significant difference was observed in reaction times for the acute stress (M = 623.31, SEM = 45.91) vs. no stress (M = 633.77, SEM = 43.81) groups, t(30) = 0.17, p > 0.15, d = 0.06.

Subjective Stress Ratings

Post-experimental subjective ratings of perceived stress experience were examined between acute stress and no stress experimental groups via independent t-tests. These included ratings of how the cold pressor arm wrap made participants feel (good to bad) and how stressful (high to low) the experience was. Compared to the no stress group, the acute stress group rated the arm wrap as feeling significantly worse [t(30) = 4.42, p < 0.001, d = 1.56] and more stressful [t(30) = 3.46, p < 0.01 d = 1.22].

Salivary Cortisol Data

Salivary cortisol data were excluded for three participants, in one case due to a corruption of the samples and in two cases due to an inability to acquire samples during MRI scanning. Thus, cortisol analyses were conducted on 29 of the 33 participants (13 no stress, 16 acute stress). Mean salivary cortisol levels (in nmol/L) for all three samples by experimental group are reported in Table 1. A 3 (Sample 1, 2, or 3) × 2 (Experimental Group: No Stress vs. Stress) repeated-measures ANOVA was performed, but no significant interaction between sample and experimental group was observed, F(2, 54) = 1.77, p = 0.18, ${\text{η}}_p^2=$ 0.061. Area under the curve with respect to increase (AUC_I) was calculated using the trapezoidal method for both experimental groups. This measure is useful in that it represents both time-related changes in salivary cortisol levels as well as the overall intensity of said changes (Pruessner et al., 2003). A one-tailed independent t-test between AUC_I for the experimental groups (No stress vs. Acute Stress) indicated a significant increase in cortisol levels for those participants who were exposed to acute stress, t(27) = 1.78, p < 0.05, d = 0.69 (Figure 2). No significant correlations were observed between cortisol and imaging data presented below.

TABLE 1

Table 1. Mean salivary cortisol levels in nmol/L at baseline, after task run 1, and after task run 2 by experimental group (Mean ± SEM).

FIGURE 2

Figure 2. Salivary cortisol area under the curve with respect to increase (AUC_I) by experimental group. Note, negative AUC_I values (indicating a decrease in salivary cortisol over the course of the experiment) were retained as an “index of decrease” as recommended by Pruessner et al. (2003).

fMRI Results

Outcome valence: reward – punishment by experimental group contrast

In the no stress group, multiple brain regions demonstrated greater BOLD signal associated with the reward – punishment contrast than were observed in the acute stress group (see Table 2). Prominently featuring among these regions were the dorsal striatum (specifically the right caudate nucleus and left putamen) and the left OFC.

TABLE 2

Table 2. Brain regions that demonstrated differences by experimental group (No Stress vs. Acute Stress) for Reward – Punishment and High – Low Magnitude contrasts (p < 0.005, corrected).

In the right caudate, post hoc paired t-tests suggested that BOLD signal in the no stress group was significantly greater for rewards than punishments, t(15) = 5.69, p < 0.001, d = 0.88 (Figures 3A–C). No significant difference was observed in the acute stress group, t(15) = 0.74, p > 0.15, d = 0.08. A similar pattern of BOLD signal was observed in the left putamen [no stress, t(15) = 6.57, p < 0.001, d = 0.73; acute stress, t(15) = 1.24, p > 0.15, d = 0.18] and left OFC [no stress, t(15) = 6.80, p < 0.001, d = 1.15; acute stress, t(15) = 0.37, p > 0.15, d = 0.06; see Figure 4]. Thus, whereas the no stress group demonstrated a clear response to rewards over punishments in these regions, the group that had been exposed to acute stress exhibited a lack of responsiveness to reward-related information. All significant t-tests survived sequential Bonferroni correction.

FIGURE 3

Figure 3. (A) Right caudate and left putamen clusters exhibiting greater BOLD signal for no stress over acute stress participants. (B) Right caudate nucleus outcome valence × experimental group parameter estimates. (C) Left putamen outcome valence × experimental group parameter estimates.

FIGURE 4

Figure 4. (A) Left orbitofrontal cortex cluster exhibiting greater BOLD signal for no stress over acute stress participants. (B) Left orbitofrontal cortex outcome valence × experimental group parameter estimates.

Parameter estimates for these three regions in the acute stress group were then examined in a second analysis for the presence of magnitude-related effects (an orthogonal factor not included in the original contrast) in reward and punishment trials. In the right caudate, post hoc paired t-tests suggested that BOLD signal in the acute stress group was significantly greater for rewards over punishments for outcomes of high magnitude, t(15) = 2.79, p < 0.05, d = 0.31, but not low magnitude, t(15) = −1.37, p > 0.15, d = −0.25. A similar pattern was observed within the left putamen. Acute stress group BOLD differentiated between high magnitude outcomes, t(15) = 2.84, p < 0.05, d = 0.43, but not low magnitude outcomes, t(15) = −0.83, p > 0.15, d = −0.20. Notably, in contrast to the above regions the left OFC in the acute stress group did not significantly differentiate between outcomes of either magnitude [high: t(15) = 1.25, p > 0.15, d = 0.27; low: t(15) = −1.71, p > 0.10, d = −0.34]. All significant t-tests survived sequential Bonferroni correction.

To examine whether or not a difference was present in the stress effect between the two task runs, a region of interest (ROI) analysis was performed investigating right dorsal striatum, left putamen, and left OFC BOLD signal between runs 1 and 2 (using ROIs from the original whole-brain analysis). Parameter estimates extracted from the three aforementioned ROIs were examined via 2 (Run: Run 1 vs. Run 2) × 2 (Outcome Valence: Reward vs. Punishment) × 2 (Experimental Group: No Stress vs. Acute Stress) repeated-measures ANOVA for the purpose of establishing whether or not a difference in BOLD existed as a function of run. No significant interaction was observed between run, experimental group, and outcome valence in the right dorsal striatum, F(1, 30) = 0.001, p > 0.15, ${\text{η}}_p^2=$ 0.000, left putamen, F(1, 30) = 0.77, p > 0.15, ${\text{η}}_p^2=$ 0.025, or left OFC, F(1, 30) = 0.31, p > 0.15, ${\text{η}}_p^2=$ 0.010, suggesting that the previously discussed effects were not different between runs.

Outcome magnitude: high – low by experimental group contrasts

A single brain region was associated with increased BOLD signal for no stress participants in the outcome magnitude contrast: the left inferior frontal gyrus (BA45). Post hoc paired t-tests indicated that no stress participants showed greater BOLD responses to high over low magnitude outcomes (across outcome valence), t(15) = 4.77, p < 0.001, d = 0.76. Acute stress participants, however, demonstrated a trend (which did not survive Bonferroni–Holm correction) toward the reverse pattern – increased BOLD for low over high magnitude outcomes, t(15) = −1.98, p < 0.10, d = −0.38.

Exploratory analyses: sex effects

Salivary cortisol AUC_I was examined via univariate ANOVA for sex-related differences in cortisol increases by experimental group. No significant main effect of sex on salivary cortisol was observed, F(1, 25) = 0.52, p = 0.48, ${\text{η}}_p^2=$ 0.020, nor was a significant sex by experimental group interaction observed, F(1, 25) = 0.03, p = 0.87, ${\text{η}}_p^2=$ 0.001. Parameter estimates extracted from significant clusters in both contrasts were subjected to a series of 2 (Outcome valence: Reward vs. Punishment) × 2 (Experimental Group: No Stress vs. Acute Stress) × 2 (Sex: male vs. female) repeated-measures ANOVAs to explore -the possible role of sex in stress-related differences in processing of reward-related information. In the right caudate a trend towards a significant experimental group × sex interaction was observed, F(1, 28) = 3.27, p < 0.10, ${\text{η}}_p^2=$ 0.105. Post hoc independent t-tests indicate that no stress group female participants exhibited greater BOLD signal overall for all outcomes than did males, t(14) = −2.57, p < 0.05, d = −1.28. In contrast acute stress group males’ BOLD was elevated as compared to the no stress group whereas females’ was reduced, resulting in a non-significant difference between the sexes, t(14) = 0.44, p > 0.15, d = 0.22. No other brain regions exhibited trending or significant sex effects.

Discussion

In this study, we sought to investigate how exposure to acute stress influenced neural responses to monetary rewards and punishments. We used a between-subjects approach and tested performance of participants after application of a cold pressor procedure (acute stress group), compared to a control procedure (no stress group) during two runs of a simple card guessing paradigm previously found to yield robust striatal activation to reward responses (e.g., Delgado et al., 2000). Salivary cortisol data and subjective stress ratings confirmed that the stressor (i.e., cold pressor arm wrap adapted for fMRI) was effective. Participants exposed to acute stress exhibited a marked alteration in neural responses to monetary rewards and punishments. Whereas dorsal striatal BOLD signal within the right caudate nucleus and left putamen differentiated between rewarding and punishing outcomes in no stress participants, this was not the case in acute stress participants. A similar pattern of activity was observed in the left OFC. Notably, high magnitude rewards and punishments were resilient to the stress effect in striatal regions but not within OFC. Taken together, these results suggest that exposure to acute stress affects reward-related processing in the dorsal striatum and OFC.

This study complements and augments a growing literature examining the influence of acute stress on human decision-making by attempting to characterize striatal responses to outcome processing under stress. Previous studies have shown modulation of striatal response under stress using different paradigms and reinforcers. For instance, acute stress-related reductions in putamen responses to primary rewards (food images) have been observed (Born et al., 2009), which complements the outcome processing of secondary reinforcers in the current paradigm observed in both caudate and putamen. The consequences of decreased sensitivity to reward processing is a question for future research, but it is informed by a recent study suggesting that increased life stress and reduced ventral striatum reactivity to rewards (i.e., positive performance feedback) interact to predict low levels of positive affect on a depression scale (Nikolova et al., 2012). This converges with previous behavioral work indicating a reduction in responsiveness to rewards under acute stress (Bogdan and Pizzagalli, 2006) which the current study builds upon with the observation of reductions in reward-related responses in the dorsal striatum after acute stress exposure.

An interesting observation from our study is that the stress modulation effect was observed in the dorsal, but not the ventral, striatum. A null finding, however, should not be interpreted as a lack of stress modulation of ventral striatum responses (in fact, stress-related ventral striatal activation has been observed in a non-reward-related task; Pruessner et al., 2008); rather, it highlights the sensitivity of dorsal striatum activity to stress modulation (e.g., Sinha et al., 2005). The dorsal striatum, particularly the caudate, has often been found to be robustly recruited by the reward paradigm used in the current paper (for review, see Delgado, 2007). Further, the dorsal striatum has been posited to function as an “actor” that maintains information about action-contingent response-reward associations to guide future decisions based on the outcomes of past ones, while the ventral portion a “critic” that predicts possible future rewards (O’Doherty et al., 2004; Tricomi et al., 2004). Thus, by impairing the ability of the dorsal striatum to distinguish between rewarding vs. punishing outcomes, acute stress may interfere with the use of information provided by past decisions to guide future choices.

Within the dorsal striatum itself, a functional subdivision suggests that the medial portion of the dorsal striatum is involved in flexible, goal-oriented, and action-contingent decision-making whereas the lateral portion mediates habitual and stimulus bound decisions (Balleine et al., 2007; Tricomi et al., 2009). In the current experiment, it is plausible that stress-related changes in BOLD signal observed in the dorsomedial striatum (i.e., caudate) and dorsolateral striatum (i.e., putamen) mark the beginning of a shift from goal-directed to habitual processing of decision outcomes, although further work is necessary to test this hypothesis using an affective learning paradigm. The hypothesis is consistent with previous behavioral work in support of stress’ ability to shift decision-related processing from goal-oriented to habitual (i.e., as in instrumental conditioning; Schwabe and Wolf, 2011). Importantly, decreased sensitivity to reward processing in the dorsal striatum may have important clinical applications with respect to decision-making and one’s general affect. For instance, stress- and drug-cue associated alterations in dorsal striatal function have been implicated in relapse in drug/alcohol addiction (Sinha and Li, 2007) and reduced dorsal striatal responses to rewards have been observed in unmedicated individuals suffering from major depressive disorder (Pizzagalli et al., 2009).

Another brain region implicated in processing of reward-related information is the OFC, which in this experiment also exhibited alterations in responsiveness to rewards and punishments. It has been suggested that this region may be involved in outcome evaluation by coding for the subjective value of said decision outcomes (O’Doherty et al., 2001a). For example, increases in OFC BOLD have been observed during delivery of pleasant as compared to aversive gustatory stimuli (O’Doherty et al., 2001b). Although stress-related reductions in brain function during reward processing have been somewhat studied in neighboring prefrontal regions such as the medial PFC (Ossewaarde et al., 2011) OFC has received less attention in this regard, making it an ideal topic for future research. This is especially the case with respect to the effects of stress on drug addiction, as this region may play a role in the inability of addicts to alter their behavior based on likely outcomes or consequences – leading to relapse (Schoenbaum and Shaham, 2008). A notable exception is a recent study suggesting the necessity of concurrent CA and glucocorticoid activation in reductions in OFC sensitivity to reward-related information (e.g., Schwabe et al., 2012).

With respect to the mechanism underlying the findings of the current study, several plausible interpretations can be considered. It has been established that glucocorticoid responses to cold pressor stress are less extreme than have been observed in other stress induction techniques, such as stressors involving a psychosocial component (e.g., McRae et al., 2006; Schwabe et al., 2008). In the current study, this is reflected by mild-to-moderate acute stress group increases in cortisol. In contrast, it is likely that sympathetic ANS activation remains comparable between cold pressor and other forms of stress. Another consideration is that in the current study initial acute stress exposure occurred immediately prior to the first card guessing task, followed 15 min later by a second stress exposure and card guessing task. As the effects of glucocorticoid release in this type of paradigm would likely be genomic (i.e., slow and long-lasting; Sapolsky et al., 2000) it is possible that they did not influence brain function in the first task run. Yet, the observed decrease in striatal and OFC responsiveness to reward-related information was present in both task runs. Further, as stress-related increases in cortisol were modest here it is possible that glucocorticoids did not contribute to the effect at all. Thus, lack of data that can speak to the dynamics of sympathetic ANS activation (e.g., skin conductance or salivary alpha amylase; Rohleder et al., 2004) constitutes a study limitation. While the paradigm employed here was not designed to address these issues, it is likely that contextual factors including the nature and timing of stress exposure and the mode of reward-related information involved in the task play an important role.

Some studies suggest that sex differences may play a role in stress-related alterations in striatal reward processing. For example, studies examining the influence of acute stress on risk-tasking have established fluctuations in dorsal striatal function as a function of gender (Lighthall et al., 2009, 2011). There participants performed the Balloon Analog Risk Task, which involves making a button press to expand a virtual balloon for monetary rewards. With each button press, more money is gained – but at a certain point the balloon will explode. Thus, participants risk losing all winnings if they continue to expand the balloon to gain additional rewards. It was observed that under acute stress males take more risks and exhibit increases in dorsal striatal function, whereas females show the reverse pattern, as compared to no stress participants. In the current study, a trend toward a sex difference along similar lines was also observed in the dorsal striatum – though to a lesser degree. No stress females’ BOLD for outcomes was elevated above males’. While BOLD signals to outcomes did decrease for acutely stressed females and increased for males, the result was more extreme in the Lighthall et al. (2009, 2011) studies. This may relate to the fact that risk-taking tasks such as the balloon task involve anticipation of potential outcomes in addition to an outcome evaluation component, while also requiring participants make complex choices balancing potential rewards against potential punishments. It may be the case that the simple outcome evaluation paradigm used in our study is less sensitive to sex differences than more dynamic and complex risk-taking paradigms.

In sum, this paper used a novel approach to induce stress in the fMRI scanner (the cold pressor arm wrap) and observed that exposure to acute stress modulated reward-related circuitry. Specifically, participants under stress showed decreased differential responses to reward and punishment in the dorsal striatum and OFC. Future studies may try to probe if this decreased differential response is driven by a diminished response to rewards (as previously observed in the literature, e.g., Born et al., 2009) or an increase in sensitivity to negative outcomes. Further, additional research is needed to clarify how neural responses to these distinct reinforcers might influence subsequent decision-making under stress.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This research was supported by funding from the National Institute on Drug Abuse to Mauricio R. Delgado (R01DA027764).

References

Balleine, B. W., Delgado, M. R., and Hikosaka, O. (2007). The role of the dorsal striatum in reward and decision-making. J. Neurosci. 27, 8161–8165.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Balleine, B. W., and O’Doherty, J. P. (2010). Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35, 48–69.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bogdan, R., and Pizzagalli, D. A. (2006). Acute stress reduces reward responsiveness: implications for depression. Biol. Psychiatry 60, 1147–1154.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Born, J. M., Lemmens, S. G., Rutters, F., Nieuwenhuizen, A. G., Formisano, E., Goebel, R., et al. (2009). Acute stress and food-related reward activation in the brain during food choice during eating in the absence of hunger. Int. J. Obes. (Lond.) 34, 172–181.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Cannon, W. B. (1915). Bodily Changes in Pain, Hunger, Fear, and Rage: An Account of Recent Researches into the Function of Emotional Excitement. New York: D. Appleton and Company.

Cavanagh, J. F., Frank, M. J., and Allen, J. J. (2010). Social stress reactivity alters reward and punishment learning. Soc. Cogn. Affect. Neurosci. 6, 311–320.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Delgado, M. R. (2007). Reward-related responses in the human striatum. Ann. N. Y. Acad. Sci. 1104, 70–88.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Delgado, M. R., Locke, H. M., Stenger, V. A., and Fiez, J. A. (2003). Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cogn. Affect. Behav. Neurosci. 3, 27–38.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., and Fiez, J. A. (2000). Tracking the hemodynamic responses to reward and punishment in the striatum. J. Neurophysiol. 84, 3072–3077.

Pubmed Abstract | Pubmed Full Text

Dias-Ferreira, E., Sousa, J. C., Melo, I., Morgado, P., Mesquita, A. R., Cerqueira, J. J., et al. (2009). Chronic stress causes frontostriatal reorganization and affects decision-making. Science 325, 621–625.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., and Noll, D. C. (1995). Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn. Reson. Med. 33, 636–647.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Goebel, R., Esposito, F., and Formisano, E. (2006). Analysis of functional image analysis contest (FIAC) data with brainvoyager QX: from single-subject to cortically aligned group general linear model analysis and self-organizing group independent component analysis. Hum. Brain Mapp. 27, 392–401.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Haber, S. N., and Knutson, B. (2010). The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35, 4–26.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65–70.

Huettel, S. A., Song, A. W., and McCarthy, G. (2008). Functional Magnetic Resonance Imaging, 2nd Edn. Sunderland, MA: Sinauer Associates, Inc.

Joels, M., Pu, Z., Wiegert, O., Oitzl, M. S., and Krugers, H. J. (2006). Learning under stress: how does it work? Trends Cogn. Sci. (Regul. Ed.) 10, 152–158.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lighthall, N. R., Mather, M., and Gorlick, M. A. (2009). Acute stress increases sex differences in risk seeking in the balloon analogue risk task. PLoS ONE 4, e6002. doi:10.1371/journal.pone.0006002

CrossRef Full Text

Lighthall, N. R., Sakaki, M., Vasunilashorn, S., Nga, L., Somayajula, S., Chen, E. Y., et al. (2011). Gender differences in reward-related decision processing under stress. Soc. Cogn. Affect. Neurosci. 7, 476–484.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lupien, S. J., Maheu, F., Tu, M., Fiocco, A., and Schramek, T. E. (2007). The effects of stress and stress hormones on human cognition: implications for the field of brain and cognition. Brain Cogn. 65, 209–237.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mather, M., Gorlick, M. A., and Lighthall, N. R. (2009). To brake or accelerate when the light turns yellow? Stress reduces older adults’ risk taking in a driving game. Psychol. Sci. 20, 174–176.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

McRae, A. L., Saladin, M. E., Brady, K. T., Upadhyaya, H., Back, S. E., and Timmerman, M. A. (2006). Stress reactivity: biological and subjective responses to the cold pressor and Trier Social stressors. Hum. Psychopharmacol. 21, 377–385.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nikolova, Y. S., Bogdan, R., Brigidi, B. D., and Hariri, A. R. (2012). Ventral striatum reactivity to reward and recent life stress interact to predict positive affect. Biol. Psychiatry 72, 157–163.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., and Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

O’Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., and Andrews, C. (2001a). Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 4, 95–102.

CrossRef Full Text

O’Doherty, J., Rolls, E. T., Francis, S., Bowtell, R., and McGlone, F. (2001b). Representation of pleasant and aversive taste in the human brain. J. Neurophysiol. 85, 1315–1321.

Ossewaarde, L., Qin, S., Van Marle, H. J., van Wingen, G. A., Fernandez, G., and Hermans, E. J. (2011). Stress-induced reduction in reward-related prefrontal cortex function. Neuroimage 55, 345–352.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Petzold, A., Plessow, F., Goschke, T., and Kirschbaum, C. (2010). Stress reduces use of negative feedback in a feedback-based learning task. Behav. Neurosci. 124, 248–255.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pizzagalli, D. A., Holmes, A. J., Dillon, D. G., Goetz, E. L., Birk, J. L., Bogdan, R., et al. (2009). Reduced caudate and nucleus accumbens response to rewards in unmedicated individuals with major depressive disorder. Am. J. Psychiatry 166, 702–710.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Porcelli, A. J., and Delgado, M. R. (2009). Acute stress modulates risk taking in financial decision making. Psychol. Sci. 20, 278–283.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Preston, S. D., Buchanan, T. W., Stansfield, R. B., and Bechara, A. (2007). Effects of anticipatory stress on decision making in a gambling task. Behav. Neurosci. 121, 257–263.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pruessner, J. C., Dedovic, K., Khalili-Mahani, N., Engert, V., Pruessner, M., Buss, C., et al. (2008). Deactivation of the limbic system during acute psychosocial stress: evidence from positron emission tomography and functional magnetic resonance imaging studies. Biol. Psychiatry 63, 234–240.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pruessner, J. C., Kirschbaum, C., Meinlschmid, G., and Hellhammer, D. H. (2003). Two formulas for computation of the area under the curve represent measures of total hormone concentration versus time-dependent change. Psychoneuroendocrinology 28, 916–931.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Rangel, A., Camerer, C., and Montague, P. R. (2008). A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 9, 545–556.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Rohleder, N., Nater, U. M., Wolf, J. M., Ehlert, U., and Kirschbaum, C. (2004). Psychosocial stress-induced activation of salivary alpha-amylase: an indicator of sympathetic activity? Ann. N. Y. Acad. Sci. 1032, 258–263.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sapolsky, R. M., Romero, L. M., and Munck, A. U. (2000). How do glucocorticoids influence stress responses? Integrating permissive, suppressive, stimulatory, and preparative actions. Endocr. Rev. 21, 55–89.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schoenbaum, G., and Shaham, Y. (2008). The role of orbitofrontal cortex in drug addiction: a review of preclinical studies. Biol. Psychiatry 63, 256–262.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schultz, W. (2006). Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schwabe, L., Haddad, L., and Schachinger, H. (2008). HPA axis activation by a socially evaluated cold-pressor test. Psychoneuroendocrinology 33, 890–895.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schwabe, L., Tegenthoff, M., Hoffken, O., and Wolf, O. T. (2012). Simultaneous glucocorticoid and noradrenergic activity disrupts the neural basis of goal-directed action in the human brain. J. Neurosci. 32, 10146–10155.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schwabe, L., and Wolf, O. T. (2011). Stress-induced modulation of instrumental behavior: from goal-directed to habitual control of action. Behav. Brain Res. 219, 321–328.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shors, T. J. (2004). Learning during stressful times. Learn. Mem. 11, 137–144.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sinha, R., Lacadie, C., Skudlarski, P., Fulbright, R. K., Rounsaville, B. J., Kosten, T. R., et al. (2005). Neural activity associated with stress-induced cocaine craving: a functional magnetic resonance imaging study. Psychopharmacology (Berl.) 183, 171–180.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sinha, R., and Li, C. S. R. (2007). Imaging stress- and cue-induced drug and alcohol craving: association with relapse and clinical implications. Drug Alcohol Rev. 26, 25–31.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Talairach, J., and Tournoux, P. (1988). Co-Planar Stereotaxic Atlas of the Human Brain. New York: Thieme Medical.

Tricomi, E., Balleine, B. W., and O’Doherty, J. P. (2009). A specific role for posterior dorsolateral striatum in human habit learning. Eur. J. Neurosci. 29, 2225–2232.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tricomi, E. M., Delgado, M. R., and Fiez, J. A. (2004). Modulation of caudate activity by action contingency. Neuron 41, 281–292.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tversky, A., and Kahneman, D. (2004). Advances in Prospect Theory: Cumulative Representation of Uncertainty, Preference, belief, and similarity: Selected writings by Amos Tversky. Cambridge, MA: MIT Press, 673–702.

Ulrich-Lai, Y. M., and Herman, J. P. (2009). Neural regulation of endocrine and autonomic stress responses. Nat. Rev. Neurosci. 10, 397–409.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text