A commentary has been posted on this article:
State-dependent value representation: evidence from the striatum
- Department of Psychology, University of Illinois at Chicago, Chicago, IL, USA
Adaptive motivated behavior requires rapid discrimination between beneficial and harmful stimuli. Such discrimination leads to the generation of either an approach or rejection response, as appropriate, and enables organisms to maximize reward and minimize punishment. Classically, the nucleus accumbens (NAc) and the dopamine projection to it are considered an integral part of the brain’s reward circuit, i.e., they direct approach and consumption behaviors and underlie positive reinforcement. This reward-centered framing ignores important evidence about the role of this system in encoding aversive events. One reason for bias toward reward is the difficulty in designing experiments in which animals repeatedly experience punishments; another is the challenge in dissociating the response to an aversive stimulus itself from the reward/relief experienced when an aversive stimulus is terminated. Here, we review studies that employ techniques with sufficient time resolution to measure responses in ventral tegmental area and NAc to aversive stimuli as they are delivered. We also present novel findings showing that the same stimulus – intra-oral infusion of sucrose – has differing effects on NAc shell dopamine release depending on the prior experience. Here, for some rats, sucrose was rendered aversive by explicitly pairing it with malaise in a conditioned taste aversion paradigm. Thereafter, sucrose infusions led to a suppression of dopamine with a similar magnitude and time course to intra-oral infusions of a bitter quinine solution. The results are discussed in the context of regional differences in dopamine signaling and the implications of a pause in phasic dopamine release within the NAc shell. Together with our data, the emerging literature suggests an important role for differential phasic dopamine signaling in aversion vs. reward.
Introduction
Since Olds and Milner’s (1954) seminal observation that animals will self-administer current to regions of their own brain, behavioral neuroscientists have been captivated by the prospect of brain “reward circuits.” During the intervening years, a strong case has been made for the nucleus accumbens (NAc) and NAc-projecting dopamine neurons in the ventral tegmental area (VTA) as being critical cogs in brain reward circuitry. Although the precise relationship between dopamine and reward is still under debate (Berridge and Robinson, 1998; Wise, 2004; Salamone, 2007; Redgrave et al., 2008; Beeler et al., 2012), it is clear that NAc and dopamine participate in processing rewarding stimuli and the generation of reward-directed actions (Schultz, 2000; Kelley et al., 2005; Fields et al., 2007; Kenny, 2011). With respect to dopamine neurotransmission, there is robust agreement using a variety of tools, that the majority of dopamine neurons increase their firing rate and dopamine concentration increases in NAc in response to unpredicted primary rewards or cues that reliably predict rewards (Schultz, 1998; Roitman et al., 2004; Matsumoto and Hikosaka, 2009; Cohen et al., 2012; McCutcheon et al., 2012). However, there has been comparatively little attention paid to dopamine responses to aversive stimuli. Although it is increasingly recognized that the NAc and dopamine process aversive stimuli, the manner in which such stimuli are encoded by this system remains unclear.
Our behavior is potently modified by both beneficial and harmful outcomes. Inappropriate affective responses are hallmarks of many psychiatric disorders including depression, bipolar, and other mood disorders. For example, in animal models drug-addicted rats will continue to respond for drug even if they must simultaneously endure a foot shock that would normally be considered aversive (Deroche-Gamonet et al., 2004; Vanderschuren and Everitt, 2004). These findings are proposed to be analogous to the insensitivity of human drug addicts to the costs associated with continued drug seeking and taking. While there has been a traditional focus on dopamine and the NAc in behavior associated with beneficial outcomes, it is imperative to gain a further understanding of the nature of NAc-dopamine signaling in aversion and to determine whether these components play as strong a role in rejection responses and avoidance learning as they do in appetitive responses and approach learning.
Definition of Aversion
Like reward (Berridge and Robinson, 2003), aversion is a multi-dimensional construct. Most aversive stimuli are intensely disliked and will motivate avoidance. However, it is important to note that dislike and avoidance are not synonymous. As such, dislike is a hedonic evaluation and is common to all aversive stimuli (Kravitz and Kreitzer, 2012). In other words, to be considered aversive, experience of the stimulus should induce a negative hedonic state. However, this definition is problematic as monitoring an animal’s hedonic state is difficult, and in some cases impossible. When using taste stimuli the well-established method of taste reactivity (Grill and Norgren, 1978) has been used to quantify hedonic evaluation in human and non-human subjects alike (Berridge, 2000; Steiner et al., 2001). In contrast, for other sensory modalities such an evaluation is more difficult to quantify. Emission of ultrasonic vocalizations (increase in 22 kHz or decrease in 50 kHz) is thought to be related to hedonia (Knutson et al., 2002) but the utility of this method in assessing hedonic state over a wide range of situations has not been comprehensively validated. Thus, due to the difficulty in assessing hedonic state, in many studies of aversive stimuli, avoidance is used as a proxy for aversion.
When considering the concept of avoidance there are important differences between the production of a behavior that avoids an aversive event (negative reinforcement) and suppression of a behavior that would lead to an aversive event (absence of punishment). Additionally, omission of an expected reward, disappointment, and subsequent extinction of behavior, can also be dissociated from aversion as defined here, and these events may invoke a different set of learning mechanisms (Redish et al., 2007). These distinctions between psychological constructs (hedonic evaluation, reinforcement, punishment, and disappointment) are important as they are likely sub-served by distinct processes at both the systems and cellular/molecular level (Kravitz and Kreitzer, 2012). Indeed, although there is often good overlap between dislike and avoidance there are instances during which these become dissociated. In summary, aversion and avoidance should not be equated and care should be taken when extrapolating the aversive nature of a particular stimulus from its ability to generate or suppress behavior.
Importantly, modulatory factors including motivational state and learning from previous experience can also have powerful effects on the hedonic evaluation of a stimulus. A striking example of motivational state affecting stimulus evaluation is that of salt appetite. Normally, hypertonic sodium chloride solutions are perceived as aversive and unpalatable. In times of need, however, such as following sodium depletion, these solutions become rewarding rather than aversive (Berridge et al., 1984; Tindell et al., 2006); this shift in hedonic valence is accompanied by changes in neuronal activity evoked by hypertonic sodium chloride solutions in NAc (Loriaux et al., 2011) and ventral pallidum (Tindell et al., 2006). Likewise, history with a hedonic stimulus can alter hedonic reactions to it when next encountered. Sweet solutions normally evoke positive hedonic responses. However, if the taste of a sweet solution is paired with malaise, a conditioned taste aversion (CTA) can develop and the same solution is now met with negative hedonic reactions. This shift in hedonic valence is accompanied by changes in neuronal activity evoked by sweet solutions in NAc (Roitman et al., 2010) and basolateral nucleus of the amygdala (Kim et al., 2010). The ability of motivational state and learning to radically alter the nature of a stimulus should make us wary of assuming the hedonic value of a stimulus. This is particularly relevant to studies performed in anesthetized animals. By its very nature, the anesthetic agent is likely to have dampened the negative hedonic state and thus removed the contribution of the neural circuits that may be of most importance for the aversive experience. In this light, the study of “aversive” stimuli under anesthesia may be fundamentally flawed and should be interpreted with caution. This topic will be returned to in the following paragraphs.
Finally, the temporal nature of aversion can vary and states such as stress and fear may consist of negative hedonic states which persist for long periods. For our purposes, we will focus on discrete stimuli that occur on a timescale of seconds. Specifically, with respect to stress, although many of the aversive experiences we will discuss have also been described as acute stressors and when used chronically produce stress-like symptoms (e.g., dysregulation of hypothalamic-pituitary axis and associated behavioral phenotypes), we will not discuss these data. Instead, we refer the interested reader to excellent reviews on stress, dopamine, and NAc (Marinelli et al., 2006; Nestler and Carlezon, 2006; Koob, 2008).
Here, we focus on how aversion may be encoded by mesolimbic dopamine and the implications for NAc processing. A possible confound when studying the encoding of aversion is that there is relief when aversion is terminated – which is likely to be rewarding. In human subjects, offset of a painful stimulus increases blood flow to the NAc, indicating that this region is activated by relief (Baliki et al., 2010). Thus, we will focus on electrophysiological and electrochemical recordings with sufficient time resolution to correlate changes in activity with the onset and duration of aversive events. We review data and present novel findings that unequivocally demonstrate that classical brain reward circuitry is also exquisitely sensitive to aversive stimuli.
Modulation of Dopamine Cell Firing and Release by Reward
Dopamine neurons within the substantia nigra pars compacta (SNc) and VTA project to dorsal and ventral striatum, respectively. In the vast majority of studies made in primate and rodent subjects, during reward-related stimuli – e.g., presentation of primary reward, reward-predictive cues, and during reward-directed actions (Schultz, 1998; Joshua et al., 2008; Matsumoto and Hikosaka, 2009; Cohen et al., 2012) – these neurons show a fairly homogenous response. That is, the majority of dopamine neurons respond to such stimuli and they do so uniformly by exhibiting brief, high frequency increases in firing rate. This pattern of neural activity is likely to cause transient increases in dopamine concentration within the striatum – which has been empirically demonstrated (Garris et al., 1997; Phillips et al., 2003; Venton et al., 2003; Roitman et al., 2004; Sombers et al., 2009; Owesson-White et al., 2012). Indeed, using the electrochemical technique of fast-scan cyclic voltammetry, which can detect fluctuations in dopamine concentration on a timescale similar to electrophysiological changes in dopamine neural activity, it has been repeatedly demonstrated that primary reward and reward-predictive stimuli evoke brief increases in dopamine concentration (Robinson et al., 2002; Phillips et al., 2003; Roitman et al., 2004; Owesson-White et al., 2008; Stuber et al., 2008; Brown et al., 2011; McCutcheon et al., 2012). Voltammetry has excellent face validity for capturing fluctuations in dopamine concentration that result from transient activations and suppressions of dopamine cell firing (Sombers et al., 2009; Owesson-White et al., 2012). Thus, combining the literature in which either electrophysiological recordings from dopamine neurons or electrochemical recordings of dopamine release were made, the population response of midbrain dopamine neurons to rewarding stimuli appears to be a transient increase in activity.
Modulation of Dopamine Cell Firing by Primary Aversive Stimuli
Relative to the reward literature, there are far fewer examinations of the dopamine neuron response to aversive events. In the studies that have been conducted with aversive stimuli, outcomes are much less uniform than seen when reward-related stimuli are used. As such, aversive events are commonly shown to have both excitatory and inhibitory effects on the firing of midbrain dopamine neurons. These studies are reviewed in Table 1. A clear conclusion on the encoding of aversive stimuli by the firing rate of dopamine neurons is limited by several factors. First, identification of neurons within VTA and SNc as dopaminergic based on electrophysiological characteristics remains somewhat controversial (Ungless and Grace, 2012). Second, responses to aversive stimuli have been characterized in either anesthetized or awake and behaving subjects. As discussed earlier, anesthesia may suppress components of the circuit that, when awake would contribute to the generation of a very different dopamine response (Koulchitsky et al., 2012). Third, a wide variety of aversive stimuli have been used to compare with reward-responses. Aversive stimuli used to date include, shock, air puff, foot or tail pinch, and aversive taste stimuli. These stimuli are transduced along very different sensory pathways. They also differ in their intensities and have been characterized from mildly aversive to noxious/painful. Finally, many studies use cues that have been associated with the occurrence of an aversive event and the cue itself comes to elicits a behavior that protects the animal against the aversive stimulus, e.g., an eye blink. Thus, the heterogeneity in dopamine responses to aversive stimuli to date may represent real heterogeneity among different pools of dopamine neurons (Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; Lammel et al., 2011) but may also reflect the heterogeneity of investigative approaches.
TABLE 1
Table 1. Neuronal responses of midbrain dopamine neurons to aversive stimuli.
Modulation of Dopamine Release by Aversion
Fluctuations in dopamine concentration in dopamine terminal regions overcome some of the limitations of recording neural activity in the ventral midbrain. There is no controversy surrounding the identity of the compound studied when microdialysis or fast-scan cyclic voltammetry are used. Microdialysis, though, lacks the sampling resolution required to resolve changes in dopamine evoked by discrete aversive stimuli. Only a handful of studies have employed fast-scan cyclic voltammetry to measure fluctuations in dopamine concentration evoked by aversion (Table 2) (Roitman et al., 2008; Anstrom et al., 2009; Wheeler et al., 2011; Budygin et al., 2012) and are subject to the issues identified earlier: specifically, stimuli that are not temporally discrete, stimuli that are transduced along different sensory pathways than rewarding stimuli, and studies that are performed in anesthetized animals. Recently, we and others have measured dopamine fluctuations during intra-oral delivery of rewarding and aversive taste stimuli. Intra-oral delivery, when paired with fast-scan cyclic voltammetry, offers several advantages. First, primary taste stimuli can be selected to evoke reliable and stereotypical appetitive and aversive responses which can be quantified using taste reactivity (Grill and Norgren, 1978; Peciña and Berridge, 2000). Second, rewarding and aversive stimuli are transduced via similar sensory machinery – that is, the taste system. Third, the animal’s exposure to a stimulus can be tightly controlled, which is particularly important when studying stimuli, e.g., a bitter solution, that an animal would actively avoid. Thus, in conjunction with fast-scan cyclic voltammetry, dopamine concentration fluctuations on a timescale commensurate with the subject’s sensory experience can be measured. Using different stimuli, we have shown that an appetitive sucrose solution increases while an aversive quinine solution suppresses phasic dopamine concentration fluctuations in the NAc shell subregion (Roitman et al., 2008). Recordings were made in a region identified as a “hedonic hotspot” (Peciña and Berridge, 2005). These effects were replicated and extended to taste solutions that are used as conditioned stimuli. When one flavored sweet solution predicted the delayed opportunity to self-administer cocaine, it acquired aversive properties (Wheeler et al., 2011). This solution also suppressed phasic fluctuations in NAc shell dopamine concentration whereas a differently flavored sweet solution increased NAc shell dopamine. Thus, rewarding taste stimuli increase and aversive taste stimuli suppress phasic fluctuations in NAc shell dopamine concentration – suggesting that reward and aversion both evoke changes in phasic dopamine signaling but in opposite directions. However, in both studies, different taste solutions were compared. Perhaps the most rigorous test of differential encoding of reward and aversion by phasic dopamine would be to use the same stimulus but in each case to change the animal’s hedonic evaluation of that stimulus. We accomplished this using a CTA paradigm. Here, we measured phasic dopamine signaling in the NAc shell during intra-oral delivery of a sucrose solution. However, for half of the rats (Paired), this sucrose solution had been previously paired with a malaise-inducing injection of lithium chloride in a CTA paradigm. This classical conditioning procedure renders the sucrose solution aversive (Roitman et al., 2010) – which we quantified using taste reactivity. As such, responses to an identical taste stimulus can be compared between rats that have undergone the CTA procedure and those that have not (Unpaired).
TABLE 2
Table 2. Phasic dopamine responses to aversive stimuli.
Sucrose Differentially Modulates Phasic Dopamine Concentration Fluctuations Depending on Its Hedonic Value
Male Sprague-Dawley rats (Charles River; n = 15) were used. Two cohorts were dedicated to the CTA experiment and were divided into Paired (n = 5) vs. Unpaired (n = 5) groups. A third group received intra-oral infusions of quinine as a comparison (n = 5). All rats were singly housed under standard housing conditions. Food and water were available ad libitum throughout the experiment. Surgical procedures were identical to Roitman et al. (2008). Briefly, under ketamine/xylazine anesthesia, rats were surgically implanted with intra-oral catheters, a guide cannula directed at the NAc shell, an Ag/AgCl reference wire in the contralateral cortex, and a bipolar stimulating electrode in the midbrain. After recovery from surgery, Paired and Unpaired rats underwent conditioning. Paired rats received 30 intra-oral sucrose infusions (0.3 M; 200 μL; 4 s; 30–90 s inter infusion interval) on Days 1 and 3 followed immediately by an injection of LiCl (0.15 M; 20 mL/kg; i.p.). On Days 2 and 4, this cohort received saline injections (0.9%; 20 mL/kg; i.p.) in their home cages. For Unpaired rats, the procedure was identical except the injection order was reversed so that intra-oral sucrose infusions were followed by saline injections on Days 1 and 3 and LiCl injections were delivered in home cage on Days 2 and 4. Thus, both groups had the same number of sucrose infusions, LiCl, and saline injections, however, Paired rats had sucrose explicitly paired with LiCl whereas Unpaired rats did not. Quinine rats underwent no conditioning sessions. Next, all rats had a carbon fiber electrode lowered into NAc shell and dopamine release was recorded using fast-scan cyclic voltammetry while rats received sucrose (CTA rats) or quinine infusions, under the same schedule as in training. Dopamine concentration was extracted from current-voltage plots using established methods (Heien et al., 2004; Keithley et al., 2010). For CTA rats, 1–5 days after the recording session, taste reactivity to intra-oral sucrose infusions was video taped and movies were scored for positive (tongue protrusions, lateral tongue protrusions), and negative (gapes, forelimb flails, chin rubs) responses consistent with previous reports (Peciña and Berridge, 2000). At the end of the experiment, in all rats, the recording site was lesioned, rats were transcardially perfused and brains were sectioned for post hoc histological confirmation of recording placement.
We (Roitman et al., 2008; Owesson-White et al., 2012) and others (Wightman et al., 2007; Sombers et al., 2009) have reported that phasic dopamine release events occur “spontaneously” without being evoked by any overt stimuli. Here, recordings in the NAc shell captured “spontaneous” dopamine release events (Figures 1A–C). Indeed, as seen in the representative trials in Figure 1, dopamine release events were observed in the seconds prior to intra-oral infusions in examples from all three groups. Intra-oral infusions differentially modulated the frequency with which these events occurred. While quinine delivered to naïve rats (Figure 1A) and sucrose delivered to Paired rats (Figure 1B) suppressed dopamine release events, sucrose delivered to Unpaired rats (Figure 1C) increased their frequency.
FIGURE 1
Figure 1. Opposing effects of aversive and rewarding taste stimuli on dopamine release in NAc shell. Representative trial examples resulting from intra-oral infusions of (A) quinine in naïve rats, (B) sucrose in rats that had experienced sucrose explicitly paired with LiCl-induced malaise, and (C) sucrose in rats that did not have sucrose paired with malaise. Color plots (top panels) show changes in current (color) at different electrode potentials (y-axis) over time (x-axis). Dopamine is distinguished by its characteristic oxidation peak (∼0.6 V; black triangle; green/purple feature). Dopamine concentration traces (lower panels) are extracted from above using principal component analysis. Horizontal bars and dashed vertical lines indicate time of infusion.
As dopamine release events occurred during the pre-infusion epoch, averaging across trials led to a baseline dopamine concentration from which quinine caused a significant decrease (p = 0.032 for pre- vs. infusion epoch; Figure 2A). In CTA rats, sucrose infusions had opposing effects on averaged dopamine concentration relative to the pre-infusion epoch dependent on the conditioning history of the animal (Epoch × CTA interaction, F1,9 = 7.89, p = 0.023; Figure 2B). In Paired rats, which had CTA induced by pairing sucrose with illness, infusions of sucrose caused a significant suppression of dopamine (post hoc Tukey’s test, p = 0.007; Figure 2B, red trace) similar to what we observed with quinine infusions. In contrast, in Unpaired rats we saw a small increase in average dopamine concentration that was not statistically significant (Figure 2B, blue trace). While in the past we have shown that intra-oral sucrose infusions increase average dopamine concentration in the NAc shell (Roitman et al., 2008), the increase was evoked in naïve rats. Using microdialysis, Di Chiara and colleagues have shown that increases in NAc shell dopamine to novel food reward dissipate with repeated exposure (Bassareo and Di Chiara, 1999). Thus, the weak increase observed in response to sucrose in Unpaired rats may be due to their familiarity with the rewarding sucrose solution.
FIGURE 2
Figure 2. Aversive stimuli suppress dopamine release in NAc shell. Averaged dopamine concentration traces showing suppression of dopamine release after quinine (A) and sucrose infusions in Paired rats [(B), red trace] and no change in dopamine release in Unpaired rats [(B), blue trace]. *p < 0.05 pre-infusion vs. infusion epoch.
Conditioned taste aversion rats received a session of sucrose infusions and their orofacial responses were analyzed. In Paired rats, sucrose infusions evoked predominantly negative orofacial movements whilst sucrose evoked predominantly positive orofacial movements in Unpaired rats (Figure 3). These differences were confirmed using Mann–Whitney U-tests: paired rats had both higher negative scores and lower positive scores than Unpaired rats (ps < 0.05). The data clearly demonstrate that while both groups of rats had equal exposure to sucrose and LiCl, a CTA was established only in Paired rats. Importantly, taken together with dopamine concentration fluctuations, the data establish that in this paradigm, the NAc shell dopamine response matches the hedonic value of the stimulus and, when aversive, the taste stimulus suppresses phasic dopamine signaling.
FIGURE 3
Figure 3. Induction of conditioned taste aversion leads to a decrease in the palatability of sucrose. In Paired rats, sucrose infusions evoke more negative and less positive orofacial movements than in Unpaired rats. *p < 0.05 vs. Paired rats.
Electrophysiological recordings from dopamine neurons suggest a heterogeneous response to aversive stimuli – with some studies supporting mostly inhibitory responses (Mirenowicz and Schultz, 1996; Ungless et al., 2004; Cohen et al., 2012) and others supporting the existence of a population of dopamine neurons that are excited by aversive stimuli (Horvitz, 2000; Joshua et al., 2008; Brischoux et al., 2009; Matsumoto and Hikosaka, 2009). Emerging evidence supports anatomical segregation of dopamine neuronal responses (Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; Bromberg-Martin et al., 2010; Lammel et al., 2011) in the midbrain with the conclusion that projection target is a key determinant of each cell’s phenotype and response profile (Lammel et al., 2011). Fast-scan cyclic voltammetry captures fluctuations in dopamine concentration likely caused by phasic changes in electrophysiological activity (e.g., increases and decreases; Garris et al., 1997; Sombers et al., 2009; Owesson-White et al., 2012). As dopamine neurons extensively arborize (Matsuda et al., 2009), cylindrical carbon fiber microelectrodes used for voltammetry likely assay dopamine released from the terminals of different dopamine neurons and thus a net population terminal response. Suppression of phasic dopamine within the NAc shell has now been consistently reported for aversive taste stimuli. This strongly suggests that the population response of NAc shell-projecting dopamine neurons to aversion is that of a decrease in activity.
We have shown here that aversive taste stimuli – those that are innately aversive or acquire aversive properties through conditioning – evoke average decreases in dopamine concentration within the NAc shell subregion. These data replicate (Roitman et al., 2008; Wheeler et al., 2011) and extend previous findings to a CTA paradigm. One difficulty with trying to reconcile studies of reward vs. aversion is that the stimuli used to elicit responses are often qualitatively different and cannot be directly compared. For example, how should an electric shock be treated relative to a sugar pellet? We have circumvented this issue by using taste stimuli, which allow reward and aversion to be studied when stimuli of different hedonic values are conveyed to the central nervous system via the same sensory modality. We deliver solutions directly into the animal’s mouth via intra-oral catheter. Intra-oral delivery gives the experimenter exquisite control over stimulus timing allowing fast neurophysiological or neurochemical events to be correlated with sampling of the stimulus. Furthermore, animals can be exposed to stimuli without requiring a volitional movement thus removing another confound that besets many studies and allowing aversive stimuli that would normally be avoided to be effectively studied.
While we did not assay other striatal dopamine terminal regions, it is possible that responses differ with respect to dopamine terminal locations. Indeed, topographical specificity for responses to reward have been demonstrated (Aragona et al., 2009; Brown et al., 2011; Cacciapaglia et al., 2012). Early studies using microdialysis showed that the dopamine response to foot shock occurs with a greatly different time course in the prefrontal cortex than in the NAc (Abercrombie et al., 1989). Thus, future work will need to consider dopamine terminal sub territories in drawing conclusions about a role for dopamine in both reward and aversion.
Implications of a Pause in Phasic Dopamine Release in the NAc Shell
Pauses in the electrophysiological activity of dopamine neurons likely underlie the pauses in dopamine release events we observed on single trials and the average decrease, relative to baseline, across trials in which rats experienced aversive taste stimuli. These pauses in dopamine release, in turn, are likely to have their strongest effects on D2 receptor-expressing medium spiny neurons (MSNs). D2 receptors are high affinity (Richfield et al., 1989) and are thought to be mostly occupied even during the asynchronous baseline firing of dopamine neurons that characterizes the absence of salient stimuli (Dreyer et al., 2010). Thus, a pause in dopamine release would lead to a reduction in D2 tone as D2 receptors become transiently uncoupled from dopamine. D2 receptor activation suppresses MSN excitability and the absence of D2 tone causes an increase in excitability (Surmeier et al., 2011). This is particularly interesting because there is strong and growing evidence that NAc neurons, and particularly shell neurons, are excited by aversive stimuli (Carlezon and Thomas, 2009). Tail pinch activates a majority of striatal neurons (Williams and Millar, 1990). Intra-oral infusions of aversive taste stimuli, identical to those used here, evoke primarily increases in the firing rate of NAc neurons (Roitman et al., 2005, 2010), particularly in the shell (Wheeler et al., 2008; Loriaux et al., 2011). In addition, D2 receptor activity has a prominent role in shaping the strength and direction of striatal synaptic plasticity and the absence of D2 receptor tone can shift the balance between long-term depression and long-term potentiation (Calabresi et al., 2007; Surmeier et al., 2011). Thus, pauses in dopamine release coupled with excitatory inputs evoked by aversive stimuli can lead to plasticity in D2 receptor-expressing MSNs and contribute to the learning of appropriate responses to aversive events. The focus on D2 receptor-expressing neurons is especially interesting since their increased activity has recently been shown to be aversive and promotes avoidance learning (Kravitz et al., 2012).
Potential Mechanisms for Suppressed Phasic Dopamine Release to Aversive Taste Stimuli
Future work must address the mechanisms by which aversive stimuli in general, and taste stimuli specifically, suppress phasic dopamine signaling. Recent publications have focused on this question. Local GABA neurons that suppress the firing rate of VTA dopamine neurons are excited by foot shock in anesthetized rats (Tan et al., 2012) and air puff in awake mice (Cohen et al., 2012). The rostromedial tegmental nucleus (RMTg) is situated just posterior to the VTA, projects to and inhibits dopamine neurons, and is activated by foot shock (Jhou et al., 2009). Neurons within the lateral habenula are activated in response to aversive stimuli, project to the VTA and the RMTg, and contribute to pauses in the firing rate of dopamine neurons (Benabid and Jeaugey, 1989; Matsumoto and Hikosaka, 2007; Stamatakis and Stuber, 2012). It remains unclear, though, how aversive tastes may suppress phasic dopamine release. The parabrachial nucleus, which is the second central relay in gustatory processing, also contains neurons that increase in activity in response to foot shock, project to the VTA, and suppress dopamine neural activity (Coizet et al., 2010). It will be of considerable interest to determine if aversive taste-responsive parabrachial cells project to the VTA and similarly suppress dopamine neural activity. Finally, NAc neurons project, in part, back to the VTA. We have shown that the kappa opioid agonist salvinorin A suppresses phasic dopamine release (Ebner et al., 2010). Since NAc neurons are mostly excited by aversive taste stimuli, dynorphin release leading to kappa receptor activation remains a strong possibility as well.
Conclusion
Here, we have reviewed literature and presented novel findings detailing the effect of brief aversive stimuli on the neuronal responses of midbrain dopamine neurons and dopamine release in terminal regions. Our data show that in one of these projection sites, NAc shell, the response to aversive stimuli is uniformly a suppression of spontaneous dopamine release. Importantly, the stimuli used were presented in the same modality as rewarding stimuli, which evoke increases in dopamine release. Future work will determine whether these patterns hold true for other projection regions.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This work was supported by NIH grant DA025634 (to Mitchell F. Roitman).
References
Abercrombie, E. D., Keefe, K. A., DiFrischia, D. S., and Zigmond, M. J. (1989). Differential effect of stress on in vivo dopamine release in striatum, nucleus accumbens, and medial frontal cortex. J. Neurochem. 52, 1655–1658.
Anstrom, K. K., Miczek, K. A., and Budygin, E. A. (2009). Increased phasic dopamine signaling in the mesolimbic pathway during social defeat in rats. Neuroscience 161, 3–12.
Aragona, B. J., Day, J. J., Roitman, M. F., Cleaveland, N. A., Wightman, R. M., and Carelli, R. M. (2009). Regional specificity in the real-time development of phasic dopamine transmission patterns during acquisition of a cue-cocaine association in rats. Eur. J. Neurosci. 30, 1889–1899.
Baliki, M. N., Geha, P. Y., Fields, H. L., and Apkarian, A. V. (2010). Predicting value of pain and analgesia: nucleus accumbens response to noxious stimuli changes in the presence of chronic pain. Neuron 66, 149–160.
Bassareo, V., and Di Chiara, G. (1999). Differential responsiveness of dopamine transmission to food-stimuli in nucleus accumbens shell/core compartments. Neuroscience 89, 637–641.
Beeler, J. A., Frazier, C. R. M., and Zhuang, X. (2012). Putting desire on a budget: dopamine and energy expenditure, reconciling reward and resources. Front. Integr. Neurosci. 6:49. doi: 10.3389/fnint.2012.00049
Benabid, A. L., and Jeaugey, L. (1989). Cells of the rat lateral habenula respond to high-threshold somatosensory inputs. Neurosci. Lett. 96, 289–294.
Berridge, K. C. (2000). Measuring hedonic impact in animals and infants: microstructure of affective taste reactivity patterns. Neurosci. Biobehav. Rev. 24, 173–198.
Berridge, K. C., Flynn, F. W., Schulkin, J., and Grill, H. J. (1984). Sodium depletion enhances salt palatability in rats. Behav. Neurosci. 98, 652–660.
Berridge, K. C., and Robinson, T. E. (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369.
Brischoux, F., Chakraborty, S., Brierley, D. I., and Ungless, M. A. (2009). Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc. Natl. Acad. Sci. U.S.A. 106, 4894–4899.
Bromberg-Martin, E. S., Matsumoto, M., and Hikosaka, O. (2010). Dopamine in motivational control. Neuron 68, 815–834.
Brown, H. D., McCutcheon, J. E., Cone, J. J., Ragozzino, M. E., and Roitman, M. F. (2011). Primary food reward and reward-predictive stimuli evoke different patterns of phasic dopamine signaling throughout the striatum. Eur. J. Neurosci. 34, 1997–2006.
Brown, M. T. C., Henny, P., Bolam, J. P., and Magill, P. J. (2009). Activity of neurochemically heterogeneous dopaminergic neurons in the substantia nigra during spontaneous and driven changes in brain state. J. Neurosci. 29, 2915–2925.
Budygin, E. A., Park, J., Bass, C. E., Grinevich, V. P., Bonin, K. D., and Wightman, R. M. (2012). Aversive stimulus differentially triggers subsecond dopamine release in reward regions. Neuroscience 201, 331–337.
Cacciapaglia, F., Saddoris, M. P., Wightman, R. M., and Carelli, R. M. (2012). Differential dopamine release dynamics in the nucleus accumbens core and shell track distinct aspects of goal-directed behavior for sucrose. Neuropharmacology 62, 2050–2056.
Calabresi, P., Picconi, B., Tozzi, A., and Di Filippo, M. (2007). Dopamine-mediated regulation of corticostriatal synaptic plasticity. Trends Neurosci. 30, 211–219.
Carlezon, W. A., and Thomas, M. J. (2009). Biological substrates of reward and aversion: a nucleus accumbens activity hypothesis. Neuropharmacology 56(Suppl. 1), 122–132.
Chiodo, L. A., Antelman, S. M., Caggiula, A. R., and Lineberry, C. G. (1980). Sensory stimuli alter the discharge rate of dopamine (DA) neurons: evidence for two functional types of DA cells in the substantia nigra. Brain Res. 189, 544–549.
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., and Uchida, N. (2012). Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85–88.
Coizet, V., Dommett, E. J., Klop, E. M., Redgrave, P., and Overton, P. G. (2010). The parabrachial nucleus is a critical link in the transmission of short latency nociceptive information to midbrain dopaminergic neurons. Neuroscience 168, 263–272.
Coizet, V., Dommett, E. J., Redgrave, P., and Overton, P. G. (2006). Nociceptive responses of midbrain dopaminergic neurones are modulated by the superior colliculus in the rat. Neuroscience 139, 1479–1493.
Deroche-Gamonet, V., Belin, D., and Piazza, P. V. (2004). Evidence for addiction-like behavior in the rat. Science 305, 1014–1017.
Dreyer, J. K., Herrik, K. F., Berg, R. W., and Hounsgaard, J. D. (2010). Influence of phasic and tonic dopamine release on receptor activation. J. Neurosci. 30, 14273–14283.
Ebner, S. R., Roitman, M. F., Potter, D. N., Rachlin, A. B., and Chartoff, E. H. (2010). Depressive-like effects of the kappa opioid receptor agonist salvinorinA are associated with decreased phasic dopamine release in the nucleus accumbens. Psychopharmacology (Berl.) 210, 241–252.
Fields, H. L., Hjelmstad, G. O., Margolis, E. B., and Nicola, S. M. (2007). Ventral tegmental area neurons in learned appetitive behavior and positive reinforcement. Annu. Rev. Neurosci. 30, 289–316.
Gao, D. M., Jeaugey, L., Pollak, P., and Benabid, A. L. (1990). Intensity-dependent nociceptive responses from presumed dopaminergic neurons of the substantia nigra, pars compacta in the rat and their modification by lateral habenula inputs. Brain Res. 529, 315–319.
Garris, P. A., Christensen, J. R., Rebec, G. V., and Wightman, R. M. (1997). Real-time measurement of electrically evoked extracellular dopamine in the striatum of freely moving rats. J. Neurochem. 68, 152–161.
Grill, H. J., and Norgren, R. (1978). The taste reactivity test. I. Mimetic responses to gustatory stimuli in neurologically normal rats. Brain Res. 143, 263–279.
Guarraci, F. A., and Kapp, B. S. (1999). An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential pavlovian fear conditioning in the awake rabbit. Behav. Brain Res. 99, 169–179.
Heien, M. L. A. V., Johnson, M. A., and Wightman, R. M. (2004). Resolving neurotransmitters detected by fast-scan cyclic voltammetry. Anal. Chem. 76, 5697–5704.
Horvitz, J. C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96, 651–656.
Jhou, T. C., Fields, H. L., Baxter, M. G., Saper, C. B., and Holland, P. C. (2009). The rostromedial tegmental nucleus (RMTg), a GABAergic afferent to midbrain dopamine neurons, encodes aversive stimuli and inhibits motor responses. Neuron 61, 786–800.
Joshua, M., Adler, A., Mitelman, R., Vaadia, E., and Bergman, H. (2008). Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. J. Neurosci. 28, 11673–11684.
Keithley, R. B., Carelli, R. M., and Wightman, R. M. (2010). Rank estimation and the multivariate analysis of in vivo fast-scan cyclic voltammetric data. Anal. Chem. 82, 5541–5551.
Kelley, A. E., Baldo, B. A., Pratt, W. E., and Will, M. J. (2005). Corticostriatal-hypothalamic circuitry and food motivation: integration of energy, action, and reward. Physiol. Behav. 86, 773–795.
Kenny, P. J. (2011). Common cellular and molecular mechanisms in obesity and drug addiction. Nat. Rev. Neurosci. 12, 638–651.
Kim, M. J., Mizumori, S. J. Y., and Bernstein, I. L. (2010). Neuronal representation of conditioned taste in the basolateral amygdala of rats. Neurobiol. Learn. Mem. 93, 406–414.
Kiyatkin, E. A. (1988). Functional properties of presumed dopamine-containing and other ventral tegmental area neurons in conscious rats. Int. J. Neurosci. 42, 21–43.
Kiyatkin, E. A. (1995). Functional significance of mesolimbic dopamine. Neurosci. Biobehav. Rev. 19, 573–598.
Knutson, B., Burgdorf, J., and Panksepp, J. (2002). Ultrasonic vocalizations as indices of affective states in rats. Psychol. Bull. 128, 961–977.
Koulchitsky, S., De Backer, B., Quertemont, E., Charlier, C., and Seutin, V. (2012). Differential effects of cocaine on dopamine neuron firing in awake and anesthetized rats. Neuropsychopharmacology 37, 1559–1571.
Kravitz, A. V., and Kreitzer, A. C. (2012). Striatal mechanisms underlying movement, reinforcement, and punishment. Physiology (Bethesda) 27, 167–177.
Kravitz, A. V., Tye, L. D., and Kreitzer, A. C. (2012). Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat. Neurosci. 15, 816–818.
Lammel, S., Ion, D. I., Roeper, J., and Malenka, R. C. (2011). Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70, 855–862.
Loriaux, A. L., Roitman, J. D., and Roitman, M. F. (2011). Nucleus accumbens shell, but not core, tracks motivational value of salt. J. Neurophysiol. 1537–1544.
Maeda, H., and Mogenson, G. J. (1982). Effects of peripheral stimulation on the activity of neurons in the ventral tegmental area, substantia nigra, and midbrain reticular formation of rats. Brain Res. Bull. 8, 7–14.
Mantz, J., Thierry, A. M., and Glowinski, J. (1989). Effect of noxious tail pinch on the discharge rate of mesocortical and mesolimbic dopamine neurons: selective activation of the mesocortical system. Brain Res. 476, 377–381.
Marinelli, M., Rudick, C. N., Hu, X.-T., and White, F. J. (2006). Excitability of dopamine neurons: modulation and physiological consequences. CNS Neurol. Disord. Drug Targets 5, 79–97.
Matsuda, W., Furuta, T., Nakamura, K. C., Hioki, H., Fujiyama, F., Arai, R., and Kaneko, T. (2009). Single nigrostriatal dopaminergic neurons form widely spread and highly dense axonal arborizations in the neostriatum. J. Neurosci. 29, 444–453.
Matsumoto, M., and Hikosaka, O. (2007). Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447, 1111–1115.
Matsumoto, M., and Hikosaka, O. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841.
McCutcheon, J. E., Beeler, J. A., and Roitman, M. F. (2012). Sucrose-predictive cues evoke greater phasic dopamine release than saccharin-predictive cues. Synapse 66, 346–351.
Mileykovskiy, B., and Morales, M. (2011). Duration of inhibition of ventral tegmental area dopamine neurons encodes a level of conditioned fear. J. Neurosci. 31, 7471–7476.
Mirenowicz, J., and Schultz, W. (1996). Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379, 449–451.
Nestler, E. J., and Carlezon, W. A. (2006). The mesolimbic dopamine reward circuit in depression. Biol. Psychiatry 59, 1151–1159.
Olds, J., and Milner, P. (1954). Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J. Comp. Physiol. Psychol. 47, 419–427.
Owesson-White, C. A., Cheer, J. F., Beyene, M., Carelli, R. M., and Wightman, R. M. (2008). Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation. Proc. Natl. Acad. Sci. U.S.A. 105, 11957–11962.
Owesson-White, C. A., Roitman, M. F., Sombers, L. A., Belle, A. M., Keithley, R. B., Peele, J. L., Carelli, R. M., and Wightman, R. M. (2012). Sources contributing to the average extracellular concentration of dopamine in the nucleus accumbens. J. Neurochem. 121, 252–262.
Park, J., Wheeler, R. A., Fontillas, K., Keithley, R. B., Carelli, R. M., and Wightman, R. M. (2012). Catecholamines in the bed nucleus of the stria terminalis reciprocally respond to reward and aversion. Biol. Psychiatry 71, 327–334.
Peciña, S., and Berridge, K. C. (2000). Opioid site in nucleus accumbens shell mediates eating and hedonic “liking” for food: map based on microinjection Fos plumes. Brain Res. 863, 71–86.
Peciña, S., and Berridge, K. C. (2005). Hedonic hot spot in nucleus accumbens shell: where do mu-opioids cause increased hedonic impact of sweetness? J. Neurosci. 25, 11777–11786.
Phillips, P. E. M., Stuber, G. D., Heien, M. L. A. V., Wightman, R. M., and Carelli, R. M. (2003). Subsecond dopamine release promotes cocaine seeking. Nature 422, 614–618.
Redgrave, P., Gurney, K., and Reynolds, J. (2008). What is reinforced by phasic dopamine signals? Brain Res. Rev. 58, 322–339.
Redish, A. D., Jensen, S., Johnson, A., and Kurth-Nelson, Z. (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol. Rev. 114, 784–805.
Richfield, E. K., Penney, J. B., and Young, A. B. (1989). Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience 30, 767–777.
Robinson, D. L., Heien, M. L., and Wightman, R. M. (2002). Frequency of dopamine concentration transients increases in dorsal and ventral striatum of male rats during introduction of conspecifics. J. Neurosci. 22, 10477–10486.
Roitman, M. F., Stuber, G. D., Phillips, P. E. M., Wightman, R. M., and Carelli, R. M. (2004). Dopamine operates as a subsecond modulator of food seeking. J. Neurosci. 24, 1265–1271.
Roitman, M. F., Wheeler, R. A., and Carelli, R. M. (2005). Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45, 587–597.
Roitman, M. F., Wheeler, R. A., Tiesinga, P. H. E., Roitman, J. D., and Carelli, R. M. (2010). Hedonic and nucleus accumbens neural responses to a natural reward are regulated by aversive conditioning. Learn. Mem. 17, 539–546.
Roitman, M. F., Wheeler, R. A., Wightman, R. M., and Carelli, R. M. (2008). Real-time chemical responses in the nucleus accumbens differentiate rewarding and aversive stimuli. Nat. Neurosci. 11, 1376–1377.
Salamone, J. D. (2007). Functions of mesolimbic dopamine: changing concepts and shifting paradigms. Psychopharmacology (Berl.) 191, 389.
Schultz, W., and Romo, R. (1987). Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J. Neurophysiol. 57, 201–217.
Sombers, L. A., Beyene, M., Carelli, R. M., and Wightman, R. M. (2009). Synaptic overflow of dopamine in the nucleus accumbens arises from neuronal activity in the ventral tegmental area. J. Neurosci. 29, 1735–1742.
Stamatakis, A. M., and Stuber, G. D. (2012). Activation of lateral habenula inputs to the ventral midbrain promotes behavioral avoidance. Nat. Neurosci. 15, 1105–1107.
Steiner, J. E., Glaser, D., Hawilo, M. E., and Berridge, K. C. (2001). Comparative expression of hedonic impact: affective reactions to taste by human infants and other primates. Neurosci. Biobehav. Rev. 25, 53–74.
Stuber, G. D., Klanker, M., Ridder, B. D., Bowers, M. S., Joosten, R. N., Feenstra, M. G., Bonci, A., and de Ridder, B. (2008). Reward-predictive cues enhance midbrain dopamine neurons. Science 321, 1690–1692.
Surmeier, D. J., Carrillo-Reid, L., and Bargas, J. (2011). Dopaminergic modulation of striatal neurons, circuits, and assemblies. Neuroscience 198, 3–18.
Tan, K. R., Yvon, C., Turiault, M., Mirzabekov, J. J., Doehner, J., Labouèbe, G., Deisseroth, K., Tye, K. M., and Lüscher, C. (2012). GABA neurons of the VTA drive conditioned place aversion. Neuron 73, 1173–1183.
Tindell, A. J., Smith, K. S., Peciña, S., Berridge, K. C., and Aldridge, J. W. (2006). Ventral pallidum firing codes hedonic reward: when a bad taste turns good. J. Neurophysiol. 96, 2399–2409.
Ungless, M. A., and Grace, A. A. (2012). Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons. Trends Neurosci. 35, 422–430.
Ungless, M. A., Magill, P. J., and Bolam, J. P. (2004). Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303, 2040–2042.
Vanderschuren, L. J. M. J., and Everitt, B. J. (2004). Drug seeking becomes compulsive after prolonged cocaine self-administration. Science 305, 1017–1019.
Venton, B. J., Zhang, H., Garris, P. A., Phillips, P. E. M., Sulzer, D., and Wightman, R. M. (2003). Real-time decoding of dopamine concentration changes in the caudate-putamen during tonic and phasic firing. J. Neurochem. 87, 1284–1295.
Wang, D. V., and Tsien, J. Z. (2011). Convergent processing of both positive and negative motivational signals by the VTA dopamine neuronal populations. PLoS ONE 6, e17047. doi:10.1371/journal.pone.0017047
Wheeler, R. A., Aragona, B. J., Fuhrmann, K. A., Jones, J. L., Day, J. J., Cacciapaglia, F., Wightman, R. M., and Carelli, R. M. (2011). Cocaine cues drive opposing context-dependent shifts in reward processing and emotional state. Biol. Psychiatry 69, 1067–1074.
Wheeler, R. A., Twining, R. C., Jones, J. L., Slater, J. M., Grigson, P. S., and Carelli, R. M. (2008). Behavioral and electrophysiological indices of negative affect predict cocaine self-administration. Neuron 57, 774–785.
Wightman, R. M., Heien, M. L. A. V., Wassum, K. M., Sombers, L. A., Aragona, B. J., Khan, A. S., Ariansen, J. L., Cheer, J. F., Phillips, P. E. M., and Carelli, R. M. (2007). Dopamine release is heterogeneous within microenvironments of the rat nucleus accumbens. Eur. J. Neurosci. 26, 2046–2054.
Williams, G. V., and Millar, J. (1990). Concentration-dependent actions of stimulated dopamine release on neuronal activity in rat striatum. Neuroscience 39, 1–16.
Zweifel, L. S., Fadok, J. P., Argilli, E., Garelick, M. G., Jones, G. L., Dickerson, T. M. K., Allen, J. M., Mizumori, S. J. Y., Bonci, A., and Palmiter, R. D. (2011). Activation of dopamine neurons is critical for aversive conditioning and prevention of generalized anxiety. Nat. Neurosci. 14, 620–626.
Keywords: voltammetry, electrophysiology, conditioned taste aversion, taste reactivity, reward, ventral tegmental area
Citation: McCutcheon JE, Ebner SR, Loriaux AL and Roitman MF (2012) Encoding of aversion by dopamine and the nucleus accumbens. Front. Neurosci. 6:137. doi: 10.3389/fnins.2012.00137
Received: 31 July 2012; Accepted: 04 September 2012;
Published online: 25 September 2012.
Edited by:
Philippe N. Tobler, University of Zurich, SwitzerlandReviewed by:
A. David Redish, University of Minnesota, USAMark Ungless, Imperial College London, UK
Copyright: © 2012 McCutcheon, Ebner, Loriaux and Roitman. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.
*Correspondence: Mitchell F. Roitman, Department of Psychology, University of Illinois at Chicago, 1007 W Harrison Street, Chicago, IL 60607, USA. e-mail: mroitman@uic.edu