Adrian G. Fischer1,2*
Markus Ullsperger1,2- 1Department of Neuropsychology, Institute of Psychology, Otto-von-Guericke University, Magdeburg, Germany
- 2Center for Behavioral Brain Sciences, Magdeburg, Germany
The specific role of serotonin and its interplay with dopamine (DA) in adaptive, reward guided behavior as well as drug dependance, still remains elusive. Recently, novel methods allowed cell type specific anatomical, functional and interventional analyses of serotonergic and dopaminergic circuits, promising significant advancement in understanding their functional roles. Furthermore, it is increasingly recognized that co-release of neurotransmitters is functionally relevant, understanding of which is required in order to interpret results of pharmacological studies and their relationship to neural recordings. Here, we review recent animal studies employing such techniques with the aim to connect their results to effects observed in human pharmacological studies and subjective effects of drugs. It appears that the additive effect of serotonin and DA conveys significant reward related information and is subjectively highly euphorizing. Neither DA nor serotonin alone have such an effect. This coincides with optogenetically targeted recordings in mice, where the dopaminergic system codes reward prediction errors (PE), and the serotonergic system mainly unsigned PE. Overall, this pattern of results indicates that joint activity between both systems carries essential reward information and invites parallel investigation of both neurotransmitter systems.
Introduction
Among the brain’s neuromodulators, serotonin (5-hydroxytryptamine, 5-HT) is likely the most ambivalent one with regard to its supposed importance for behavior and level of understanding. Serotonergic drugs are widely used in psychiatric disorders, abused as recreational drugs and liabilities in the serotonergic system have been identified as important etiological factors in many prevalent disorders. On the other hand, 5-HT is commonly described as “mysterious” in scientific contexts (Daw et al., 2002; Luo et al., 2015; Li et al., 2016), reflecting the fact that a unifying function of its physiological role has not been established and data are often inconclusive. Although an involvement of 5-HT in rewarding and aversive processing, hedonic experience, mood and higher cognitive functions such as consciousness or self reflection are undisputed, its precise contribution is controversial. These ambiguities reach deep into the history of studies on the serotonergic system. It has long been known from animal studies across different species that the raphe nuclei, the origin of most of the forebrain’s serotonergic innervation, are among the most potent areas inducing self stimulation equivalent to stimulation of the medial forebrain bundle or the ventral tegmental area (VTA; Miliaressis et al., 1975; Van Der Kooy et al., 1978; Rompre and Miliaressis, 1985). On the other hand, influential theories of 5-HT functioning position it as a behavioral inhibitor (Soubrié, 1986) essential in facilitating aversive processing (Tye et al., 1977), therefore, opposing the role ascribed to dopamine (DA; Deakin and Graeff, 1991; Daw et al., 2002; Cools et al., 2008a; Dayan and Huys, 2009). These partly contradictory functions may be explained by interactions between different cell types within the raphe that depend on serotonin.
Here, we will outline the role of different cell types in raphe nuclei and the cell-type specific anatomy of the serotonergic system. We first review recent studies on serotonin’s role in human and animal behavior related to rewarding and aversive processing. We then relate these findings to novel results in which serotonergic and non-serotonergic signals including co-release of other transmitters related to the 5-HT system have been investigated via genetic targeting approaches. We thereafter focus on self-stimulation studies as a measure of reward in animals, that offers the possibility to relate findings to human behavior including subjectively experienced reward induced by drugs. In conclusion, we will summarize the importance of considering different cells and the interplay of neuromodulatory systems and neurotransmitter co-release when discussing the role of neuromodulators like 5-HT and DA.
The Serotonergic System
The density of 5-HT receptors in the cortex shows a descending rostro-caudal gradient, indicative of an especially prominent involvement of 5-HT in higher cognitive functions (Kranz et al., 2010; Celada et al., 2013). Serotonergic afferents are provided by only a small group of cells located in the raphe area of the midbrain. Two major nuclei here consist in the dorsal and median raphe (DRN, MRN) which comprise around 160,000 and 60,000 serotonergic neurons in humans, respectively (Charnay and Léger, 2010). The majority of serotonergic input to the forebrain is provided by the DRN, on which we focus here. There are considerable differences in the percentage of serotonergic neurons within the DRN across species: in humans, 70% of DRN neurons were found to contain 5-HT (Baker et al., 1991), whereas in cats (Léger and Wiklund, 1982) and rats, serotonergic neurons do not constitute the majority of the cell population in the DRN. Of the remaining cells, glutamatergic and GABAergic neurons constitute the majority (Hornung, 2003; Bang and Commons, 2012) and the GABAergic cells mediate inhibitory feedback within the DRN (Liu et al., 2000). Aside from cell type diversity, co-release of neurotransmitters is increasingly recognized as functionally relevant (El Mestikawy et al., 2011). Neurons that express tryptophan hydroxylase 2 (TpH2), the rate-limiting enzyme in 5-HT synthesis that is almost exclusively expressed in serotonergic neurons, have been found to express the vesicular glutamate transporter type 3 (VGluT3) which transports glutamate into presynaptic vesicles in non-primarily glutamatergic neurons (Hioki et al., 2010). Glutamate release from these 5-HT neurons has been repeatedly observed (Johnson, 1994; Liu et al., 2014; Qi et al., 2014). Co-release is by no means restricted to the serotonergic system, but has been acknowledged in dopaminergic neurotransmission as well (Stuber et al., 2010; Tecuapetla et al., 2010). This intrinsic interlacing between neurotransmitter systems suggests that the functional relevance of neuromodulatory systems is difficult to judge when dissecting individual components of a physiological system consistently releasing more than a single neurotransmitter.
A major goal in understanding neuromodulatory systems is to combine correlational evidence, for example from imaging studies (Macoveanu, 2014), with pharmacological interventions, and provide a plausible picture bridging results from direct recordings of specific activation or lesion studies in animals and pharmacological interventions, eventually consistent with self reports in humans. Methods that affect overall serotonergic tone are dietary acute tryptophan depletion (ATD; Fadda et al., 2000) which reduces 5-HT, tryptophan loading which presumably increases 5-HT (Young, 1996), and administration of SSRI which increase 5-HT levels, but could decrease co-release of glutamate (Fischer et al., 2015c). Direct agonists to 5-HT receptors have been used in various studies with complex results, likely due to the diversity of 5-HT receptors (Hayes and Greenshaw, 2011). A major expectation of novel optogenetic methods, that allow to specifically target, record, activate and inhibit monoaminergic neurons, and to dissociate them, for example from inhibitory GABAergic neurons, is to provide evidence for 5-HT’s major physiological function which then could be used to reconcile results of pharmacological studies.
Afferents of the Serotonergic System
Given the relevance of serotonergic neuromodulation for the development of anxiety or depression (Caspi et al., 2003; Sachs et al., 2015), control of serotonergic signaling is highly important. It has only recently become possible to specifically trace inputs to identified serotonergic neurons, differentiating them, for example, from afferent neurons that synapse onto GABAergic cells. This can be done by specifically targeting a tracer to genetically defined cell populations (such as 5-HT neurons) that express a protein required for the tracer to travel retrogradely under a specific promoter active in the cell population. When targeting, for example, the serotonin transporter promoter gene, this technique provides a means to specifically localize monosynaptic inputs to 5-HT neurons in mice. Studies using this technique found that multiple brain regions project monosynaptically to DRN 5-HT neurons (Ogawa et al., 2014; Pollak Dorocic et al., 2014), demonstrating the high level of control exerted over the serotonergic system. Input to defined serotonergic DRN, but not MRN, and defined dopaminergic VTA neurons is quantitatively and hodologically similar (Watabe-Uchida et al., 2012; Ogawa et al., 2014), compatible with strongly complementary functions in both systems. Among these input regions are the PFC and the lateral habenula (LHb), which both provide mainly excitatory input to serotonergic as well as GABAergic neurons in the DRN (Pollak Dorocic et al., 2014; Weissbourd et al., 2014; Geddes et al., 2016; Zhou et al., 2017). The net effect of such an input may be excitatory or mediate feed-forward inhibition. Additionally, another major input pathway to the raphe nuclei includes a mainly GABAergic relay of DRN inputs from LHb via the rostromedial tegmental nucleus (Jhou et al., 2009; Sego et al., 2014).
Inputs from the PFC can modulate and shift the degree of inhibition or excitation in the DRN (Geddes et al., 2016). Functionally, stimulation of PFC neurons projecting to the DRN was found to influence motivation: stimulation of this pathway increased, whereas silencing decreased motivation in rats in the forced-swim task (Warden et al., 2013). Subcortical areas send both excitatory and inhibitory projections onto serotonergic neurons, constituting a push-pull regulatory mechanism (Zhou et al., 2017). The LHb has been suggested to mediate aversive signals transmitted to the dopaminergic and serotonergic system (Matsumoto and Hikosaka, 2007; Hikosaka, 2010). Electrical stimulation of the LHb inhibits DRN cell firing (Wang and Aghajanian, 1977) and LHb lesions increase DRN 5-HT levels (Yang et al., 2008) in rats. This up-stream input to the DRN could be integrated by GABAergic neurons that control serotonergic neuronal activity. Functional relevance of this interplay between GABAergic and serotonergic DRN neurons has been demonstrated for the expression of avoidance following social defeat stress in mice. Following repeated exposure to dominant conspecifics, GABAergic neurons in the DRN increased in excitability which led to decreased 5-HT activity (Challis et al., 2013). Optogenetic silencing of GABAergic DRN neurons prevented expression of behavioral avoidance, indicative of a causative influence of GABAergic DRN neurons on the development of stress-related avoidance behavior. Furthermore, pharmacogenetically specific inactivation of LHb ameliorated the consequences of social defeat stress in an antidepressant like fashion even in mice in which SSRI were not effective due to a lack of 5-HT synthesis (Sachs et al., 2015), compatible with the idea that the LHb DRN pathway exerts strong control over DRN signaling where 5-HT itself is only one part of the whole picture. However, an additional debate exists as to the actual valence of serotonergic signaling along the reward axis.
Valence and Serotonin
An influential hypothesis holds that 5-HT controls behavioral inhibition in an aversive context (Soubrié, 1986) such as punishment, monetary losses or omitted rewards, thus ascribing opposite functions to DA and 5-HT (Daw et al., 2002). Vice versa, many studies have shown that 5-HT influences processing of rewards (Kranz et al., 2010), suggesting rather complementary roles for both systems.
Overall, pharmacological manipulations of serotonergic neurotransmission in humans provided striking evidence for an involvement of 5-HT in punishment processing (Evers et al., 2005; Chamberlain et al., 2006; Crockett et al., 2009; Geurts et al., 2013; Macoveanu et al., 2013). Dietary lowering of serotonergic activity abolished reaction slowing induced by punishments (Crockett et al., 2012), whereas acute SSRI administration as well as genetically determined higher 5-HT levels are associated with increased reaction slowing following errors (Fischer et al., 2015b). Similarly, ATD disturbed the association of past actions with punishments, but not rewards (Tanaka et al., 2009), which would suggest that 5-HT levels positively covary with the ability to memorize and utilize aversive events. However, ATD increased participants’ ability to predict negative outcomes of observed choices (Cools et al., 2008b; Robinson et al., 2012) and increased the BOLD response to errors in fMRI in the pre-frontal cortex (Evers et al., 2005). On the other hand, the suggestion that 5-HT mediates reward processing (Kranz et al., 2010; Luo et al., 2016) has likewise received considerable empirical support (Rogers et al., 2002; Cools et al., 2005; Del-Ben et al., 2005; Roiser et al., 2006; Tanaka et al., 2007; Seymour et al., 2012), while other studies support both punishment as well as reward processing (McCabe et al., 2010; Palminteri et al., 2012; Worbe et al., 2016; Scholl et al., 2017). Thus, manipulations of serotonergic tone affect reward and punishment processing, but the net effect, an impairment or facilitation, is not entirely consistent across studies.
Comparable to human studies, animal research has demonstrated involvement of the serotonergic system in processing both punishments and rewards. Using a reversal learning task with both rewards (juice) and punishments (noise) in marmosets, Rygula et al. (2015) found that local depletion of 5-HT in the amygdala or frontal cortex reduced overall feedback sensitivity, independent of valence. Similarly, rodent studies found that the overall effect of 5-HT manipulations can affect punishment or reward sensitivity, depending on method (SSRI, ATD, 5-HT depletion), dosage and duration of treatment (Bari et al., 2010). SSRI and genetically increased 5-HT levels were found to reduce appetitive operant responses (Sanders et al., 2007) but increased win-stay behavior during reversal learning in mice (Brown et al., 2012). Strong depletion of 5-HT decreased instrumental reward-based reinforcement-learning in rats (Izquierdo et al., 2012). Similarly, 5-HT depletion in the frontal cortex of marmosets disrupted acquisition of responding to appetitive conditioned reinforcement but not extinction (Walker et al., 2009). Akin to human studies, overall manipulations of serotonergic tone in animals sometimes affect reward and sometimes punishment processing. Overall, this suggests the need to extend the scope of one neuromodulatory system to its anatomical and neurochemical context.
Is Serotonin or DRN Activity Rewarding?
The DRN has since long been identified as one of the primary brain areas that promote self stimulation that strongly reinforces behavior via the dopaminergic system (Rompre and Miliaressis, 1985). However, it was until recently unclear if this mechanism actually relied upon cells originating within the DRN or if stimulation activated passing fibers, as well as if mediating cells are truly serotonergic. The structural connectivity between VTA and DRN has recently been described considerably more precisely by novel cell-type specific tracing techniques. It was found that dopaminergic VTA neurons in mice receive both the densest projections, as well as the second most numerous, after striatal projections, from the DRN (Watabe-Uchida et al., 2012). This projection was found to be mainly glutamatergic, but additionally contains 5-HT co-releasing neurons (McDevitt et al., 2014; Qi et al., 2014). This pathway drives self-stimulation and conditioned place preference via asymmetrical synapses to mesostriatal VTA DA neurons (Qi et al., 2014). Both blockade of striatal D1 receptors as well as blockade of glutamatergic signals in the VTA, abolishes these effects. Additionally, inhibitory GABAergic projections from VTA and striatum regulate 5-HT activity, forming a feedback-loop that intertwines DA and 5-HT systems in the reward circuitry (Pollak Dorocic et al., 2014), although tracing studies suggest that 5-HT is in a stronger position to control DA activity than vice versa (Ogawa et al., 2014).
Single-neuron recording studies suggested that DRN neurons, that putatively contain 5-HT, process reward. In rats and monkeys, these DRN neurons were found to reflect the magnitude of delivered liquid rewards as well as overall reward likelihood (Nakamura et al., 2008; Bromberg-Martin et al., 2010; Inaba et al., 2013). Using calcium imaging and fiber photometry, a method to observe activity in specifically targeted neuronal populations, it was recently shown that serotonin transporter positive (SERT+) DRN 5-HT neurons displayed strong reward- but not aversion-related activity modulations in freely behaving mice (Li et al., 2016). These neurons increased activity during consumption of primary reinforcers (sugar, sex), and while animals waited for delivery of rewards. This confirms a series of previous experiments which suggested 5-HT to mediate impulse control whilst anticipating a future reward (Miyazaki K. W. et al., 2012; Miyazaki K. et al., 2012; Miyazaki et al., 2014), and speaks towards involvement of SERT+ 5-HT neurons in primary reward processing. Interestingly, activity of putative (Inaba et al., 2013; Hayashi et al., 2015) and optogenetically defined 5-HT neurons display value-related signals on different time scales (Cohen et al., 2015). In head-restrained mice, Cohen et al. (2015) found that serotonergic neurons display very brief responses to rewards and punishments, but sustained activity possibly reflecting the “beneficialness” (Luo et al., 2016) of the current environment, or the motivational state, for periods of up to 10 s (Cohen et al., 2015). These long-lasting changes are reflected in rather low changes in firing rates in the range of 1–2 spikes/s, which nonetheless might exert significant influences in serotonergic innervation on target structures especially given the low baseline firing rate observed in most 5-HT neurons.
Optogenetic stimulation provided new insights in the functional effects of activation of specific DRN neurons. Liu et al. (2014) expressed a light sensitive ion channel in DRN neurons targeted via an enhancer region coupled to the Pet-1 gene, which is selectively (yet not exclusively) expressed in serotonergic neurons (Scott et al., 2005). Stimulation of Pet-1+ neurons was found to be highly rewarding, inducing conditioned place preference outlasting the stimulation period, instrumental learning and favorable competition against a natural reinforcer (Liu et al., 2014). These effects mostly depended on co-release of glutamate and were reduced, yet not absent, in mice lacking VGluT3. Consistently, direct targeting of VGluT3+ neurons projecting from DRN to VTA, out of which only some contain 5-HT, drove vigorous self stimulation (Qi et al., 2014), indicating that DRN neurons can induce burst firing in VTA DA neurons which drives behavioral reinforcement (Grace et al., 2007). Another study found that orbitofrontal cortex neurons whose activity reflected prospective natural reinforcers also coded the prospective intensity of Pet-1+ DRN stimulation (Zhou et al., 2015), indicating similarity between natural reinforcers and artificial DRN stimulation. However, a very similar stimulation protocol again targeting Pet-1+ DRN neurons did not replicate the rewarding effects reported before (McDevitt et al., 2014).
Another targeting approach for 5-HT neurons within the DRN for optogenetic activation, is genetic tagging of the SERT. Targeting SERT is hypothesized to reduce overlap with glutamatergic populations (Luo et al., 2015), yet additional empirical validation for this assumption is needed. As a caveat, targeting 5-HT neurons via SERT requires heterozygous knock-in, effectively de-activating one SERT allele, which in itself alters serotonergic activity (Mathews et al., 2004). In this line of research, rewarding effects of DRN 5-HT neuron stimulation could not be replicated. Stimulation was neither found to induce sustained place preference, nor reinforce behavior (Fonseca et al., 2015; Correia et al., 2017). Additionally, when 5-HT neurons were targeted via the regulatory elements of the rate-limiting enzyme in 5-HT synthesis, TpH2, no directly rewarding effects of stimulation were observed (Miyazaki et al., 2014). Recently, Correia et al. (2017) found that short-term activation of SERT+ DRN neurons led to behavioral slowing again in the absence of rewarding effects, which however did not interfere with motivated behavior.
Another core feature that ties reward processing to learning and behavioral adaptation is the computation of teaching signals in the form of reward or punishment prediction errors (PE), which has been ascribed as a central function of the DA system. Recently, it was found that different subpopulations of dopaminergic neurons with specific projection regions respond differentially to reward, novelty and aversiveness. Projection neurons to the ventral striatum reflect reward, but appear insensitive to aversive events and novelty (Eshel et al., 2016; Menegas et al., 2017), which additionally can be modulated by reward context (Matsumoto et al., 2016). On the contrary, other VTA neurons (Matsumoto and Hikosaka, 2009), as well as defined dopaminergic neurons (Cohen et al., 2012), reflect aversive stimuli and novelty, but these seem to project to more caudal striatal regions (Menegas et al., 2017). Some recent studies investigated coding of reward PEs in defined serotonergic neurons. Matias et al. (2017) reported that SERT+ 5-HT neurons in DRN on population level reflect positive reward PEs, consistent with several other studies (Nakamura et al., 2008; Inaba et al., 2013; Cohen et al., 2015; Hayashi et al., 2015) and very similar to DA neurons in VTA. However, while reward PE coding DA neurons decreased firing rates when an expected reward was omitted or replaced with likely aversive events (an air puff targeted to the eye), constituting a signed PE signal, 5-HT neurons reflected mainly unsigned PEs, or surprise. Interestingly, 5-HT neurons were slower to adapt to expectancy changes, causing a shift in putative relative availability of 5-HT and DA during the learning of reward expectations. This leads to a situation in which, for example in reversal learning tasks, a switch from a previously good to a now bad stimulus is accompanied by an early decline in DA activity, but a longer lasting 5-HT signal that only later on adapts to the new expected level of reward. This may explain why 5-HT has in many cases been ascribed a mainly aversive role opponent to DA (Daw et al., 2002). It remains an open question if GABAergic neurons in DRN represent expected value, similar to the role of GABAergic neurons in VTA in dopaminergic reward PE calculation (Cohen et al., 2012; Eshel et al., 2015).
An interesting observation is that optogenetically defined serotonergic neurons were found to respond earlier, although less vigorously, to reward predicting cues than DA neurons (Cohen et al., 2015), yet later than some cortical reward related signals (Ullsperger et al., 2014). This may indicate that DRN reward related activity represents a high level modulatory control of the brain’s reward system, depending on input from upstream cortical areas involved in reward processing.
Overall, it appears that serotonergic neurons code reward signals consistent across different species, yet their optogenetic activation is only rewarding when either glutamate is co-released or glutamatergic DRN neurons are targeted. These glutamatergic cells project strongly to VTA and are able to recruit the dopaminergic reward circuitry. Aversive events are not consistently (Li et al., 2016) coded by serotonergic neurons in animal studies. This may be explained by the fact that in most of these studies animals were head-restrained (Cohen, 2015; Matias et al., 2017). Restrainment itself might be a stressor interfering with physiological neuronal activity patterns of a neural modulator thought to be involved in reward processing and mood.
The Role of 5-HT in Drug-Induced Euphoria and Controlling Reward Seeking Behavior
Low levels of 5-HT are a known diathesis for the development of anxiety disorders, depression (Jacobsen et al., 2012; Sachs et al., 2015), but also drug addiction (Ducci and Goldman, 2012). Additionally, most addictive drugs either cause increased 5-HT release or directly agonize 5-HT receptors. Two essential components of reward can be defined as “liking”, reflecting a pleasurable subjective state, and “wanting”, reflecting the reinforcing properties of a situation or substance (Berridge et al., 2009). Assessing liking in animal models mostly relies on facial expression studies, for example in mice, where it was demonstrated that DA, while reinforcing behavior, does not induce liking (Wyvell and Berridge, 2000; Leyton et al., 2002; Berridge and Robinson, 2003). On the other hand, subjective drug effects reported in humans can be informative about underlying neurotransmitter systems and how they mediate effects.
The group of monoamine releasers, which increase DA, 5-HT and norepinephrine (NE), and their most commonly abused variants such as cocaine, amphetamine and Methylenedioxymethamphetamine (MDMA or ecstasy) are, after cannabis, the most widely abused illegal drugs1. Contrary to initial assumptions, potency to release NE does not seem to significantly alter addictive potency of these substances (Banks et al., 2014), but the relative potential to release 5-HT over DA has been found to affect both a drug’s euphoric, as well as addictive properties. Interestingly, the potency of drugs to release 5-HT, even when effects on DA and NE release are comparable, negatively correlates with its potency as a behavioral reinforcer (Wee et al., 2005). This was demonstrated using various amphetamine derivates in monkeys with comparable NE- and DA-, yet different 5-HT-releasing properties. Furthermore, individual monkeys with more pronounced 5-HT release induced by MDMA consistently self-administered the drug less than individuals with lower release. Addition of the serotonin releasing agent fenfluramine to self-administered amphetamine also decreased self-administration (Wee and Woolverton, 2006). Consistently, destruction of the serotonergic system using the selective neurotoxin 5,7-DHT increased MDMA self-administration rate and acquisition speed in rats (Bradbury et al., 2013). This suggests that 5-HT balances the behaviorally reinforcing effects of DA.
Another way to assess behavioral control over reward behavior in laboratory settings is to combine intracranial self stimulation (ICSS) of rewarding areas (often the medium forebrain bundle) with administration of drugs. DA releasing agents have been shown in this setting to increase self stimulation, which is interpreted as a context dependent facilitation of addictive behavior. Interestingly, selective 5-HT releasers inhibit ICSS (Olds and Yuwiler, 1992), and moreover the relative specificity to release 5-HT compared with DA negatively covaries with ICSS facilitation (Bauer et al., 2013). However, this general inhibitory influence of 5-HT over ICSS is simplified, and other studies indicate that the overall effect of 5-HT on brain stimulation depends, among other factors, on the locus of 5-HT application (Kranz et al., 2010). Although addictive behavior evolves over a longer time scale and includes complex adaptations in the serotonergic system (Müller and Homberg, 2014), there is evidence from animal studies suggesting that integrity of the serotonergic system and higher 5-HT release, reduces drug self-administration.
Human studies that combine pharmacological manipulations of the 5-HT system and imaging techniques, have not yielded an unambiguous picture (Macoveanu, 2014), in part due to the difficulty to capture brainstem signals related to DRN or MRN activity. Tanaka et al. (2004) found increased activity overlapping with the DRN, when participants had to endure short-term losses to obtain long-term rewards, akin to effects observed in the dorsal striatum. This striatal signal was modulated by manipulations of 5-HT levels in human participants, and the covariation between dorsal striatum and long-term rewards was positively dependent on 5-HT levels (Tanaka et al., 2007). Furthermore, ATD in humans was found to reduce the impact of previous rewards on current choices, and increased the tendency to repeat previous choices in a dynamically changing reinforcement learning task (Seymour et al., 2012).
These findings translate to human abuse behavior and also appear to dissect a drug’s rewarding properties, mapping wanting to DA and speculatively liking to 5-HT, in an additive manner. Epidemiological data suggest that MDMA has a significantly lower abuse rate compared to monoamine releasers with less serotonergic action, like amphetamines (Degenhardt et al., 2010). On the other hand, even compared to high doses of amphetamine, MDMA is subjectively experienced as more pleasurable, positive mood inducing and euphorizing (Camí et al., 2000; Tancer and Johanson, 2003; Carhart-Harris et al., 2015). More selective 5-HT releasers that spare DA, surprisingly, are neither reinforcing (Woods and Tessel, 1974), nor experienced as pleasurable by humans. In fact, the selective 5-HT releaser m-chlorophenylpiperazine (mCPP) was found to reduce positive mood and euphoria (Tancer and Johanson, 2003), and acute intravenous administration of an SSRI in healthy participants, increased self rated sadness and incompetence (Fischer et al., 2015a). This suggests that 5-HT under physiological circumstances balances reinforcing effects of DA while itself being neither reinforcing nor rewarding, yet may be essential to inhibit reinforced behavior in order to promote behavioral flexibility (Branchi, 2011; Fischer et al., 2015b; Matias et al., 2017).
In short, consistent with subjective drug effects, positive reward may be reflected by a combination of DA and 5-HT, whereas punishment could be reflected by 5-HT and absence, or reduction, of DA (Cohen et al., 2015).
Methodological Considerations
We put effort in this review to include especially findings from studies employing recent state-of-the art methods, such as optogenetics. These promised to resolve many contradictions regarding the 5-HT system by unequivocally identifying and interfering with specific neural populations, for which thus far identification was relatively unspecific using mostly either firing patterns or receptor dependent changes in activity. However, even these specific methods yield contradictory results. For example, doubtlessly direct DRN stimulation is rewarding, as known from self stimulation studies (Miliaressis et al., 1975; Van Der Kooy et al., 1978; Rompre and Miliaressis, 1985), and confirmed by optogenetic stimulation of either mostly glutamatergic non-serotonergic DRN output neurons (McDevitt et al., 2014) or 5-HT and glutamate co-releasing neurons (Liu et al., 2014; Qi et al., 2014). A more 5-HT specific stimulation profile, targeting SERT+ or TpH2+ neurons, on the other hand, does not reproduce rewarding effects (Miyazaki et al., 2014; Fonseca et al., 2015; Correia et al., 2017), possibly due to a lack of glutamatergic signaling, and acute SSRI administration is rarely reported as rewarding (Fischer et al., 2015a). An interpretation of this absence of effects appears complicated if such stimulation differs from physiological activity. It is currently unknown whether physiological inputs from specific regions to the DRN asymmetrically synapse onto purely serotonergic, glutamatergic, co-releasing, or other, e.g., GABAergic neurons. Thus, optogenetic stimulation may in some cases be over-specific in the sense of evoking an unphysiological signal that may not induce the same behavioral effects as normally occurring activation of serotonergic pathways by stimulating neurons that would usually fire independently.
On the other hand, it is well established that the group of raphe nuclei is involved in different physiological functions depending on their topography (Hale and Lowry, 2011). Moreover, distinct afferent projection profiles have been demonstrated. While PFC and LHb project bilaterally to the DRN, other regions such as amygdala and hypothalamus, asymmetrically synapse to ipsilateral parts (Zhou et al., 2017). Recently, it was found that optogenetic stimulation of SERT+ DRN projection neurons to the bed nucleus of the stria terminalis induces fear and anxiety (Marcinkiewcz et al., 2016), whereas the same stimulation of SERT+ neurons in the DRN itself, does not display this effect (Correia et al., 2017). This raises the question whether: (a) isolated stimulation of single pathways may be over-specific; or (b) activation of neuronal populations within a region may still not be specific enough.
Furthermore, it is crucial to incorporate the possibility of co-release of other neurotransmitters from monoaminergic neurons, which can lead to very different effects when comparing drug studies, e.g., employing agonists or releasers, which circumvent co-release, with stimulation studies, which induce co-release depending on the exact method used to target specific neurons (Hu, 2016). Considering such co-release-dependent discrepancies may explain puzzling effects of drugs, which often do not align with physiological hypotheses (Fischer et al., 2015c).
Conclusion
Afferent and efferent projections, hodological properties, the time course of individual neural activity bridging short latency and longer lasting activity modulations, all position the serotonergic system ideally to extract motivationally salient information and induce focused attention to guide goal oriented actions towards rewards. Human and animal studies that manipulated 5-HT levels strongly support serotonin’s role in affective processing, but the direction of association and specificity for aversive or rewarding events remains controversial. Optogenetic studies in rodents suggest that DA and 5-HT jointly guide learning by positive coding of reward PE signals, whereas absence of DA and presence of 5-HT is associated with reward omission, and possibly punishment PE signals. Furthermore, the DRN controls release of 5-HT to many projection regions and can recruit dopaminergic reward circuits via glutamate. Activation of purely serotonergic cells was found to be mostly non-rewarding or purely inhibitory. Taken together, this could be interpreted such that manipulations increasing 5-HT signals without a consecutive DA increase, mimics aversive events. On the other hand, direct DRN stimulation, specific stimulation of glutamate-releasing and co-releasing DRN cells, induces rewarding effects comparable to direct VTA stimulation (McDevitt et al., 2014). It therefore appears that specific DRN neuronal populations control DA-related reward behavior via glutamate release and co-release. Thus, reward appears most likely mediated via glutamatergic excitation of DA neurons, at least in part, by input from DRN (Liu et al., 2014; McDevitt et al., 2014; Qi et al., 2014) and parallel release of 5-HT. In accordance with this, subjective self-reports of human subjects suggest that a singular increase in 5-HT or DA release is not experienced as pleasurable, but the conjoint increase is highly pleasurable. An interesting additional possibility is that 5-HT neurons may furthermore integrate DA effects over time into longer-lasting affective signals conveyed in a tonic fashion (Cohen et al., 2015). This is compatible with the idea that 5-HT encodes beneficialness (Luo et al., 2016), and may signal motivation to either maintain or switch current behavior, for example displayed by 5-HT’s role in facilitating patience for future rewards (Miyazaki et al., 2014). Combined with the finding that 5-HT levels via immediate release or pretreatment bivalently modulate the potency of highly addictive substances such as amphetamines or cocaine (Wee et al., 2005; Cunningham and Anastasio, 2014), it may be that it functionally orchestrates the brains reward systems via parallel fast activation and slow longer-lasting inhibition. Thus, 5-HT additionally appears to control motivationally rewarding effects of DA as evidenced via drug addiction.
Highly specific methods to stimulate neuromodulatory systems reveal the complexity and entanglement between neuromodulatory systems on multiple levels, but do not provide simple answers. As more data is collected, it appears that judging the effect of one neuromodulatory system alone is unphysiological. However, converging between pharmacological effects in humans and optogenetic stimulation in animals, DA and 5-HT combined provide signals that interdependently are sufficient to guide reward related behavioral adaptations, and induce subjective reward. This complexity has direct consequences for the interpretation of pharmacological, genetic, and correlational studies in humans and indicates that DA and 5-HT do not have opposing functions (Daw et al., 2002), but rather could in concert provide a combined reward signal, whereas its dissociation may encode punishment.
Author Contributions
AGF wrote the manuscript, AGF and MU discussed, corrected and conceived the manuscript.
Funding
AGF is funded by CBBS ScienceCampus financed by the Leibniz Association (SAS-2015-LIN-LWC).
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We thank Gerhard Jocham for comments on the manuscript.
Footnotes
References
Baker, K. G., Halliday, G. M., Hornung, J.-P., Geffen, L. B., Cotton, R. G. H., and Törk, I. (1991). Distribution, morphology and number of monoamine-synthesizing and substance P-containing neurons in the human dorsal raphe nucleus. Neuroscience 42, 757–775. doi: 10.1016/0306-4522(91)90043-n
Bang, S. J., and Commons, K. G. (2012). Forebrain GABAergic projections from the dorsal raphe nucleus identified by using GAD67-GFP knock-in mice. J. Comp. Neurol. 520, 4157–4167. doi: 10.1002/cne.23146
Banks, M. L., Bauer, C. T., Blough, B. E., Rothman, R. B., Partilla, J. S., Baumann, M. H., et al. (2014). Abuse-related effects of dual dopamine/serotonin releasers with varying potency to release norepinephrine in male rats and rhesus monkeys. Exp. Clin. Psychopharmacol. 22, 274–284. doi: 10.1037/a0036595
Bari, A., Theobald, D. E., Caprioli, D., Mar, A. C., Aidoo-Micah, A., Dalley, J. W., et al. (2010). Serotonin modulates sensitivity to reward and negative feedback in a probabilistic reversal learning task in rats. Neuropsychopharmacology 35, 1290–1301. doi: 10.1038/npp.2009.233
Bauer, C. T., Banks, M. L., Blough, B. E., and Negus, S. S. (2013). Use of intracranial self-stimulation to evaluate abuse-related and abuse-limiting effects of monoamine releasers in rats. Br. J. Pharmacol. 168, 850–862. doi: 10.1111/j.1476-5381.2012.02214.x
Berridge, K. C., and Robinson, T. E. (2003). Parsing reward. Trends Neurosci. 26, 507–513. doi: 10.1016/s0166-2236(03)00233-9
Berridge, K. C., Robinson, T. E., and Aldridge, J. W. (2009). Dissecting components of reward: ‘liking’, ‘wanting’ and learning. Curr. Opin. Pharmacol. 9, 65–73. doi: 10.1016/j.coph.2008.12.014
Bradbury, S., Bird, J., Colussi-Mas, J., Mueller, M., Ricaurte, G., and Schenk, S. (2013). Acquisition of MDMA self-administration: pharmacokinetic factors and MDMA-induced serotonin release. Addict. Biol. 19, 874–884. doi: 10.1111/adb.12069
Branchi, I. (2011). The double edged sword of neural plasticity: increasing serotonin levels leads to both greater vulnerability to depression and improved capacity to recover. Psychoneuroendocrinology 36, 339–351. doi: 10.1016/j.psyneuen.2010.08.011
Bromberg-Martin, E. S., Hikosaka, O., and Nakamura, K. (2010). Coding of task reward value in the dorsal raphe nucleus. J. Neurosci. 30, 6262–6272. doi: 10.1523/JNEUROSCI.0015-10.2010
Brown, H. D., Amodeo, D. A., Sweeney, J. A., and Ragozzino, M. E. (2012). The selective serotonin reuptake inhibitor, escitalopram, enhances inhibition of prepotent responding and spatial reversal learning. J. Psychopharmacol. 26, 1443–1455. doi: 10.1177/0269881111430749
Camí, J., Farré, M., Mas, M., Roset, P. N., Poudevida, S., Mas, A., et al. (2000). Human Pharmacology of 3,4-Methylenedioxymeth-amphetamine (“Ecstasy”): psychomotor performance and subjective effects. J. Clin. Psychopharmacol. 20, 455–466. doi: 10.1097/00004714-200008000-00010
Carhart-Harris, R. L., Murphy, K., Leech, R., Erritzoe, D., Wall, M. B., Ferguson, B., et al. (2015). The effects of acutely administered 3,4-methylenedioxymethamphetamine on spontaneous brain function in healthy volunteers measured with arterial spin labeling and blood oxygen level-dependent resting state functional connectivity. Biol. Psychiatry 78, 554–562. doi: 10.1016/j.biopsych.2013.12.015
Caspi, A., Sugden, K., Moffitt, T. E., Taylor, A., Craig, I. W., Harrington, H., et al. (2003). Influence of life stress on depression: moderation by a polymorphism in the 5-HTT gene. Science 301, 386–389. doi: 10.1126/science.1083968
Celada, P., Puig, M. V., and Artigas, F. (2013). Serotonin modulation of cortical neurons and networks. Front. Integr. Neurosci. 7:25. doi: 10.3389/fnint.2013.00025
Challis, C., Boulden, J., Veerakumar, A., Espallergues, J., Vassoler, F. M., Pierce, R. C., et al. (2013). Raphe GABAergic neurons mediate the acquisition of avoidance after social defeat. J. Neurosci. 33, 13978–13988. doi: 10.1523/JNEUROSCI.2383-13.2013
Chamberlain, S. R., Müller, U., Blackwell, A. D., Clark, L., Robbins, T. W., and Sahakian, B. J. (2006). Neurochemical modulation of response inhibition and probabilistic learning in humans. Science 311, 861–863. doi: 10.1126/science.1121218
Charnay, Y., and Léger, L. (2010). Brain serotonergic circuitries. Dialogues Clin. Neurosci. 12, 471–487.
Cohen, J. Y. (2015). Dopamine and serotonin signals for reward across time scales. Science 350, 47–48. doi: 10.1126/science.aad3003
Cohen, J. Y., Amoroso, M. W., and Uchida, N. (2015). Serotonergic neurons signal reward and punishment on multiple timescales. Elife 4:e06346. doi: 10.7554/eLife.06346
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., and Uchida, N. (2012). Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85–88. doi: 10.1038/nature10754
Cools, R., Blackwell, A., Clark, L., Menzies, L., Cox, S., and Robbins, T. W. (2005). Tryptophan depletion disrupts the motivational guidance of goal-directed behavior as a function of trait impulsivity. Neuropsychopharmacology 30, 1362–1373. doi: 10.1038/sj.npp.1300704
Cools, R., Roberts, A. C., and Robbins, T. W. (2008a). Serotoninergic regulation of emotional and behavioural control processes. Trends Cogn. Sci. 12, 31–40. doi: 10.1016/j.tics.2007.10.011
Cools, R., Robinson, O. J., and Sahakian, B. (2008b). Acute tryptophan depletion in healthy volunteers enhances punishment prediction but does not affect reward prediction. Neuropsychopharmacology 33, 2291–2299. doi: 10.1038/sj.npp.1301598
Correia, P. A., Lottem, E., Banerjee, D., Machado, A. S., Carey, M. R., and Mainen, Z. F. (2017). Transient inhibition and long-term facilitation of locomotion by phasic optogenetic activation of serotonin neurons. Elife 6:e20975. doi: 10.7554/eLife.20975
Crockett, M. J., Clark, L., Apergis-Schoute, A. M., Morein-Zamir, S., and Robbins, T. W. (2012). Serotonin modulates the effects of pavlovian aversive predictions on response vigor. Neuropsychopharmacology 37, 2244–2252. doi: 10.1038/npp.2012.75
Crockett, M. J., Clark, L., and Robbins, T. W. (2009). Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J. Neurosci. 29, 11993–11999. doi: 10.1523/JNEUROSCI.2513-09.2009
Cunningham, K. A., and Anastasio, N. C. (2014). Neuropharmacology. Neuropharmacology 76, 460–478. doi: 10.1016/j.neuropharm.2013.06.030
Daw, N. D., Kakade, S., and Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616. doi: 10.1016/s0893-6080(02)00052-7
Dayan, P., and Huys, Q. J. M. (2009). Serotonin in affective control. Annu. Rev. Neurosci. 32, 95–126. doi: 10.1146/annurev.neuro.051508.135607
Deakin, J. F., and Graeff, F. G. (1991). 5-HT and mechanisms of defence. J. Psychopharmacol. 5, 305–315. doi: 10.1177/026988119100500414
Degenhardt, L., Bruno, R., and Topp, L. (2010). Is ecstasy a drug of dependence? Drug Alcohol Depend. 107, 1–10. doi: 10.1016/j.drugalcdep.2009.09.009
Del-Ben, C. M., Deakin, J. F. W., McKie, S., Delvai, N. A., Williams, S. R., Elliott, R., et al. (2005). The effect of citalopram pretreatment on neuronal responses to neuropsychological tasks in normal volunteers: an fMRI study. Neuropsychopharmacology 30, 1724–1734. doi: 10.1038/sj.npp.1300728
Ducci, F., and Goldman, D. (2012). The genetic basis of addictive disorders. Psychiatr. Clin. North Am. 35, 495–519. doi: 10.1016/j.psc.2012.03.010
El Mestikawy, S., Wallén-Mackenzie, Å., Fortin, G. M., Descarries, L., and Trudeau, L.-E. (2011). From glutamate co-release to vesicular synergy: vesicular glutamate transporters. Nat. Rev. Neurosci. 12, 204–216. doi: 10.1038/nrn2969
Eshel, N., Bukwich, M., Rao, V., Hemmelder, V., Tian, J., and Uchida, N. (2015). Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525, 243–246. doi: 10.1038/nature14855
Eshel, N., Tian, J., Bukwich, M., and Uchida, N. (2016). Dopamine neurons share common response function for reward prediction error. Nat. Neurosci. 19, 479–486. doi: 10.1038/nn.4239
Evers, E. A. T., Cools, R., Clark, L., van der Veen, F. M., Jolles, J., Sahakian, B. J., et al. (2005). Serotonergic modulation of prefrontal cortex during negative feedback in probabilistic reversal learning. Neuropsychopharmacology 30, 1138–1147. doi: 10.1038/sj.npp.1300663
Fadda, F., Cocco, S., and Stancampiano, R. (2000). A physiological method to selectively decrease brain serotonin release. Brain Res. Protoc. 5, 219–222. doi: 10.1016/s1385-299x(00)00016-7
Fischer, A. G., Endrass, T., Goebel, I., Reuter, M., Montag, C., Kubisch, C., et al. (2015a). Interactive effects of citalopram and serotonin transporter genotype on neural correlates of response inhibition and attentional orienting. Neuroimage 116, 59–67. doi: 10.1016/j.neuroimage.2015.04.064
Fischer, A. G., Endrass, T., Reuter, M., Kubisch, C., and Ullsperger, M. (2015b). Serotonin reuptake inhibitors and serotonin transporter genotype modulate performance monitoring functions but not their electrophysiological correlates. J. Neurosci. 35, 8181–8190. doi: 10.1523/JNEUROSCI.5124-14.2015
Fischer, A. G., Jocham, G., and Ullsperger, M. (2015c). Dual serotonergic signals: a key to understanding paradoxical effects? Trends Cogn. Sci. 19, 21–26. doi: 10.1016/j.tics.2014.11.004
Fonseca, M. S., Murakami, M., and Mainen, Z. F. (2015). Activation of dorsal raphe serotonergic neurons promotes waiting but is not reinforcing. Curr. Biol. 25, 306–315. doi: 10.1016/j.cub.2014.12.002
Geddes, S. D., Assadzada, S., Lemelin, D., Sokolovski, A., Bergeron, R., Haj-Dahmane, S., et al. (2016). Target-specific modulation of the descending prefrontal cortex inputs to the dorsal raphe nucleus by cannabinoids. Proc. Natl. Acad. Sci. U S A 113, 5429–5434. doi: 10.1073/pnas.1522754113
Geurts, D. E. M., Huys, Q. J. M., den Ouden, H. E. M., and Cools, R. (2013). Serotonin and aversive pavlovian control of instrumental behavior in humans. J. Neurosci. 33, 18932–18939. doi: 10.1523/JNEUROSCI.2749-13.2013
Grace, A. A., Floresco, S. B., Goto, Y., and Lodge, D. J. (2007). Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends Neurosci. 30, 220–227. doi: 10.1016/j.tins.2007.03.003
Hale, M. W., and Lowry, C. A. (2011). Functional topography of midbrain and pontine serotonergic systems: implications for synaptic regulation of serotonergic circuits. Psychopharmacology 213, 243–264. doi: 10.1007/s00213-010-2089-z
Hayashi, K., Nakao, K., and Nakamura, K. (2015). Appetitive and aversive information coding in the primate dorsal raphe nucleus. J. Neurosci. 35, 6195–6208. doi: 10.1523/JNEUROSCI.2860-14.2015
Hayes, D. J., and Greenshaw, A. J. (2011). 5-HT receptors and reward-related behaviour: a review. Neurosci. Biobehav. Rev. 35, 1419–1449. doi: 10.1016/j.neubiorev.2011.03.005
Hikosaka, O. (2010). The habenula: from stress evasion to value-based decision-making. Nature 11, 503–513. doi: 10.1038/nrn2866
Hioki, H., Nakamura, H., Ma, Y.-F., Konno, M., Hayakawa, T., Nakamura, K. C., et al. (2010). Vesicular glutamate transporter 3-expressing nonserotonergic projection neurons constitute a subregion in the rat midbrain raphe nuclei. J. Comp. Neurol. 518, 668–686. doi: 10.1002/cne.22237
Hornung, J.-P. (2003). The human raphe nuclei and the serotonergic system. J. Chem. Neuroanat. 26, 331–343. doi: 10.1016/j.jchemneu.2003.10.002
Hu, H. (2016). Reward and aversion. Annu. Rev. Neurosci. 39, 297–324. doi: 10.1146/annurev-neuro-070815-014106
Inaba, K., Mizuhiki, T., Setogawa, T., Toda, K., Richmond, B. J., and Shidara, M. (2013). Neurons in monkey dorsal raphe nucleus code beginning and progress of step-by-step schedule, reward expectation, and amount of reward outcome in the reward schedule task. J. Neurosci. 33, 3477–3491. doi: 10.1523/JNEUROSCI.4388-12.2013
Izquierdo, A., Carlos, K., Ostrander, S., Rodriguez, D., McCall-Craddolph, A., Yagnik, G., et al. (2012). Impaired reward learning and intact motivation after serotonin depletion in rats. Behav. Brain Res. 233, 494–499. doi: 10.1016/j.bbr.2012.05.032
Jacobsen, J. P. R., Medvedev, I. O., and Caron, M. G. (2012). The 5-HT deficiency theory of depression: perspectives from a naturalistic 5-HT deficiency model, the tryptophan hydroxylase 2Arg439His knockin mouse. Philos. Trans. R. Soc. Lond. B Biol. Sci. 367, 2444–2459. doi: 10.1098/rstb.2012.0109
Jhou, T. C., Geisler, S., Marinelli, M., Degarmo, B. A., and Zahm, D. S. (2009). The mesopontine rostromedial tegmental nucleus: a structure targeted by the lateral habenula that projects to the ventral tegmental area of Tsai and substantia nigra compacta. J. Comp. Neurol. 513, 566–596. doi: 10.1002/cne.21891
Johnson, M. D. (1994). Synaptic glutamate release by postnatal rat serotonergic neurons in microculture. Neuron 12, 433–442. doi: 10.1016/0896-6273(94)90283-6
Kranz, G. S., Kasper, S., and Lanzenberger, R. (2010). Reward and the serotonergic system. Neuroscience 166, 1023–1035. doi: 10.1016/j.neuroscience.2010.01.036
Léger, L., and Wiklund, L. (1982). Distribution and numbers of indoleamine cell bodies in the cat brainstem determined with Falck-Hillarp fluorescence histochemistry. Brain Res. Bull. 9, 245–251. doi: 10.1016/0361-9230(82)90137-x
Leyton, M., Boileau, I., Bekelfat, C., Diksic, M., Baker, G., and Dagher, A. (2002). Amphetamine-induced increases in extracellular dopamine, drug wanting, and novelty seeking a PET/[11C]raclopride study in healthy men. Neuropsychopharmacology 27, 1027–1035. doi: 10.1016/s0893-133x(02)00366-4
Li, Y., Zhong, W., Wang, D., Feng, Q., Liu, Z., Zhou, J., et al. (2016). Serotonin neurons in the dorsal raphe nucleus encode reward signals. Nat. Commun. 7:10503. doi: 10.1038/ncomms10503
Liu, R., Jolas, T., and Aghajanian, G. (2000). Serotonin 5-HT 2 receptors activate local GABA inhibitory inputs to serotonergic neurons of the dorsal raphe nucleus. Brain Res. 873, 34–45. doi: 10.1016/s0006-8993(00)02468-9
Liu, Z., Zhou, J., Li, Y., Hu, F., Lu, Y., Ma, M., et al. (2014). Dorsal raphe neurons signal reward through 5-HT and glutamate. Neuron 81, 1360–1374. doi: 10.1016/j.neuron.2014.02.010
Luo, M., Li, Y., and Zhong, W. (2016). Do dorsal raphe 5-HT neurons encode “beneficialness?”. Neurobiol. Learn. Mem. 135, 40–49. doi: 10.1016/j.nlm.2016.08.008
Luo, M., Zhou, J., and Liu, Z. (2015). Reward processing by the dorsal raphe nucleus: 5-HT and beyond. Learn. Mem. 22, 452–460. doi: 10.1101/lm.037317.114
Macoveanu, J. (2014). Serotonergic modulation of reward and punishment: evidence from pharmacological fMRI studies. Brain Res. 1556, 19–27. doi: 10.1016/j.brainres.2014.02.003
Macoveanu, J., Hornboll, B., Elliott, R., Erritzoe, D., Paulson, O. B., Siebner, H., et al. (2013). Serotonin 2A receptors, citalopram and tryptophan-depletion: a multimodal imaging study of their interactions during response inhibition. Neuropsychopharmacology 38, 996–1005. doi: 10.1038/npp.2012.264
Marcinkiewcz, C. A., Mazzone, C. M., D’Agostino, G., Halladay, L. R., Hardaway, J. A., DiBerto, J. F., et al. (2016). Serotonin engages an anxiety and fear-promoting circuit in the extended amygdala. Nature 537, 97–101. doi: 10.1038/nature19318
Mathews, T. A., Fedele, D. E., Coppelli, F. M., Avila, A. M., Murphy, D. L., and Andrews, A. M. (2004). Gene dose-dependent alterations in extraneuronal serotonin but not dopamine in mice with reduced serotonin transporter expression. J. Neurosci. Methods 140, 169–181. doi: 10.1016/j.jneumeth.2004.05.017
Matias, S., Lottem, E., Dugué, G. P., and Mainen, Z. F. (2017). Activity patterns of serotonin neurons underlying cognitive flexibility. Elife 6:e20552. doi: 10.7554/eLife.20552
Matsumoto, H., Tian, J., Uchida, N., and Watabe-Uchida, M. (2016). Midbrain dopamine neurons signal aversion in a reward-context-dependent manner. Elife 5:e17328. doi: 10.7554/eLife.17328
Matsumoto, M., and Hikosaka, O. (2007). Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447, 1111–1115. doi: 10.1038/nature05860
Matsumoto, M., and Hikosaka, O. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841. doi: 10.1038/nature08028
McCabe, C., Mishor, Z., Cowen, P. J., and Harmer, C. J. (2010). Diminished neural processing of aversive and rewarding stimuli during selective serotonin reuptake inhibitor treatment. Biol. Psychiatry 67, 439–445. doi: 10.1016/j.biopsych.2009.11.001
McDevitt, R. A., Tiran-Cappello, A., Shen, H., Balderas, I., Britt, J. P., Marino, R. A. M., et al. (2014). Serotonergic versus nonserotonergic dorsal raphe projection neurons: differential participation in reward circuitry. Cell Rep. 8, 1857–1869. doi: 10.1016/j.celrep.2014.08.037
Menegas, W., Babayan, B. M., Uchida, N., and Watabe-Uchida, M. (2017). Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice. Elife 6:e21886. doi: 10.7554/eLife.21886
Miliaressis, E. R., Bouchard, A., and Jacobowitz, D. M. (1975). Strong positive reward in median raphe: specific inhibition bypara-chlorophenylalanine. Brain Res. 98, 194–201. doi: 10.1016/0006-8993(75)90521-1
Miyazaki, K. W., Miyazaki, K., and Doya, K. (2012). Activation of dorsal raphe serotonin neurons is necessary for waiting for delayed rewards. J. Neurosci. 32, 10451–10457. doi: 10.1523/JNEUROSCI.0915-12.2012
Miyazaki, K., Miyazaki, K. W., and Doya, K. (2012). The role of serotonin in the regulation of patience and impulsivity. Mol. Neurobiol. 45, 213–224. doi: 10.1007/s12035-012-8232-6
Miyazaki, K. W., Miyazaki, K., Tanaka, K. F., Yamanaka, A., Takahashi, A., Tabuchi, S., et al. (2014). Optogenetic activation of dorsal raphe serotonin neurons enhances patience for future rewards. Curr. Biol. 24, 2033–2040. doi: 10.1016/j.cub.2014.07.041
Müller, C. P., and Homberg, J. R. (2014). The role of serotonin in drug use and addiction. Behav. Brain Res. 277, 146–192. doi: 10.1016/j.bbr.2014.04.007
Nakamura, K., Matsumoto, M., and Hikosaka, O. (2008). Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus. J Neurosci. 28, 5331–5343. doi: 10.1523/JNEUROSCI.0021-08.2008
Ogawa, S. K., Cohen, J. Y., Hwang, D., Uchida, N., and Watabe-Uchida, M. (2014). Organization of monosynaptic inputs to the serotonin and dopamine neuromodulatory systems. Cell Rep. 8, 1105–1118. doi: 10.1016/j.celrep.2014.06.042
Olds, M. E., and Yuwiler, A. (1992). Effects of acute and chronic fenfluramine on self-stimulation and its facilitation by amphetamine. Eur. J. Pharmacol. 216, 363–372. doi: 10.1016/0014-2999(92)90432-4
Palminteri, S., Clair, A.-H., Mallet, L., and Pessiglione, M. (2012). Similar improvement of reward and punishment learning by serotonin reuptake inhibitors in obsessive-compulsive disorder. Biol. Psychiatry 72, 244–250. doi: 10.1016/j.biopsych.2011.12.028
Pollak Dorocic, I., Fürth, D., Xuan, Y., Johansson, Y., Pozzi, L., Silberberg, G., et al. (2014). A whole-brain atlas of inputs to serotonergic neurons of the dorsal and median raphe nuclei. Neuron 83, 663–678. doi: 10.1016/j.neuron.2014.07.002
Qi, J., Zhang, S., Wang, H.-L., Wang, H., de Jesus Aceves Buendia, J., Hoffman, A. F., et al. (2014). A glutamatergic reward input from the dorsal raphe to ventral tegmental area dopamine neurons. Nat. Commun. 5:5390. doi: 10.1038/ncomms6390
Robinson, O. J., Cools, R., and Sahakian, B. J. (2012). Tryptophan depletion disinhibits punishment but not reward prediction: implications for resilience. Psychopharmacology 219, 599–605. doi: 10.1007/s00213-011-2410-5
Rogers, R. D., Tunbridge, E. M., Bhagwagar, Z., Drevets, W. C., Sahakian, B. J., and Carter, C. S. (2002). Tryptophan depletion alters the decision-making of healthy volunteers through altered processing of reward cues. Neuropsychopharmacology 28, 153–162. doi: 10.1038/sj.npp.1300001
Roiser, J. P., Blackwell, A. D., Cools, R., Clark, L., Rubinsztein, D. C., Robbins, T. W., et al. (2006). Serotonin transporter polymorphism mediates vulnerability to loss of incentive motivation following acute tryptophan depletion. Neuropsychopharmacology 31, 2264–2272. doi: 10.1038/sj.npp.1301055
Rompre, P.-P., and Miliaressis, E. (1985). Pontine and mesencephalic substrates of self-stimulation. Brain Res. 359, 246–259. doi: 10.1016/0006-8993(85)91435-0
Rygula, R., Clarke, H. F., Cardinal, R. N., Cockcroft, G. J., Xia, J., Dalley, J. W., et al. (2015). Role of central serotonin in anticipation of rewarding and punishing outcomes: effects of selective amygdala or orbitofrontal 5-HT depletion. Cereb. Cortex 25, 3064–3076. doi: 10.1093/cercor/bhu102
Sachs, B. D., Ni, J. R., and Caron, M. G. (2015). Brain 5-HT deficiency increases stress vulnerability and impairs antidepressant responses following psychosocial stress. Proc. Natl. Acad. Sci. U S A 112, 2557–2562. doi: 10.1073/pnas.1416866112
Sanders, A. C., Hussain, A. J., Hen, R., and Zhuang, X. (2007). Chronic blockade or constitutive deletion of the serotonin transporter reduces operant responding for food reward. Neuropsychopharmacology 32, 2321–2329. doi: 10.1038/sj.npp.1301368
Scholl, J., Kolling, N., Nelissen, N., Browning, M., Rushworth, M. F. S., and Harmer, C. J. (2017). Beyond negative valence: 2-week administration of a serotonergic antidepressant enhances both reward and effort learning signals. PLoS Biol. 15:e2000756. doi: 10.1371/journal.pbio.2000756
Scott, M. M., Krueger, K. C., and Deneris, E. S. (2005). A differentially autoregulated pet-1 enhancer region is a critical target of the transcriptional cascade that governs serotonin neuron development. J. Neurosci. 25, 2628–2636. doi: 10.1523/jneurosci.4979-04.2005
Sego, C., Gonçalves, L., Lima, L., Furigo, I. C., Donato, J. Jr., and Metzger, M. (2014). Lateral habenula and the rostromedial tegmental nucleus innervate neurochemically distinct subdivisions of the dorsal raphe nucleus in the rat. J. Comp. Neurol. 522, 1454–1484. doi: 10.1002/cne.23533
Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., and Dolan, R. (2012). Serotonin selectively modulates reward value in human decision-making. J. Neurosci. 32, 5833–5842. doi: 10.1523/jneurosci.0053-12.2012
Soubrié, P. (1986). Reconciling the role of central serotonin neurons in human and animal behavior. Behav. Brain Sci. 9, 319–335. doi: 10.1017/s0140525x00022871
Stuber, G. D., Hnasko, T. S., Britt, J. P., Edwards, R. H., and Bonci, A. (2010). Dopaminergic terminals in the nucleus accumbens but not the dorsal striatum corelease glutamate. J. Neurosci. 30, 8229–8233. doi: 10.1523/jneurosci.1754-10.2010
Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., and Yamawaki, S. (2004). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci. 7, 887–893. doi: 10.1038/nn1279
Tanaka, S. C., Schweighofer, N., Asahi, S., Shishida, K., Okamoto, Y., Yamawaki, S., et al. (2007). Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum. PLoS One 2:e1333. doi: 10.1371/journal.pone.0001333
Tanaka, S. C., Shishida, K., Schweighofer, N., Okamoto, Y., Yamawaki, S., and Doya, K. (2009). Serotonin affects association of aversive outcomes to past actions. J. Neurosci. 29, 15669–15674. doi: 10.1523/jneurosci.2799-09.2009
Tancer, M., and Johanson, C.-E. (2003). Reinforcing, subjective, and physiological effects of MDMA in humans: a comparison with d-amphetamine and mCPP. Drug Alcohol Depend. 72, 33–44. doi: 10.1016/s0376-8716(03)00172-8
Tecuapetla, F., Patel, J. C., Xenias, H., English, D., Tadros, I., Shah, F., et al. (2010). Glutamatergic signaling by mesolimbic dopamine neurons in the nucleus accumbens. J. Neurosci. 30, 7105–7110. doi: 10.1523/JNEUROSCI.0265-10.2010
Tye, N. C., Everitt, B. J., and Iversen, S. D. (1977). 5-Hydroxytryptamine and punishment. Nature 268, 741–743. doi: 10.1038/268741a0
Ullsperger, M., Fischer, A. G., Nigbur, R., and Endrass, T. (2014). Neural mechanisms and temporal dynamics of performance monitoring. Trends Cogn. Sci. 18, 259–267. doi: 10.1016/j.tics.2014.02.009
Van Der Kooy, D., Fibiger, H. C., and Phillips, A. G. (1978). An analysis of dorsal and median raphe self-stimulation: effects of para-chlorophenylalanine. Pharmacol. Biochem. Behav. 8, 441–445. doi: 10.1016/0091-3057(78)90083-7
Walker, S. C., Robbins, T. W., and Roberts, A. C. (2009). Differential contributions of dopamine and serotonin to orbitofrontal cortex function in the marmoset. Cereb. Cortex 19, 889–898. doi: 10.1093/cercor/bhn136
Wang, R. Y., and Aghajanian, G. K. (1977). Physiological evidence for habenula as major link between forebrain and midbrain raphe. Science 197, 89–91. doi: 10.1126/science.194312
Warden, M. R., Selimbeyoglu, A., Mirzabekov, J. J., Lo, M., Thompson, K. R., Kim, S.-Y., et al. (2013). A prefrontal cortex-brainstem neuronal projection that controls response to behavioural challenge. Nature 492, 428–432. doi: 10.1038/nature11617
Watabe-Uchida, M., Zhu, L., Ogawa, S. K., Vamanrao, A., and Uchida, N. (2012). Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74, 858–873. doi: 10.1016/j.neuron.2012.03.017
Wee, S., Anderson, K. G., Baumann, M. H., Rothman, R. B., Blough, B. E., and Woolverton, W. L. (2005). Relationship between the serotonergic activity and reinforcing effects of a series of amphetamine analogs. J. Pharmacol. Exp. Ther. 313, 848–854. doi: 10.1124/jpet.104.080101
Wee, S., and Woolverton, W. L. (2006). Self-administration of mixtures of fenfluramine and amphetamine by rhesus monkeys. Pharmacol. Biochem. Behav. 84, 337–343. doi: 10.1016/j.pbb.2006.05.022
Weissbourd, B., Ren, J., DeLoach, K. E., Guenthner, C. J., Miyamichi, K., and Luo, L. (2014). Presynaptic partners of dorsal raphe serotonergic and GABAergic neurons. Neuron 83, 645–662. doi: 10.1016/j.neuron.2014.06.024
Woods, J. H., and Tessel, R. E. (1974). Fenfluramine: amphetamine congener that fails to maintain drug-taking behavior in the rhesus monkey. Science 185, 1067–1069. doi: 10.1126/science.185.4156.1067
Worbe, Y., Palminteri, S., Savulich, G., Daw, N. D., Fernandez-Egea, E., Robbins, T. W., et al. (2016). Valence-dependent influence of serotonin depletion on model-based choice strategy. Mol. Psychiatry 21, 624–629. doi: 10.1038/mp.2015.46
Wyvell, C. L., and Berridge, K. C. (2000). Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward “wanting” without enhanced ‘liking’ or response reinforcement. J. Neurosci. 20, 8122–8130.
Yang, L.-M., Hu, B., Xia, Y.-H., Zhang, B.-L., and Zhao, H. (2008). Lateral habenula lesions improve the behavioral response in depressed rats via increasing the serotonin level in dorsal raphe nucleus. Behav. Brain Res. 188, 84–90. doi: 10.1016/j.bbr.2007.10.022
Young, S. N. (1996). Behavioral effects of dietary neurotransmitter precursors: basic and clinical aspects. Neurosci. Biobehav. Rev. 20, 313–323. doi: 10.1016/0149-7634(95)00022-4
Zhou, J., Jia, C., Feng, Q., Bao, J., and Luo, M. (2015). Prospective coding of dorsal raphe reward signals by the orbitofrontal cortex. J. Neurosci. 35, 2717–2730. doi: 10.1523/jneurosci.4017-14.2015
Keywords: serotonin, 5-HT, dopamine, reward, optogenetics, drugs, co-release
Citation: Fischer AG and Ullsperger M (2017) An Update on the Role of Serotonin and its Interplay with Dopamine for Reward. Front. Hum. Neurosci. 11:484. doi: 10.3389/fnhum.2017.00484
Received: 02 June 2017; Accepted: 19 September 2017;
Published: 11 October 2017.
Edited by:
Isadora Olivé, University of Western Ontario, CanadaReviewed by:
Mitsuko Watabe-Uchida, Harvard University, United StatesRoshan Cools, University of Cambridge, United Kingdom
Copyright © 2017 Fischer and Ullsperger. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Adrian G. Fischer, adrian.fischer@ovgu.de