- 1 Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA
- 2 Centre for Cognitive Neuroimaging, Donders Institute for Brain, Cognition and Behavior, Radboud University Nijmegen Medical Centre, Nijmegen, Netherlands
- 3 Department of Psychiatry, Radboud University Nijmegen Medical Centre, Nijmegen, Netherlands
Brain dopamine has long been known to be implicated in the domains of appetitive motivation and cognition. Recent work indicates that dopamine also plays a role in the interaction between appetitive motivation and cognition. Here we review this work. Animal work has revealed an arrangement of spiraling connections between the midbrain and the striatum that subserves a mechanism by which dopamine can direct information flow from ventromedial to more dorsal regions in the striatum. In line with current knowledge about dopamine’s effects on cognition, we hypothesize that these striato-nigro-striatal connections provide the basis for functionally specific effects of appetitive motivation on cognition. One implication of this hypothesis is that appetitive motivation can induce cognitive improvement or impairment depending on task demands.
Introduction
The ability to control our behavior requires our actions to be goal-directed, and our goals to be organized hierarchically. Goals can be defined at different levels: motivational goals (e.g., rewards), cognitive goals (e.g., task-sets), and action goals (e.g., stimulus–response mappings). Thus, goal-directed behavior requires, among other things, the transformation of information about reward into abstract cognitive decisions, which in turn need to be translated into specific actions. The mechanisms underlying this hierarchy of goal-directed control are not well understood.
This paper focuses on the degree to which such goal-directed behavior is controlled by incentive motivation. We have restricted our discussion to the effects of appetitive motivation, while taking note of the wealth of evidence indicating that stimuli that activate the appetitive motivational system have an inhibitory influence on behavior that is controlled by the aversive motivational system (Konorsky, 1967; Dickinson and Balleine, 2002). Unlike aversive motivation, appetitive motivation refers to the state triggered by external stimuli that have rewarding properties and has been argued to have a general potentiating or enhancing effect on behavior and cognition (Dickinson and Balleine, 2002; Robbins and Everitt, 2003; Krawczyk et al., 2007; Pessoa, 2009; Jimura et al., 2010; Pessoa and Engelmann, 2010; Savine and Braver, 2010). Its effects on behavior and cognition have been associated with changes in neurochemical activity, such as increases in dopamine signaling in the striatum (Lyon and Robbins, 1975; Ikemoto and Panksepp, 1999; Robbins and Everitt, 2003; Berridge, 2007). This observation is generally in keeping with proposals that dopamine plays an important role in reward-related effort (Salamone et al., 2007) and generalized activation/energization of behavior (Robbins and Everitt, 2007). It is also consistent with data suggesting that dopamine might direct information flow from ventromedial frontostriatal circuits, implicated in reward and motivation, to more dorsal frontostriatal circuits, associated with cognition and action (Alexander et al., 1986; Haber and Knutson, 2010; Figure 1).
Figure 1. Ventromedial to dorsolateral direction of information flow through frontostriatal-nigral circuitry. Interactions between the different frontostriatal loops involved in motivational control (red), cognitive control (yellow), and motor control (blue) can take place at the level of the SNS connections (bend arrows) or at the level of the frontostriatal connections (straight arrows). The direction of information flow is always from ventromedial to dorsolateral regions in the frontostriatal circuitry. SNS, striato-nigral-striatal; N. Acc, nucleus accumbens (ventromedial striatum); Caud, caudate nucleus (dorsomedial striatum); Put, putamen (dorsolateral striatum); OFC, orbitofrontal cortex; ACC, anterior cingulate cortex; DLPFC, dorsolateral prefrontal cortex; PMC, premotor cortex.
Although the widely distributed and diffuse nature of its projection system to large parts of the forebrain concurs with an account of dopamine in relatively non-specific terms, such as serving activation or energization, it is also clear that dopamine does not simply amplify (or suppress) all forebrain activity in a functionally non-specific manner. Indeed extensive evidence indicates that effects of dopamine depend on specific task demands and their underlying neural systems (Robbins, 2000; Cools et al., 2001a; Frank et al., 2004). In line with these insights, we suggest here that changes in appetitive motivation, which may result from changes in neurochemical activity, for example, due to stress, fatigue, or neuropsychiatric abnormality, also have functionally selective consequences for cognition.
More specifically, we put forward the working hypothesis that appetitive motivation might promote selectively our ability to switch between different tasks, providing us with some of the cognitive flexibility that is required in our constantly changing environment. Conversely, we speculate, based on preliminary data, that dopamine-mediated appetitive motivation might also have detrimental consequences for cognition, e.g., by impairing cognitive focusing and increasing distractibility. The implication of this speculation is that dopamine-mediated appetitive motivation might potentiate flexible behavior, albeit not by potentiating the impact of current goals on behavior. This speculation stems partly from the recognition that the motivational forces that drive behavior are not always under goal-direct control and can be maladaptive (Dickinson and Balleine, 2002). Moreover dopamine is well known to play an important role in mediating the detrimental (i.e., non-goal-directed) consequences of reward (Berridge, 2007; Robbins and Everitt, 2007).
Our working hypothesis is grounded in (albeit preliminary) empirical evidence indicating opposite effects of both dopaminergic and motivational/affective state manipulations on cognitive flexibility and cognitive focusing, which have been argued to reflect distinct striatal and prefrontal brain regions respectively (Crofts et al., 2001; Bilder et al., 2004; Dreisbach and Goschke, 2004; Dreisbach, 2006; Hazy et al., 2006; Cools et al., 2007; Rowe et al., 2007; van Steenbergen et al., 2009; Cools and D’Esposito, 2011). Indeed current models highlight a role for dopamine, particularly in the striatum, in the flexible updating of current task-representations (Hazy et al., 2006; Maia and Frank, 2011). The finding that appetitive motivation is associated with robust changes in dopamine levels particularly in the striatum, thus concurs with our hypothesis that appetitive motivation potentiates (at least some forms of) cognitive flexibility, perhaps even at the expense of cognitive focusing. Such a bias toward cognitive flexibility should be generally adaptive, given that motivational goals in the real world are not often readily available, thus requiring preparatory behavior that is flexible rather than focused (Baldo and Kelley, 2007).
Together these observations suggest that appetitive motivation acts to enhance cognition in a manner that is functionally specific, varying as a function of task demands, and that these functionally specific effects are mediated by dopamine. Clearly, as in the case of dopamine (Cools and Robbins, 2004; Cools et al., 2009), effects of appetitive motivation will vary not only as a function of task demands, but also as a function of the baseline state of the system. Thus both motivational and neurochemical state changes will have rather different effects in individuals with low and high baseline levels of motivation, consistent with the existence of multiple Yerkes Dodson “inverted U shaped” functions (Yerkes and Dodson, 1908; Cools and Robbins, 2004).
Let us briefly discuss the role of striatal dopamine in the two separate domains of motivation and cognitive control before addressing its role in their interaction.
Dopamine and Appetitive Motivation
The ventromedial striatum (VMS, including the nucleus accumbens) is highly innervated by mesolimbic dopaminergic neurons and is well known to be implicated in reward and motivation (Robbins and Everitt, 1992; Berridge and Robinson, 1998; Ikemoto and Panksepp, 1999; Schultz, 2002; Knutson and Cooper, 2005; Baldo and Kelley, 2007). Thus dopamine manipulations in the VMS affect performance on multiple paradigms thought to measure motivated behavior, including conditioned reinforcement, Pavlovian-instrumental transfer paradigms, effort-based decision making tasks, and progressive ratio schedules (Taylor and Robbins, 1984; Dickinson et al., 2000; Wyvell and Berridge, 2000, 2001; Parkinson et al., 2002). These experiments primarily reveal effects of dopamine on so-called preparatory conditioned responses, which are thought to reflect activation of a motivational system (Dickinson and Balleine, 2002), while leaving unaffected, or if anything, having the opposite effect on the more stereotypic patterns of consummatory responding (Robbins and Everitt, 1992; Baldo and Kelley, 2007). Thus administration of the indirect catecholamine enhancer amphetamine in the VMS of hungry rats potentiated locomotor excitement in the presence of food and increased lever pressing in response to, or in anticipation of a reward-predictive cue, while decreasing or leaving unaffected food intake as well as appetitive hedonic responses like taste reactivity (Taylor and Robbins, 1984; Bakshi and Kelley, 1991; Pecina et al., 1997; Wyvell and Berridge, 2000, 2001). Conversely, dopamine receptor blockade or dopamine lesions in the VMS reduced locomotor activity and cue-evoked incentive motivation for reward (Dickinson et al., 2000; Parkinson et al., 2002), while again leaving unaffected or even increasing food intake (Koob et al., 1978). These animal studies emphasize the importance of VMS dopamine in appetitive motivation and suggest that the hedonic or consummatory aspects of reward are likely mediated by a different, possible antagonistic system (Floresco et al., 1996; Robbins and Everitt, 1996, 2003; Berridge and Robinson, 1998; Ikemoto and Panksepp, 1999; Baldo and Kelley, 2007; Berridge, 2007; Phillips et al., 2007; Salamone et al., 2007; for similar suggestions in humans, see Aarts et al., 2010).
At first sight, this well-established observation provides apparently clear grounds for assuming that dopamine contributes to optimal reward- or goal-directed behavior. However, psychologists have also long recognized that there are multiple distinct components to the motivation of behavior (Konorsky, 1967; Dickinson and Balleine, 2002). Thus instrumental behavior is motivated not only by the goals that we set ourselves, but also by generalized drives and/or so-called Pavlovian “wanting,” the latter two processes not necessarily always contributing to adaptive, optimized behavior. To clarify this point, it may help to consider the operational definition that psychologists have invoked for distinguishing instrumental behavior that is goal-directed from instrumental behavior that is not goal-directed, i.e., habitual (Dickinson and Balleine, 2002). Following this tradition, behavior is goal-directed only if it accords to two criteria; first, it has to be driven by knowledge about the contingency between the action and the outcome (as measured with contingency degradation tests); second, it has to be sensitive to changes in the value of the goal (as measured with outcome devaluation tests, involving for example selective satiety). Using these operational definitions, Dickinson and Balleine (2002) have established that Pavlovian conditioned stimuli that induce so-called “wanting” can modify instrumental behavior without accessing action–outcome representations, that is, in a manner that is not goal-directed. This is illustrated most clearly by the role of reward-predictive stimuli in compulsive craving for drugs of abuse or other targets of addiction, which of course almost always implicates dopamine dysfunction (Berridge and Robinson, 1998; Everitt and Robbins, 2005; Volkow et al., 2009). In keeping with this observation are suggestions that motivational influences on instrumental behavior by Pavlovian stimulus reinforcer contingencies might reflect modulation of well-established habits rather than of goal-directed behavior (Dickinson and Balleine, 2002). Data showing that dopamine D1/D2 receptor antagonists attenuated Pavlovian-instrumental transfer without affecting instrumental incentive learning (Dickinson et al., 2000) indeed suggested that dopamine might act through Pavlovian processes rather than through modifying action–outcome representations (Dickinson and Balleine, 2002).
In this context, it is perhaps not surprising that the effects of appetitive motivation on cognition that are mediated by dopamine are functionally specific, leading to cognitive improvement or cognitive impairment depending on the specific task demands under study. An important implication of this observation is that effects of dopamine on interactions between motivation and cognitive control that appear to be mediated by a modification of motivational influences on cognitively mediated, goal-directed behavior may in fact reflect modification of motivational influences on habitual behavior.
Dopamine and Cognition
Accumulating evidence in the domain of cognition indicates that manipulations of dopamine can have contrasting effects as a function of task demands. For example, opposite effects have been observed in terms of cognitive flexibility and cognitive focusing (Crofts et al., 2001; Bilder et al., 2004; Cools et al., 2007; Durstewitz and Seamans, 2008; Durstewitz et al., 2010; Cools and D’Esposito, 2011). Mehta et al. (2004) have shown that dopamine D2 receptor blockade after acute administration of the antagonist sulpiride impaired cognitive flexibility (measured in terms of task-switching), but improved cognitive focusing (measured in terms of delayed response performance with task-irrelevant distractors). Similar contrasting effects on cognitive flexibility and focusing have been reported after dopamine lesions in non-human primates (Roberts et al., 1994; Collins et al., 2000; Crofts et al., 2001), after dopaminergic medication withdrawal in patients with Parkinson’s disease (PD; Cools et al., 2001a, 2003, 2010a) and as a function of genetic variation in human dopamine genes (Bilder et al., 2004; Colzato et al., 2010). Evidence from functional neuroimaging and computational modeling work has suggested that these opposite effects might reflect modulation of distinct brain regions, with the striatum mediating effects on at least some forms of cognitive flexibility, but the prefrontal cortex (PFC) mediating effects on cognitive focusing (Hazy et al., 2006; Cools et al., 2007; Cools and D’Esposito, 2011). This hypothesis likely reflects an oversimplified view of dopamine’s complex effects on cognition, with different forms of cognitive flexibility implicating distinct neural and neurochemical systems (Robbins and Arnsten, 2009; Kehagia et al., 2010; Floresco and Jentsch, 2011). In particular, the striatum seems implicated predominantly in a form of cognitive flexibility that involves shifting to well-established (“habitized”) stimulus–response sets, that does not require new learning or working memory. For example 6-OHDA lesions in the striatum of marmosets impaired set-shifting to an already established set, but left unaffected set-shifting to a new, to-be-learned set (Collins et al., 2000). This finding paralleled the beneficial effects of dopaminergic medication in PD, which implicates primarily the striatum. These effects were restricted to task-set switching between well-established sets, and did not extend to set shifting to new, to-be-learned sets (Cools et al., 2001b; Lewis et al., 2005; Slabosz et al., 2006). The PFC might well be implicated in higher-order forms of set shifting that do involve new learning and/or working memory (Monchi et al., 2004; Floresco and Magyar, 2006; Cools et al., 2010b; Kehagia et al., 2010). Interestingly, the beneficial effects of dopaminergic medication in PD on this striatal form of well-established, habit-like task-set switching were accompanied by detrimental effects on cognitive focusing, as measured in terms of distractor–resistance during the performance of a delayed response task (Cools et al., 2010a). These findings paralleled pharmacological neuroimaging work with the same delayed response paradigm demonstrating that effects of dopamine D1/D2 receptor agonist administration to healthy young volunteers on flexibility (task-switching) and focusing (distractor–resistance) were accompanied by drug effects on the striatum and the PFC respectively (Cools et al., 2007).
In sum, dopamine’s effects on cognition are known to be functionally specific rather than global, with opposite effects on cognitive flexibility and cognitive focusing. These opposite effects have been proposed to reflect modulation of distinct brain regions, with dopamine in the striatum playing a prominent role in a form of flexibility that involves shifting to well-established, i.e., “habitized” stimulus–response sets.
Dopamine and the Motivation–Cognition Interaction
So far we have seen that striatal dopamine’s effect on motivated behavior is most prominent in terms of its preparatory component and that such preparatory effects can be maladaptive. This observation that dopamine’s effect on motivation might have maladaptive consequences for behavior concurs with observations that effects of dopamine in the cognitive domain depend on task demands and associated neural systems, so that dopaminergic drugs can have detrimental as well as beneficial consequences for cognition. Together these insights have led to the speculation that incentive motivation might act to enhance cognitive performance by potentiating dopamine in the striatum in a manner that is functionally specific, i.e., restricted to a form of cognitive flexibility that involves shifting to well-established habits, and not extending to, or even at the expense of cognitive focusing. Below we review empirical evidence that address the different aspects of this working hypothesis.
Evidence from Neuroanatomical Studies
Motivation–cognition interactions have long been proposed to reflect dopamine-dependent interfacing between different parallel frontostriatal circuits associated with motivation and cognition (Figure 1). For example, neuroanatomical studies in rats from the 1970s have suggested that activity in the dorsal striatum is modulated by activity in the ventral striatum via the dopaminergic cells in the substantia nigra (Nauta et al., 1978). Tracer experiments in non-human primates have revived this notion by revealing an arrangement of spiraling striato-nigro-striatal (SNS) connections between the dopaminergic cells in the midbrain and striatal regions that were defined on the basis of their frontal cortical input (Haber et al., 2000; Haber, 2003). Similar connections have been found in rodents (Ikemoto, 2007). The SNS connections are thought to direct information flow in a feed-forward manner via stepwise disinhibition of the ascending dopaminergic projections from the VMS (including the nucleus accumbens), via the dorsomedial striatum (DMS, caudate nucleus), to the dorsolateral striatum (DLS, putamen). The resulting information flow from ventromedial to dorsolateral striatal regions provides a hierarchical (or heterarchical, see Haruno and Kawato, 2006) mechanism by which motivational goals can influence cognitive and subsequent motor control processes. Indeed, the VMS has long been hypothesized to provide the basis for the interface between motivation and action on the basis of its major inputs from limbic areas like the amygdala, hippocampus and the anterior cingulate cortex (ACC) and output to the motor areas via the globus pallidus (Mogenson et al., 1980; Groenewegen et al., 1996). However, rather than a direct limbic-motor connection, the SNS connections provide a more physiologically and psychologically plausible mechanism by which motivational goals exert their influence on action (Haber et al., 2000).
Evidence from Psychopharmacological Studies in Animals
Rodent research on drug addiction has provided evidence for the functional importance of dopamine-mediated interactions between ventral and dorsal parts of the striatum. For example, Belin and Everitt (2008) have adopted an intrastriatal disconnection procedure in rats to investigate the necessity of the SNS connections in the transition of reward-directed drug-seeking behavior to habitual behavior associated with the DLS. The authors lesioned the VMS selectively on one side of the rat brain and, concomitantly, blocked dopaminergic input from the substantia nigra in the DLS with a receptor antagonist on the contralateral side of the brain. Thus, they functionally disconnected the VMS and DLS on both sides of the brain, while leaving unilateral VMS and DLS on opposite sites intact. This functional disconnection between VMS and DLS greatly reduced the transition of VMS-associated to DLS-associated habitual behavior, whereas the unilateral manipulations were ineffective in isolation (Belin and Everitt, 2008). These data show the functional importance of the spiraling SNS connections in VMS control over dorsal striatal functioning in addiction (Belin et al., 2009).
Functional evidence for a role of dopamine in interactions between motivation and DMS-associated functions has also been established in non-human primates. For example, neurophysiological recordings by Hikosaka and colleagues during the performance of a memory-guided saccadic eye-movement task revealed sensitivity of neuronal firing in the DMS as well as midbrain dopamine neurons to appetitive motivation. In this task, one of four directions was randomly assigned as the target location by a cue that also signaled the anticipation of reward. Subsequently, the monkey had to make a saccade to the remembered location. It was found that cues that predicted reward resulted in earlier and faster saccades relative to cues that predicted no reward. Firing patterns in caudate nucleus (DMS) neurons correlated with the change in saccade behavior, changing their preferred direction to the rewarded direction (Kawagoe et al., 1998). In a follow-up study, the authors observed that reward-predictive cues resulted in increased firing of dopaminergic neurons in the midbrain, as well as in neurons of the caudate nucleus (DMS; Kawagoe et al., 2004). Together, these findings demonstrate that effects of reward anticipation on DMS activity and associated motor-planning behavior were accompanied by changes in dopamine activity.
In humans, a role for dopamine in the effects of motivation on cognition has so far been addressed only in the domain of long-term memory associated with the hippocampus (Wittmann et al., 2005; Adcock et al., 2006; Schott et al., 2006; for a review, see Shohamy and Adcock, 2010). This relatively young field suggests that dopamine may well play a role in the long-term plasticity-enhancing effects of motivation. In the next section, we address studies that focus on dopamine-dependent effects of motivation on shorter term plasticity, involving the striatum.
Evidence from Human Studies: Motivation and Cognitive Flexibility
Data from two recent studies support the hypothesis that dopamine is critical for interactions between motivation and cognition. Specifically, these studies highlight an important role for dopamine in the modification by appetitive motivation of switching between well-established habits. The set-shifting paradigm involved cued task-switching between well-learnt task-sets, minimizing learning and working memory processes (Rogers and Monsell, 1995). Subjects switched between responding according to the direction of the arrow (task A) and responding according to the direction indicated by the word (task B) of a series of arrow-word targets (consisting of the words “left” or “right” in a left or right pointing arrow; Figure 2A). Repetitions or switches of task-set were pseudo-randomly preceded by high or low reward cues. In the first study, young healthy adults performed the task in the magnetic resonance scanner and both behavioral and neural responses were assessed as a function of inter-individual variability in dopamine genes (Aarts et al., 2010). In particular, we focused on a common variable number of tandem repeats (VNTR) polymorphism in the dopamine transporter gene (DAT1), expressed predominantly in the striatum. Relative to the 10R homozygotes, the 9R carriers – with presumably increased striatal dopamine levels – exhibited significant reward benefits in terms of overall performance and increased reward-related BOLD responses in VMS. However, most critically, they also demonstrated significant reward benefits in terms of task-switching (i.e., reduced switch costs in the high versus low reward condition). This effect was accompanied by a potentiation of switch-related BOLD responses in DMS (caudate nucleus) in the high reward versus the low reward condition (Figures 2B,C). Importantly, the reward-related activity in VMS correlated positively with the effects of reward on subsequent switch-related activity during the targets in DMS, with high dopamine subjects demonstrating high activity in both striatal regions (Figure 2D; Aarts et al., 2010). These dopamine-mediated motivation–cognition interaction effects were recently replicated in an independent dataset (van Holstein et al., 2011) and strengthened our working hypothesis that striatal dopamine mediates motivational modification of certain cognitive functions in humans.
Figure 2. Experimental evidence for the beneficial effect of motivation on cognitive flexibility in humans. (A) The rewarded set-shifting paradigm used in our studies to investigate the motivation–cognition interface. (B) In our genetic imaging study (Aarts et al., 2010), participants with genetically determined high striatal dopamine levels benefited more from reward anticipation in terms of set-shifting than participants with low dopamine levels. (C) In our genetic imaging study (Aarts et al., 2010), reward cues elicited activity in VMS (in red), whereas the dopamine-dependent effect of reward prediction on set shifting was observed in DMS (in yellow). (D) Activity in these striatal sub-regions [see (C)] was positively correlated, with high striatal dopamine subjects showing high activity in both VMS and DMS during reward anticipation and rewarded set-shifting respectively. (E) In our SPECT study in Parkinson’s disease (Aarts et al., under review), patients showed the most marked dopamine depletion in dorsolateral striatum (posterior putamen), whereas ventromedial striatum (n. accumbens) was least affected. (F) Patients with the greatest dopamine depletion (i.e., least dopamine cell integrity) showed the greatest effects of anticipated reward in reducing the switch cost in the dominant arrow task [(switch-repeat)low − (switch-repeat)high]; presumably by increased reward-induced dopamine release in the relatively intact neurons in ventromedial striatum.
In a second study, we investigated the effect of appetitive motivation on cognitive flexibility in patients with PD using the same paradigm (Figure 2A). Effects within the PD group were associated with the degree of dopamine depletion in different striatal sub-regions as measured with 123I-FP-CIT single photon emission computed tomography (SPECT). First, we replicated previous studies by demonstrating a switch deficit in PD relative to healthy controls. Interestingly, this deficit was restricted to certain conditions of the task, revealing a disproportionate difficulty with switching to the best established, most dominant “arrow” task. Additionally, the SPECT measurements showed that this switch deficit in PD was associated with dopamine cell loss in the most affected striatal sub-region (posterior putamen, Figure 2E), thus demonstrating the involvement of striatal dopamine in this particular “habit-like” type of cognitive flexibility. More critically, our results demonstrated compensatory capacity of reward-predictive signals to facilitate cognitive flexibility in mild PD. Specifically, when anticipating reward, patients were able to reduce the switch cost in the dominant arrow task to such an extent that the switch cost no longer differed from that of controls on high reward trials. Interestingly, the use of reward was also highly correlated with the amount of dopamine depletion in the most affected striatal sub-region (Aarts et al., under review). Patients with greater dopamine cell loss made more use of anticipated reward for reducing the switch cost than did patients with less dopamine cell loss (Figure 2F). Further exploration of this finding demonstrated that this effect of motivation on task-switching was driven by two opponent processes: first, patients with more dopamine depletion made more errors on repeat trials under high than under low reward. This detrimental effect of reward on repeat trials could reflect a form of impulsivity, where the current task representation is rendered unstable by reward, leading to reduced cognitive “perseverance” or maintenance (see also Hazy et al., 2006). Controls did not show such detrimental impulsive behavior on repeat trials under high reward. Second, patients with more dopamine depletion made fewer errors on switch trials under high than under low reward. Thus, anticipated reward proved beneficial for switching to the other task-set, which profits from reduced cognitive perseverance. This effect of reward on switch trials in patients did not differ from that of controls. The beneficial effects of anticipated reward on task-switching in the young healthy adults mentioned above (Aarts et al., 2010) was driven by a beneficial effect of reward on switch trials only, instead of opposite effects of reward on repeat and switch trials. In sum, PD patients differed from controls in showing detrimental effects of reward on repeat trials, which were greatest in patients with most dopamine cell loss in the striatum (Aarts et al., under review). This result fits with previous findings that a low baseline dopamine state contributes to trait impulsivity and addictive behavior (Cools et al., 2007; Dalley et al., 2007); presumably due to reduced auto-regulatory mechanisms, resulting in increased dopamine release (Buckholtz et al., 2010). Hence, we speculate that reward-induced impulsivity in our PD group was caused by increased reward-related dopamine release in the relatively intact dopamine cells projecting to the ventral striatum (Figure 2E). In line with this view are the findings of increased dopamine release in ventral striatum in PD patients diagnosed with impulsive–compulsive behavior relative to those without (Evans et al., 2006; Steeves et al., 2009; O’Sullivan et al., 2011). Our PD data are also in accordance with the working hypothesis that striatal dopamine mediates motivational effects on cognition depending on task demands.
Evidence from Human Studies: Functionally Specific Effects of Motivation
Motivation has been shown to improve attentional processes in many perceptual and cognitive control domains (for reviews, see Pessoa, 2009; Pessoa and Engelmann, 2010). Data from a number of human imaging studies have suggested that motivation might have non-specific enhancing effects on cognitive processing. For example, in a functional neuroimaging study, motivational incentives increased PFC activity and connectivity during cognitive control tasks, in a manner that seemed to depend on the cognitive effort (i.e., cost–benefit ratio) rather than on the specific qualitative cognitive demand of the tasks (Kouneiher et al., 2009). Based on these data the authors argued that motivation and cognitive control can be regarded as two separate, additive instead of interactive factors of executive functioning (Kouneiher et al., 2009). However, such an additive view of motivation and cognition contrasts with the conclusion drawn by a different set of recent studies which enabled the disentangling of different cognitive control components. These studies have found that effects of appetitive motivation and affect may well depend on the type of cognitive processing at hand (Dreisbach and Goschke, 2004; Dreisbach, 2006; Rowe et al., 2007), consistent with our working hypothesis. Before turning to these studies, we will discuss preliminary data from our own lab.
So far we have seen that appetitive motivation can potentiate certain forms of task-switching to well-established stimulus–response mappings in a dopamine-dependent manner. The observation that these effects were driven by detrimental effects of anticipated reward on repeat trials and beneficial effects on switch trials in the PD group (Aarts et al., submitted) already indicates a level of functional specificity. To test more directly the hypothesis that these beneficial effects of appetitive motivation on some cognitive functions might come at the expense of impairments on other cognitive functions, we designed a Stroop-like conflict task with high and low reward conditions. This task resembled the previously used task-switching paradigm in many ways except that it required cognitive focusing instead of cognitive switching. Seventeen participants performed this Stroop-like task by responding with a left or right button press to the words “left” or “right” in a left or right pointing arrow (Figure 3A). The direction denoted by the word was either congruent or incongruent with the direction indicated by the arrow. Similar to the task-switching paradigm discussed above (Aarts et al., under review), all trials began with a cue predicting high or low reward for correct performance. Critically, following the reward cues, we explicitly informed participants about the (in)congruency of the upcoming Stroop target (see Aarts et al., 2008). In half of the trials, participants were informed about this congruency by informative cues (Figure 3A). In the other half of the trials, the targets were preceded by cues that gave no information about the upcoming congruency. The idea here was that incongruency-predictive cues (relative to non-informative cues) would encourage participants to reduce their attentional focus, whereas the congruency-predictive cues would encourage participants to widen their attentional focus. In other words, cues that signaled upcoming incongruent targets would encourage participants to proactively focus on the task-relevant word, preventing distraction by the task-irrelevant arrow, whereas cues that signal upcoming congruent words encouraged participants to proactively widen attention in order to comprise both the task-relevant word as well as the task-irrelevant arrow (see Aarts and Roelofs, 2010). The combination of reward and information cues enabled us to determine the effects of appetitive motivation on the cognitive focusing of attention.
Figure 3. Incentive motivation might have detrimental effects on cognitive focusing. (A) The rewarded Stroop paradigm, including a reward cue (1 or 15 cent), an information cue about the upcoming target congruency [informative: incongruent (this example) or congruent (green circle); or uninformative (gray question mark)], and an arrow-word Stroop target. The task was to respond to the direction indicated by the word. (B) Reward anticipation had opposite effects on widening and focusing of attention as measured with the information benefit (uninformed–informed) on congruent and incongruent targets respectively; with high anticipated reward particularly impairing proactive focusing on the incongruent trials (M. van Holstein, E. Aarts, R. Cools, unpublished observations).
Consistent with our previous results (Aarts et al., 2008) we showed that (irrespective of reward condition) participants responded faster and made less errors when informative cues preceded the congruent and incongruent targets relative to uninformed targets (M. van Holstein, E. Aarts, R. Cools, unpublished observations). Importantly, as predicted, appetitive motivation significantly altered the information benefit depending on the congruency of the targets. That is, proactive widening of attention (uninformed–informed congruent targets) benefited from anticipated reward (15 versus 1 cent), whereas proactive focusing of attention (uninformed–informed incongruent targets) was hampered by anticipated reward (Figure 3B). Intriguingly, these data show that, depending on the task at hand, appetitive motivation can have both beneficial as well as detrimental effects on cognitive function.
Similar findings have been obtained when studying the effects of positive affect on cognitive control. Thus, positive affect has been shown to increase cognitive flexibility (i.e., decreasing perseveration), while increasing distractibility (i.e., decreasing cognitive stability) on different types of trials in a set-shifting paradigm (Dreisbach and Goschke, 2004). Opposite effects have also been observed in an AX continuous performance task: positive affect increased cognitive flexibility when a maintained goal unexpectedly changed (Dreisbach, 2006; van Wouwe et al., 2011), but, within the same task, positive affect decreased the ability to maintain the goal when nothing changed (Dreisbach, 2006). Functionally specific effects of positive affect have also been demonstrated in conflict paradigms, like the Eriksen flanker task. Some authors have shown that positive affect increased attention toward the distracting flanker arrows, thus, increasing “the breadth of attentional selection” (Rowe et al., 2007); others have found that positive affect reduced the ability to focus on the target arrow after experienced conflict (van Steenbergen et al., 2010). Our preliminary results from the rewarded Stroop conflict paradigm extend these effects of positive affect in the flanker conflict task, by revealing contrasting effects of appetitive motivation on the widening and focusing of attention within the same task and within the same participants. In sum, both appetitive motivation and positive affect enhance certain forms of cognitive flexibility at the expense of cognitive focusing. According to our working hypothesis, these effects might reflect dopamine-dependent flow of information processing related to Pavlovian incentives from ventromedial parts of the striatum to more dorsal regions in the striatum, associated with habit-like information processing.
It might be noted here again that multiple mechanisms have been proposed to underlie the motivational control of behavior (Dickinson and Balleine, 2002). We have highlighted that some motivational influences can be maladaptive, and these might implicate dopamine. However, there is also evidence for motivational influences on goal-direct behavior, that is, those mediated by instrumental incentive learning and acquisition of action–outcome representations (Dickinson and Balleine, 2002). These alternate mechanisms might account for findings that at first sight seem incompatible with the current working hypothesis. Specifically, appetitive motivation has been shown to increase spatial orienting to a target location in the face of distractors (Engelmann and Pessoa, 2007; Engelmann et al., 2009), or to reduce conflict by biasing visual selection (Padmala and Pessoa, 2011). Furthermore, in young and old adults as well as in medicated patients with PD, motivation increased anti-saccade performance, encompassing incompatible stimulus–response mappings like in Stroop and flanker paradigms (Harsay et al., 2010). The critical question is whether these effects are also dependent on striatal dopamine, or whether they implicate modulation by different neurochemical systems. Addressing this question requires controlled dopaminergic medication withdrawal and/or pharmacological manipulation approaches.
Frontal Control of Dopamine-Dependent Striatal Processing
The striatum does not act alone and requires interactions with specific frontal regions to operate effectively (Alexander et al., 1986; Passingham, 1993; Figure 1). Recent neuroimaging work in humans and monkeys has revealed that effects of appetitive motivation on cognitive control are accompanied by modulation of responses in the PFC (Ichihara-Takeda and Funahashi, 2008; Kouneiher et al., 2009; Beck et al., 2010; Ichihara-Takeda et al., 2010; Jimura et al., 2010; Savine and Braver, 2010; Wallis and Kennerley, 2010). For example, functional interactions between the medial and the lateral PFC have been shown to accompany effects of appetitive motivation on the cognitive control processes involved in set shifting (Kouneiher et al., 2009). Another functional neuroimaging study concluded that the lateral PFC incorporates reward value in goal-directed control during working memory processes (Jimura et al., 2010).
These data concur with the existence of multiple mechanisms for the motivational control of behavior, which may interact in multiple ways, either competitively or synergistically. For example, signals in the PFC might control dopaminergic activity in striatal areas in a top-down manner, thus allowing controlled influences on value assignment to states or actions (Daw et al., 2005; Doll et al., 2009; see Figure 1). Consistent with this hypothesis are observations that stimulation of different parts of the frontal cortex (using transcranial magnetic stimulation) alters focal dopamine release in strongly connected topographically specific parts of the striatum (as measured using [11C]raclopride positron emission tomography; Strafella et al., 2001, 2003, 2005; Ko et al., 2008). The role of the PFC in integrating motivation, cognition, and action is also highlighted by anatomical tracer studies in non-human primates showing that value-sensitive regions in ventromedial PFC (i.e., ACC/orbitofrontal cortex) project not only to strongly connected regions in VMS, but also diffusely to more dorsal regions in the striatum that receive most projections from the DLPFC (Haber et al., 2006; Figure 1). Electrophysiological work with rodents has revealed that changes in dopamine release and receptor stimulation in the striatum can alter such PFC input to the striatum (Goto and Grace, 2005). More specifically, changes in tonic dopamine release were shown to modulate PFC inputs into the VMS – and to influence set-shifting behavior – through dopamine D2 receptors (Goto and Grace, 2005). These results show that striatal dopamine can modulate motivated behavior not only via altering striatal output but also via altering striatal input from the PFC.
Conclusion and Future Directions
There are multiple mechanisms for the control of behavior and cognition by motivation. This paper focuses on the appetitive motivational system, while recognizing that opponent influences on behavior are likely seen of the aversive motivational system. In particular, we have concentrated on those effects of appetitive motivation that implicate dopamine. These dopamine-dependent effects of motivation likely have both detrimental as well as beneficial consequences for cognition, via altering information flow from ventromedial to dorsolateral parts of the striatum. This general observation is in line with the observation that motivational influences on behavior are not necessarily driven by representations of the goals of instrumental behavior, but might well reflect Pavlovian or habit-like anomalies. This is particularly likely in the case of dopamine, which is recognized to play a special role in Pavlovian and habit systems.
An important implication of this observation is that effects of dopamine on interactions between motivation and cognition that appear to be mediated by a modification of motivational influences on cognitively mediated, goal-directed behavior, like task-switching, may in fact reflect modification of motivation influences on habitual behavior. Findings that the dopamine-dependent effects of motivation on task-switching are strongest when participants are required to switch to well-established stimulus–response mappings are in line with this hypothesis, which requires testing in future work.
A further issue to be addressed in future research is the degree to which the contrasting effects of motivation on habit-like switching and on proactive focusing can be understood in terms of competition between a striatal system controlling habit-like processing and a prefrontal system controlling goal-directed behavior (Dickinson, 1985; Daw et al., 2005). Clearly these questions require a careful integration of traditional psychological approaches, which leverage well-operationalized behavioral definitions of goal-directed and habitual behavior, with pharmacological studies of cognitive control.
Furthermore, given the proposed opponency between appetitive and aversive motivational systems, one might ask what is the effect of punishment-predictive stimuli on cognition? This is particularly interesting in the context of empirical findings that conditioned inhibitors, i.e., stimuli predictive of reward omission do not trigger an increase, but rather if anything a decrease in midbrain dopamine firing (Tobler et al., 2005). Moreover, there is increasing speculation about the involvement of the part-opponent system of serotonin (Daw et al., 2002; Dayan and Huys, 2009; Boureau and Dayan, 2011; Cools et al., 2011), an area that is wide open for empirical work.
Finally, progress in the understanding of the motivational control of cognition will depend on the degree to which the balance between transient and sustained, e.g., context effects are taken into account (e.g., Higgins et al., 1997; Maddox and Markman, 2010; Savine et al., 2010). For example, Maddox and Markman (2010) propose that performance does not only depend on local incentives and task demands (as discussed in the current review), but also interacts with global incentives like an overall bonus or punishment at the end of a task. Such advances will no doubt benefit from the recognition that the impact of transient (phasic) changes in neurotransmitter activity depends critically on the tonic neurochemical state of the system.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
Roshan Cools is supported by a fellowship of the Hersenstichting Nederland for dementia research [F2008 (1)-01], a Vidi Grant from the Innovational Research Incentives Scheme of the Netherlands Organization for Scientific Research and a Human Frontiers Research Program grant. Esther Aarts is supported by a stipend of the Niels Stensen foundation.
References
Aarts, E., and Roelofs, A. (2010). Attentional control in anterior cingulate cortex based on probabilistic cueing. J. Cogn. Neurosci. 23, 716–727.
Aarts, E., Roelofs, A., Franke, B., Rijpkema, M., Fernandez, G., Helmich, R. C., and Cools, R. (2010). Striatal dopamine mediates the interface between motivational and cognitive control in humans: evidence from genetic imaging. Neuropsychopharmacology 35, 1943–1951.
Aarts, E., Roelofs, A., and van Turennout, M. (2008). Anticipatory activity in anterior cingulate cortex can be independent of conflict and error likelihood. J. Neurosci. 28, 4671–4678.
Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., and Gabrieli, J. D. (2006). Reward-motivated learning: mesolimbic activation precedes memory formation. Neuron 50, 507–517.
Alexander, G. E., DeLong, M. R., and Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357–381.
Bakshi, V. P., and Kelley, A. E. (1991). Dopaminergic regulation of feeding behavior: II. Differential effects of amphetamine microinfusion into three striatal subregions. Psychobiology 19, 233–242.
Baldo, B. A., and Kelley, A. E. (2007). Discrete neurochemical coding of distinguishable motivational processes: insights from nucleus accumbens control of feeding. Psychopharmacology (Berl.) 191, 439–459.
Beck, S. M., Locke, H. S., Savine, A. C., Jimura, K., and Braver, T. S. (2010). Primary and secondary rewards differentially modulate neural activity dynamics during working memory. PLoS ONE 5, e9251. doi: 10.1371/journal.pone.0009251
Belin, D., and Everitt, B. J. (2008). Cocaine seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron 57, 432–441.
Belin, D., Jonkman, S., Dickinson, A., Robbins, T. W., and Everitt, B. J. (2009). Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav. Brain Res. 199, 89–102.
Berridge, K. C. (2007). The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology (Berl.) 191, 391–431.
Berridge, K. C., and Robinson, T. E. (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369.
Bilder, R. M., Volavka, J., Lachman, H. M., and Grace, A. A. (2004). The catechol-O-methyltransferase polymorphism: relations to the tonic-phasic dopamine hypothesis and neuropsychiatric phenotypes. Neuropsychopharmacology 29, 1943–1961.
Boureau, Y. L., and Dayan, P. (2011). Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36, 74–97.
Buckholtz, J. W., Treadway, M. T., Cowan, R. L., Woodward, N. D., Li, R., Ansari, M. S., Baldwin, R. M., Schwartzman, A. N., Shelby, E. S., Smith, C. E., Kessler, R. M., and Zald, D. H. (2010). Dopaminergic network differences in human impulsivity. Science 329, 532.
Collins, P., Wilkinson, L. S., Everitt, B. J., Robbins, T. W., and Roberts, A. C. (2000). The effect of dopamine depletion from the caudate nucleus of the common marmoset (Callithrix jacchus) on tests of prefrontal cognitive function. Behav. Neurosci. 114, 3–17.
Colzato, L. S., Waszak, F., Nieuwenhuis, S., Posthuma, D., and Hommel, B. (2010). The flexible mind is associated with the catechol-O-methyltransferase (COMT) Val158Met polymorphism: evidence for a role of dopamine in the control of task-switching. Neuropsychologia 48, 2764–2768.
Cools, R., Barker, R. A., Sahakian, B. J., and Robbins, T. W. (2001a). Enhanced or impaired cognitive function in Parkinson’s disease as a function of dopaminergic medication and task demands. Cereb. Cortex 11, 1136–1143.
Cools, R., Barker, R. A., Sahakian, B. J., and Robbins, T. W. (2001b). Mechanisms of cognitive set flexibility in Parkinson’s disease. Brain 124(Pt 12), 2503–2512.
Cools, R., Barker, R. A., Sahakian, B. J., and Robbins, T. W. (2003). L-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson’s disease. Neuropsychologia 41, 1431–1441.
Cools, R., and D’Esposito, M. (2011). Inverted-U-shaped dopamine actions on human working memory and cognitive control. Biol. Psychiatry 69, e113–e125.
Cools, R., Frank, M. J., Gibbs, S. E., Miyakawa, A., Jagust, W., and D’Esposito, M. (2009). Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration. J. Neurosci. 29, 1538–1543.
Cools, R., Miyakawa, A., Sheridan, M., and D’Esposito, M. (2010a). Enhanced frontal function in Parkinson’s disease. Brain 133(Pt 1), 225–233.
Cools, R., Rogers, R., Barker, R. A., and Robbins, T. W. (2010b). Top-down attentional control in Parkinson’s disease: salient considerations. J. Cogn. Neurosci. 22, 848–859.
Cools, R., Nakamura, K., and Daw, N. D. (2011). Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 36, 98–113.
Cools, R., and Robbins, T. W. (2004). Chemistry of the adaptive mind. Philos. Transact. A Math. Phys. Eng. Sci. 362, 2871–2888.
Cools, R., Sheridan, M., Jacobs, E., and D’Esposito, M. (2007). Impulsive personality predicts dopamine-dependent changes in frontostriatal activity during component processes of working memory. J. Neurosci. 27, 5506–5514.
Crofts, H. S., Dalley, J. W., Collins, P., Van Denderen, J. C., Everitt, B. J., Robbins, T. W., and Roberts, A. C. (2001). Differential effects of 6-OHDA lesions of the frontal cortex and caudate nucleus on the ability to acquire an attentional set. Cereb. Cortex 11, 1015–1026.
Dalley, J. W., Fryer, T. D., Brichard, L., Robinson, E. S., Theobald, D. E., Laane, K., Pena, Y., Murphy, E. R., Shah, Y., Probst, K., Abakumova, I., Aigbirhio, F. I., Richards, H. K., Hong, Y., Baron, J. C., Everitt, B. J., and Robbins, T. W. (2007). Nucleus accumbens D2/3 receptors predict trait impulsivity and cocaine reinforcement. Science 315, 1267–1270.
Daw, N. D., Kakade, S., and Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616.
Daw, N. D., Niv, Y., and Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711.
Dickinson, A. (1985). Actions and habits: the development of behavioural autonomy. Philos. Trans. R. Soc. Lond. 308B, 67–78.
Dickinson, A., and Balleine, B. W. (2002). “The role of learning in the operation of motivational systems,” in Stevens’ Handbook of Experimental Psychology Vol. 3: Learning, Motivation and Emotion, ed. C. R. Gallistel (New York: Wiley), 497–533.
Dickinson, A., Smith, J., and Mirenowicz, J. (2000). Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav. Neurosci. 114, 468–483.
Doll, B. B., Jacobs, W. J., Sanfey, A. G., and Frank, M. J. (2009). Instructional control of reinforcement learning: a behavioral and neurocomputational investigation. Brain Res. 1299, 74–94.
Dreisbach, G. (2006). How positive affect modulates cognitive control: the costs and benefits of reduced maintenance capability. Brain Cogn. 60, 11–19.
Dreisbach, G., and Goschke, T. (2004). How positive affect modulates cognitive control: reduced perseveration at the cost of increased distractibility. J. Exp. Psychol. Learn Mem. Cogn. 30, 343–353.
Durstewitz, D., and Seamans, J. K. (2008). The dual-state theory of prefrontal cortex dopamine function with relevance to catechol-O-methyltransferase genotypes and schizophrenia. Biol. Psychiatry 64, 739–749.
Durstewitz, D., Vittoz, N. M., Floresco, S. B., and Seamans, J. K. (2010). Abrupt transitions between prefrontal neural ensemble states accompany behavioral transitions during rule learning. Neuron 66, 438–448.
Engelmann, J. B., Damaraju, E., Padmala, S., and Pessoa, L. (2009). Combined effects of attention and motivation on visual task performance: transient and sustained motivational effects. Front. Hum. Neurosci. 3:4. doi: 10.3389/neuro.09.004.2009
Engelmann, J. B., and Pessoa, L. (2007). Motivation sharpens exogenous spatial attention. Emotion 7, 668–674.
Evans, A. H., Pavese, N., Lawrence, A. D., Tai, Y. F., Appel, S., Doder, M., Brooks, D. J., Lees, A. J., and Piccini, P. (2006). Compulsive drug use linked to sensitized ventral striatal dopamine transmission. Ann. Neurol. 59, 852–858.
Everitt, B. J., and Robbins, T. W. (2005). Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 8, 1481–1489.
Floresco, S. B., and Jentsch, J. D. (2011). Pharmacological enhancement of memory and executive functioning in laboratory animals. Neuropsychopharmacology 36, 227–250.
Floresco, S. B., and Magyar, O. (2006). Mesocortical dopamine modulation of executive functions: beyond working memory. Psychopharmacology (Berl.) 188, 567–585.
Floresco, S. B., Seamans, J. K., and Phillips, A. G. (1996). A selective role for dopamine in the nucleus accumbens of the rat in random foraging but not delayed spatial win-shift-based foraging. Behav. Brain Res. 80, 161–168.
Frank, M. J., Seeberger, L. C., and O’Reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 1940–1943.
Goto, Y., and Grace, A. A. (2005). Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nat. Neurosci. 8, 805–812.
Groenewegen, H. J., Wright, C. I., and Beijer, A. V. (1996). The nucleus accumbens: gateway for limbic structures to reach the motor system? Prog. Brain Res. 107, 485–511.
Haber, S. N. (2003). The primate basal ganglia: parallel and integrative networks. J. Chem. Neuroanat. 26, 317–330.
Haber, S. N., Fudge, J. L., and McFarland, N. R. (2000). Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382.
Haber, S. N., Kim, K. S., Mailly, P., and Calzavara, R. (2006). Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J. Neurosci. 26, 8368–8376.
Haber, S. N., and Knutson, B. (2010). The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35, 4–26.
Harsay, H. A., Buitenweg, J. I., Wijnen, J. G., Guerreiro, M. J., and Ridderinkhof, K. R. (2010). Remedial effects of motivational incentive on declining cognitive control in healthy aging and Parkinson’s disease. Front Aging Neurosci. 2:144. doi: 10.3389/fnagi.2010.00144
Haruno, M., and Kawato, M. (2006). Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus-action-reward association learning. Neural Netw. 19, 1242–1254.
Hazy, T. E., Frank, M. J., and O’Reilly, R. C. (2006). Banishing the homunculus: making working memory work. Neuroscience 139, 105–118.
Higgins, E. T., Shah, J., and Friedman, R. (1997). Emotional responses to goal attainment: strength of regulatory focus as moderator. J. Pers. Soc. Psychol. 72, 515–525.
Ichihara-Takeda, S., and Funahashi, S. (2008). Activity of primate orbitofrontal and dorsolateral prefrontal neurons: effect of reward schedule on task-related activity. J. Cogn. Neurosci. 20, 563–579.
Ichihara-Takeda, S., Takeda, K., and Funahashi, S. (2010). Reward acts as a signal to control delay-period activity in delayed-response tasks. Neuroreport 21, 367–370.
Ikemoto, S. (2007). Dopamine reward circuitry: two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Res. Rev. 56, 27–78.
Ikemoto, S., and Panksepp, J. (1999). The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res. Brain Res. Rev. 31, 6–41.
Jimura, K., Locke, H. S., and Braver, T. S. (2010). Prefrontal cortex mediation of cognitive enhancement in rewarding motivational contexts. Proc. Natl. Acad. Sci. U.S.A. 107, 8871–8876.
Kawagoe, R., Takikawa, Y., and Hikosaka, O. (1998). Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1, 411–416.
Kawagoe, R., Takikawa, Y., and Hikosaka, O. (2004). Reward-predicting activity of dopamine and caudate neurons – a possible mechanism of motivational control of saccadic eye movement. J. Neurophysiol. 91, 1013–1024.
Kehagia, A. A., Murray, G. K., and Robbins, T. W. (2010). Learning and cognitive flexibility: frontostriatal function and monoaminergic modulation. Curr. Opin. Neurobiol. 20, 199–204.
Knutson, B., and Cooper, J. C. (2005). Functional magnetic resonance imaging of reward prediction. Curr. Opin. Neurol. 18, 411–417.
Ko, J. H., Monchi, O., Ptito, A., Bloomfield, P., Houle, S., and Strafella, A. P. (2008). Theta burst stimulation-induced inhibition of dorsolateral prefrontal cortex reveals hemispheric asymmetry in striatal dopamine release during a set-shifting task: a TMS-[(11)C]raclopride PET study. Eur. J. Neurosci. 28, 2147–2155.
Konorsky, J. (1967). Integrative Activity of the Brain: An Interdisciplinary Approach. Chicago: University of Chicago Press.
Koob, G. F., Riley, S. J., Smith, S. C., and Robbins, T. W. (1978). Effects of 6-hydroxydopamine lesions of the nucleus accumbens septi and olfactory tubercle on feeding, locomotor activity, and amphetamine anorexia in the rat. J. Comp. Physiol. Psychol. 92, 917–927.
Kouneiher, F., Charron, S., and Koechlin, E. (2009). Motivation and cognitive control in the human prefrontal cortex. Nat. Neurosci. 12, 939–945.
Krawczyk, D. C., Gazzaley, A., and D’Esposito, M. (2007). Reward modulation of prefrontal and visual association cortex during an incentive working memory task. Brain Res. 1141, 168–177.
Lewis, S., Slabosz, A., Robbins, T. W., Barker, R. A., and Owen, A. M. (2005). Dopaminergic basis for deficits in working memory but not attentional set-shifting in Parkinson’s disease. Neuropsychologia 43, 823–832.
Lyon, M., and Robbins, T. W. (1975). “The action of central nervous system drugs: a general theory concerning amphetamine effects,” in Current Developments in Psychopharmacology, ed. W. Essman (New York: Spectrum Publications), 80–163.
Maddox, W. T., and Markman, A. B. (2010). The motivation-cognition interface in learning and decision-making. Curr. Dir. Psychol. Sci. 19, 106–110.
Maia, T. V., and Frank, M. J. (2011). From reinforcement learning models to psychiatric and neurological disorders. Nat. Neurosci. 14, 154–162.
Mehta, M. A., Manes, F. F., Magnolfi, G., Sahakian, B. J., and Robbins, T. W. (2004). Impaired set-shifting and dissociable effects on tests of spatial working memory following the dopamine D2 receptor antagonist sulpiride in human volunteers. Psychopharmacology (Berl.) 176, 331–342.
Mogenson, G. J., Jones, D. L., and Yim, C. Y. (1980). From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 14, 69–97.
Monchi, O., Petrides, M., Doyon, J., Postuma, R. B., Worsley, K., and Dagher, A. (2004). Neural bases of set-shifting deficits in Parkinson’s disease. J. Neurosci. 24, 702–710.
Nauta, W. J., Smith, G. P., Faull, R. L., and Domesick, V. B. (1978). Efferent connections and nigral afferents of the nucleus accumbens septi in the rat. Neuroscience 3, 385–401.
O’Sullivan, S. S., Wu, K., Politis, M., Lawrence, A. D., Evans, A. H., Bose, S. K., Djamshidian, A., Lees, A. J., and Piccini, P. (2011). Cue-induced striatal dopamine release in Parkinson’s disease-associated impulsive-compulsive behaviours. Brain 134(Pt 4), 969–978.
Padmala, S., and Pessoa, L. (2011). Reward reduces conflict by enhancing attentional control and biasing visual cortical processing. J. Cogn. Neurosci. PMID: 21452938. [Epub ahead of print].
Parkinson, J. A., Dalley, J. W., Cardinal, R. N., Bamford, A., Fehnert, B., Lachenal, G., Rudarakanchana, N., Halkerston, K. M., Robbins, T. W., and Everitt, B. J. (2002). Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behav. Brain Res. 137, 149–163.
Passingham, R. E. (1993). The Frontal Lobes and Voluntary Action. New York, NY: Oxford University Press.
Pecina, S., Berridge, K. C., and Parker, L. A. (1997). Pimozide does not shift palatability: separation of anhedonia from sensorimotor suppression by taste reactivity. Pharmacol. Biochem. Behav. 58, 801–811.
Pessoa, L. (2009). How do emotion and motivation direct executive control? Trends Cogn. Sci. (Regul. Ed.) 13, 160–166.
Pessoa, L., and Engelmann, J. B. (2010). Embedding reward signals into perception and cognition. Front. Neurosci. 4:17. doi: 10.3389/fnins.2010.00017
Phillips, P. E., Walton, M. E., and Jhou, T. C. (2007). Calculating utility: preclinical evidence for cost-benefit analysis by mesolimbic dopamine. Psychopharmacology (Berl.) 191, 483–495.
Robbins, T. W. (2000). Chemical neuromodulation of frontal-executive functions in humans and other animals. Exp. Brain Res. 133, 130–138.
Robbins, T. W., and Arnsten, A. F. (2009). The neuropsychopharmacology of fronto-executive function: monoaminergic modulation. Annu. Rev. Neurosci. 32, 267–287.
Robbins, T. W., and Everitt, B. J. (1992). Functions of dopamine in the dorsal and ventral striatum. Neurosciences 4, 119–127.
Robbins, T. W., and Everitt, B. J. (1996). Neurobehavioural mechanisms of reward and motivation. Curr. Opin. Neurobiol. 6, 228–236.
Robbins, T. W., and Everitt, B. J. (2003). “Motivation and reward,” in Fundamental Neuroscience, eds L. R. Squire, F. Bloom, S. McConnellet, J. L. Roberts, N. C. Spitzer, and M. J. Zigmond (London: Academic Press), 1109–1126.
Robbins, T. W., and Everitt, B. J. (2007). A role for mesencephalic dopamine in activation: commentary on Berridge (2006). Psychopharmacology (Berl.) 191, 433–437.
Roberts, A. C., De Salvia, M. A., Wilkinson, L. S., Collins, P., Muir, J. L., Everitt, B. J., and Robbins, T. W. (1994). 6-Hydroxydopamine lesions of the prefrontal cortex in monkeys enhance performance on an analog of the Wisconsin Card Sort Test: possible interactions with subcortical dopamine. J. Neurosci. 14(5 Pt 1), 2531–2544.
Rogers, R. D., and Monsell, S. (1995). Costs of a predictable switch between simple cognitive tasks. J. Exp. Psychol. Gen. 124, 207–231.
Rowe, G., Hirsh, J. B., and Anderson, A. K. (2007). Positive affect increases the breadth of attentional selection. Proc. Natl. Acad. Sci. U.S.A. 104, 383–388.
Salamone, J. D., Correa, M., Farrar, A., and Mingote, S. M. (2007). Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology (Berl.) 191, 461–482.
Savine, A. C., Beck, S. M., Edwards, B. G., Chiew, K. S., and Braver, T. S. (2010). Enhancement of cognitive control by approach and avoidance motivational states. Cogn. Emot. 24, 338–356.
Savine, A. C., and Braver, T. S. (2010). Motivated cognitive control: reward incentives modulate preparatory neural activity during task-switching. J. Neurosci. 30, 10294–10305.
Schott, B. H., Seidenbecher, C. I., Fenker, D. B., Lauer, C. J., Bunzeck, N., Bernstein, H. G., Tischmeyer, W., Gundelfinger, E. D., Heinze, H. J., and Duzel, E. (2006). The dopaminergic midbrain participates in human episodic memory formation: evidence from genetic imaging. J. Neurosci. 26, 1407–1417.
Shohamy, D., and Adcock, R. A. (2010). Dopamine and adaptive memory. Trends Cogn. Sci. (Regul. Ed.) 14, 464–472.
Slabosz, A., Lewis, S. J., Smigasiewicz, K., Szymura, B., Barker, R. A., and Owen, A. M. (2006). The role of learned irrelevance in attentional set-shifting impairments in Parkinson’s disease. Neuropsychology 20, 578–588.
Steeves, T. D., Miyasaki, J., Zurowski, M., Lang, A. E., Pellecchia, G., Van Eimeren, T., Rusjan, P., Houle, S., and Strafella, A. P. (2009). Increased striatal dopamine release in Parkinsonian patients with pathological gambling: a [11C] raclopride PET study. Brain 132(Pt 5), 1376–1385.
Strafella, A. P., Ko, J. H., Grant, J., Fraraccio, M., and Monchi, O. (2005). Corticostriatal functional interactions in Parkinson’s disease: a rTMS/[11C]raclopride PET study. Eur. J. Neurosci. 22, 2946–2952.
Strafella, A. P., Paus, T., Barrett, J., and Dagher, A. (2001). Repetitive transcranial magnetic stimulation of the human prefrontal cortex induces dopamine release in the caudate nucleus. J. Neurosci. 21, RC157.
Strafella, A. P., Paus, T., Fraraccio, M., and Dagher, A. (2003). Striatal dopamine release induced by repetitive transcranial magnetic stimulation of the human motor cortex. Brain 126(Pt 12), 2609–2615.
Taylor, J. R., and Robbins, T. W. (1984). Enhanced behavioural control by conditioned reinforcers following microinjections of d-amphetamine into the nucleus accumbens. Psychopharmacology (Berl.) 84, 405–412.
Tobler, P. N., Fiorillo, C. D., and Schultz, W. (2005). Adaptive coding of reward value by dopamine neurons. Science 307, 1642–1645.
van Holstein, M., Aarts, E., van der Schaaf, M. E., Geurts, D. E., Verkes, R. J., Franke, B., van Schouwenburg, M. R., and Cools, R. (2011). Human cognitive flexibility depends on dopamine D2 receptor signaling. Psychopharmacology (Berl.). PMID: 21611724. [Epub ahead of print].
van Steenbergen, H., Band, G. P., and Hommel, B. (2009). Reward counteracts conflict adaptation: evidence for a role of affect in executive control. Psychol. Sci. 20, 1473–1477.
van Steenbergen, H., Band, G. P., and Hommel, B. (2010). In the mood for adaptation: how affect regulates conflict-driven control. Psychol. Sci. 21, 1629–1634.
van Wouwe, N. C., Band, G. P., and Ridderinkhof, K. R. (2011). Positive affect modulates flexibility and evaluative control. J. Cogn. Neurosci. 23, 524–539.
Volkow, N., Fowler, J. S., Wang, G. J., Baler, R., and Telang, F. (2009). Imaging dopamine’s role in drug abuse and addiction. Neuropharmacology 56(Suppl. 1), 3–8.
Wallis, J. D., and Kennerley, S. W. (2010). Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198.
Wittmann, B. C., Schott, B. H., Guderian, S., Frey, J. U., Heinze, H. J., and Duzel, E. (2005). Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation. Neuron 45, 459–467.
Wyvell, C. L., and Berridge, K. C. (2000). Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward “wanting” without enhanced “liking” or response reinforcement. J. Neurosci. 20, 8122–8130.
Wyvell, C. L., and Berridge, K. C. (2001). Incentive sensitization by previous amphetamine exposure: increased cue-triggered “wanting” for sucrose reward. J. Neurosci. 21, 7831–7840.
Keywords: dopamine, motivation, cognition, striatum, flexibility, prefrontal cortex, reward, Parkinson’s disease
Citation: Aarts E, van Holstein M and Cools R (2011) Striatal dopamine and the interface between motivation and cognition. Front. Psychology 2:163. doi: 10.3389/fpsyg.2011.00163
Received: 01 March 2011;
Paper pending published: 21 April 2011;
Accepted: 30 June 2011;
Published online: 14 July 2011
Edited by:
Wim Notebaert, Ghent University, BelgiumReviewed by:
K. Richard Ridderinkhof, University of Amsterdam, NetherlandsPatrick Santens, Gent University, Belgium
Copyright: © 2011 Aarts, van Holstein and Cools. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.
*Correspondence: Esther Aarts, Helen Wills Neuroscience Institute, University of California at Berkeley, 132 Barker Hall, Berkeley, CA 94720-3190, USA. e-mail: estheraarts@berkeley.edu