Skip to main content

REVIEW article

Front. Behav. Neurosci., 07 September 2009
Sec. Motivation and Reward
This article is part of the Research Topic Neuroeconomics View all 13 articles

Dopamine, behavioral economics, and effort

Department of Psychology, University of Connecticut, Storrs, CT, USA
There are numerous problems with the hypothesis that brain dopamine (DA) systems, particularly in the nucleus accumbens, directly mediate the rewarding or primary motivational characteristics of natural stimuli such as food. Research and theory related to the functions of mesolimbic DA are undergoing a substantial conceptual restructuring, with the traditional emphasis on hedonia and primary reward yielding to other concepts and lines of inquiry. The present review is focused upon the involvement of nucleus accumbens DA in behavioral activation and effort-related processes. Viewed from the framework of behavioral economics, the effects of accumbens DA depletions and antagonism on food-reinforced behavior are highly dependent upon the work requirements of the instrumental task, and DA depleted rats are more sensitive to increases in response costs (i.e., ratio requirements). Moreover, interference with accumbens DA transmission exerts a powerful influence over effort-related choice behavior. Rats with accumbens DA depletions or antagonism reallocate their instrumental behavior away from food-reinforced tasks that have high response requirements, and instead these rats select a less-effortful type of food-seeking behavior. Nucleus accumbens DA and adenosine interact in the regulation of effort-related functions, and other brain structures (anterior cingulate cortex, amygdala, ventral pallidum) also are involved. Studies of the brain systems regulating effort-based processes may have implications for understanding drug abuse, as well as energy-related disorders such as psychomotor slowing, fatigue or anergia in depression and other neurological disorders.

Limitations of the Reward Hypothesis of Dopaminergic Function

The last several years have seen substantial theoretical developments related to the hypothesized behavioral functions of nucleus accumbens dopamine (DA). It has become evident to many investigators that there are conceptual limitations and empirical problems with the traditional DA hypothesis of “reward” (Baldo and Kelley, 2007 ; Salamone et al., 1997 , 2005 , 2007 ; Barbano and Cador, 2007 ; ; Salamone, in press ). Even the use of the term “reward” itself often is problematic (Cannon and Bseikri, 2004 ; Salamone et al., 2005 ; Salamone, 2006 ; Sanchis-Segura and Spanagel, 2006 ; Yin et al., 2008 ). Researchers rarely define what they mean by “reward” when they are using it to describe a psychological process; some use it as though it were a synonym for “reinforcement”, or in reference to “appetite” or “primary motivation”, while still others employ it as a code word to mean “pleasure”. In some papers, the word “reward” seems to be used as a rather monolithic, all-encompassing term that refers to any and all aspects of appetitive learning, motivation and emotion, whether conditioned or unconditioned. Used in this way, the term reward is a rather blunt instrument. These problems are not merely semantic, as it is difficult to test a hypothesis which maintains that a neurotransmitter mediates such an ill-defined set of functions. It has been suggested that it is advantageous to maintain the distinction between the terms reward and reinforcement; with this usage, reinforcement refers more directly to instrumental learning mechanisms (Wise, 2004 ; Sanchis-Segura and Spanagel, 2006 ), while reward connotes the primary motivational and emotional effects of reinforcing stimuli (Everitt and Robbins, 2005 ; Salamone et al., 2005 , 2007 ).
Against the backdrop of these conceptual and terminological issues, there is a tremendous weight of empirical evidence that has built up against the various iterations of the DA hypothesis of “reward”. It is somewhat ironic that the processes most directly linked to the use of the term reward (i.e., primary motivation, subjective pleasure) are the ones that have proven to be most problematic in terms of demonstrating the involvement of mesolimbic DA (Salamone et al., 2007 ). For example, low doses of DA antagonists and depletions of nucleus accumbens DA have been shown to produce effects that do not closely resemble extinction (Salamone, 1986 ; Salamone et al., 1995 , 1997 ; Rick et al., 2006 ), pre-feeding (Salamone et al., 1991 ; Aberman and Salamone, 1999 ), or appetite suppressant drugs (Cousins et al., 1994 ; Salamone et al., 2002 ; Sink et al., 2008 ). Although it is well known that whole forebrain DA depletions can produce aphagia (i.e., lack of eating), it is DA depletions in the lateral or ventrolateral caudate/putamen, rather than the nucleus accumbens, which have most conclusively been linked to this effect (Ungerstedt, 1971 ; Dunnett and Iversen, 1982 ; Salamone et al., 1993a ). It has been shown repeatedly that nucleus accumbens DA depletions or antagonism do not substantially impair appetite for food, or produce a general disruption of primary food motivation (Ungerstedt, 1971 ; Koob et al., 1978 ; Bakshi and Kelley, 1991 ; Salamone et al., 1993a ). In DA deficient mice, restoration of DA production in caudate putamen, but not nucleus accumbens, was able to rescue feeding behavior (Szczypka et al., 2001 ). In summarizing their findings that injections of DA D1 or D2 family antagonists into either the core or the shell subregions of nucleus accumbens impaired locomotion and rearing, but did not suppress food intake, Baldo et al. (2002) stated that DA receptor blockade “did not abolish the primary motivation to eat” (p. 176).
Furthermore, the idea that nucleus accumbens DA mediates the pleasure associated with positive reinforcers has been strongly challenged (Berridge, 2007 ; Salamone et al., 2007 ; Berridge and Kringlebach, 2008 ). Interference with accumbens DA transmission does not impair appetitive taste reactivity for sucrose (Berridge, 2007 ; Berridge and Kringlebach, 2008 ). Several studies in humans have reported that DA antagonists did not blunt the subjective euphoria produced by drugs of abuse (Gawin, 1986 ; Brauer and De Wit, 1997 ; Haney et al., 2001 ; Nann-Vernotica et al., 2001 ; Wachtel et al., 2002 ). Moreover, the potential role of DA systems in instrumental behavior or learning is not limited to situations involving appetitive motivation. There is considerable evidence that striatal mechanisms in general, and mesolimbic DA in particular, also participate in aspects of aversive learning and aversive motivation (Salamone, 1994 ; Munro and Kokkinidis, 1997 ; Blazquez et al., 2002 ; Pezze and Feldon, 2004 ; Delgado et al., 2008 ; Faure et al., 2008 ; Martinez et al., 2008 ). Although imaging studies often are used to support the idea that nucleus accumbens mediates pleasure (e.g., Sarchiapone et al., 2006 ; Wacker et al., 2009 ), this appears to be oversimplified; indeed, research employing various imaging methods has demonstrated that the human nucleus accumbens also responds to stress, aversion and hyperarousal/irritability (Liberzon et al., 1999 ; Jensen et al., 2003 ; Pavic, 2003 ; Phan et al., 2004 ; Pruessner et al., 2004 ; Levita et al., 2009 ). Physiological and neurochemical studies in animals clearly indicate that DA neuron activity is not simply tied to the delivery of primary reinforcers or pleasurable stimuli. Rather, VTA neuron activity and DA release can be activated by a number of different appetitive and aversive conditions (McCullough and Salamone, 1992 ; McCullough et al., 1993 ; Guarraci and Kapp, 1999 ; Roitman et al., 2004 ; Young, 2004 ; Anstrom and Woodward, 2005 ; Broom and Yamamoto, 2005 ; Marinelli et al., 2005 ; Schultz, 2007a ,b ; Brischoux et al., 2009 ), with changes seen across varying time scales, including tonic, slow phasic and fast phasic signals (Salamone, 1996 ; Salamone et al., 2007 ; Schultz, 2007a ,b ; Salamone, in press ; see also Lapish et al., 2007 for a discussion of various time scales associated with the postsynaptic effects of DA release and DA receptor stimulation).
Of course, one would not want to throw the baby out with the bathwater. It is apparent that mesolimbic DA participates in several complex functions related to aspects of instrumental behavior, learning and incentive motivation, and pavlovian/instrumental interactions (Wise, 2004 ; Everitt and Robbins, 2005 ; Kelley et al., 2005 ; Salamone et al., 2005 , 2007 ; Berridge, 2007 ; Robbins and Everitt, 2007 ; Redgrave et al., 2008 ; Yin et al., 2008 ). The more difficult aspect of research and theory in this area is to ask – which specific aspects? Exploration of these diverse areas of dopaminergic function has become a rich and fruitful area of inquiry. Indeed, this literature is so extensive that a thorough review of the behavioral functions of nucleus accumbens DA is beyond the scope of the present article (see Salamone et al., 2007 ). For the purposes of this special issue, the present review will focus upon the role of nucleus accumbens DA in effort-related processes, with a special emphasis on effort-based choice behavior that depends upon cost/benefit analyses.

Behavioral Activation, Exertion of Effort, and Nucleus Accumbens DA

Even as the popularity of the DA hypothesis of reward was growing during the 1980s, it was becoming apparent that there were alternative conceptual frameworks available for organizing what was known about the behavioral functions of DA systems, particularly mesolimbic DA. Mogenson et al. (1980) suggested that nucleus accumbens acted as a functional interface between the limbic system and the motor system, facilitating the ability of information related to emotion and motivation to impinge upon the neural systems involved in the instigation of action. It had been emphasized for several decades that behavioral activation, i.e., the vigor, persistence and effort seen in the pursuit of motivational stimuli, and the heightened activity induced by conditioned stimuli that predict reinforcers, was a fundamental aspect of motivation (e.g., Cofer and Appley, 1964 ). Several investigators suggested that DA systems were involved in behavioral activation. DA antagonists or accumbens DA depletions were shown to suppress the activities such as excessive drinking, wheel running, and locomotion that are induced by scheduled presentation of food (Robbins and Koob, 1980 ; Wallace et al., 1983 ; Salamone, 1986 , 1988 ). It also was reported that the effects of DA antagonists on reinforced behavior interacted powerfully with the kinetic requirements of the instrumental response. For example, doses of DA antagonists that suppressed reinforced lever pressing had minimal effects on reinforced nose poking behavior (Ettenberg et al., 1981 ; Mekarski, 1988 ). Although 0.1 mg/kg haloperidol severely reduced responding on a fixed ratio (FR) 20 schedule of lever pressing, a dose four times that size had no effect on the reinforced response of simply being in proximity to the food dish on a fixed interval 30 s schedule (Salamone, 1986 ). As this research was being reported, investigators began to employ economic concepts, such as exertion of effort and cost-benefit analyses, to describe the behavioral functions of accumbens DA. Neill and Justice (1981) hypothesized that injection of amphetamine into nucleus accumbens could be increasing the “willingness” of rats to exert effort to obtain a given level of reinforcement. In a contemporary review of the behavioral functions of DA systems (Salamone, 1987 ), it was noted that DA in nucleus accumbens could be involved in the “exertion of effort”, and it was suggested that future experiments could “offer animals choices between various reinforcers that are associated with operants of varying difficulty” (p. 602) so that researchers could determine if the allocation of behavioral resources could be biased toward or away from more or less effortful responses by administration of dopaminergic drugs.
This recognition of dopaminergic involvement in the exertion of effort, and effort-based choices related to cost benefit analyses, fit nicely with an emerging emphasis in the behavioral literature on work, response costs or constraints, and economic models of operant behavior. Several behavioral investigators have emphasized how response costs or constraints affect operant response output (Staddon, 1979 ; Kaufman, 1980 ; Kaufman et al., 1980 ; Foltin, 1991 ). Collier and colleagues studied how work requirements, such as the number of lever presses necessary for obtaining food, could serve as determinants of response output and affect consumption parameters (Collier and Jennings, 1969 ; Johnson and Collier, 1987 ). Economic models of operant behavior have emphasized how a number of factors, including not only reinforcement value, but also conditions related to the characteristics of the instrumental response, can determine behavioral output (Lea, 1978 ; Allison, 1981 , 1993 ; Bickel et al., 2000 ). Hursh et al. (1988) suggested that, in terms of behavioral economics, the price of food reinforcement as a commodity is a cost/benefit ratio expressed as the effort expended per unit of food value consumed. Optimal foraging theory was proposed to account for the observation that the amount of effort or time expended to obtain motivational stimuli was an important determinant of foraging choice (Krebs, 1977 ), an idea that is still very influential in the ethology research today (e.g., Hengeveld et al., 2009 ).
Over the last two decades, several lines of evidence have converged to strengthen the original observation that the effects of interference with DA transmission interact powerfully with the work requirements of an instrumental task. One of the ways of controlling work requirements in an operant schedule is to vary the ratio requirement (i.e., the number of times the animal must press the lever to receive a unit of reinforcement). The effects of the DA antagonist haloperidol on food-reinforced behavior were shown to be dependent upon the particular ratio schedule that was used [i.e., FR1 vs. progressive ratio; Caul and Brindle (2001 )]. Accumbens DA depletions also produce effects that interact powerfully with the ratio requirement of the schedule employed. Ishiwari et al. (2004) found that accumbens DA depletions that substantially impaired FR5 lever pressing had no significant effect on FR1 performance. Aberman and Salamone (1999) systematically studied a wide range of ratio schedules (FR1, 4, 16, and 64) to assess the effects of accumbens DA depletions. While FR1 performance was not affected by DA depletion, and FR4 responding was only transiently and mildly suppressed, the schedules with large ratio requirements (i.e., FR16 and FR64) were severely impaired (Figure 1 A). In fact, DA depleted animals responding on the FR64 schedule showed significantly fewer responses than those responding on the FR16 schedule (Aberman and Salamone, 1999 ). This pattern indicates that accumbens DA depletions exacerbate an effect known as ratio strain. In untreated animals, the overall relation between ratio size and response output is inverted-U shaped. Up to a point, as ratio requirements get larger, animals adjust to this challenge by increasing response output. However, if the ratio requirement is high enough (i.e., if the cost is too high), the animal reaches the point at which additional responses being required actually tend to suppress responding. For normal rats, responding at levels of FR64, FR100 or higher, even if there is only one 45 mg food pellet being delivered, does not seem to be problematic. A completely different function is shown by rats with accumbens DA depletions, which are much more sensitive to the size of the ratio requirement. In behavioral economic terms, this pattern can be described as reflecting an increase in the elasticity of demand for food reinforcement (Salamone et al., 1997 ; Aberman and Salamone, 1999 ;see Figure 1 B). The term elasticity is widely used in economics, but price elasticity of demand refers to the sensitivity of consumption to changes in price (Vuchinich and Heather, 2003 ). Thus, if the ratio requirement is analogous to the price of the commodity (in this case, reinforcement pellets), it appears that rats with accumbens DA depletions are more sensitive than control animals to the price of the food reinforcers. Of course, rats do not use currency to purchase operant pellets; rather, it has been argued that an operant procedure is more of a barter system, in which the rat trades its work (or reductions in leisure) for a commodity (Rachlin, 2003 ). Thus, another way of describing this effect of impaired DA transmission is to say that rats with accumbens DA depletions are more sensitive than control animals to work-related response costs, or that they are less likely to trade high levels of work for food. In another study (Salamone et al., 2001 ), the increased effects of accumbens DA depletions with increasing ratio requirements were observed when rats were tested across a broader range of ratio schedules as high as FR300, even when the overall relation between lever pressing and food delivered per lever press was kept constant (i.e., FR50, one pellet every 50 responses; FR100, two pellets every 100 responses; FR200, four pellets every 200 responses; FR300, six pellets every 300 responses; Salamone et al., 2001 ). Thus, both the magnitude and the organization of the ratio requirement appear to be critical determinants of the sensitivity of an operant schedule to the effects of accumbens DA depletions.
Figure 1. This figure shows the effect of ratio requirement on the number of lever presses emitted and operant pellets consumed in rats with accumbens DA depletions compared to rats in the vehicle control group. Figure (A) is re-drawn based upon Aberman and Salamone (1999) ; these data are depicted in terms of number of responses, as in the original article. For (B), the data are represented as number of operant pellets consumed. Each data point shown is the mean value from each group at each ratio level. Although comparable levels of consumption in DA depleted and control groups were seen with the FR1 schedule, DA-depleted rats showed markedly reduced consumption relative to the control group at higher ratio levels.
In order to be sure that these results reflected the influence of ratio size, as opposed to other variables such as time, additional studies examined the effects of accumbens DA depletions on tandem schedules, in which a ratio requirement was attached to an interval requirement. In a conventional variable interval (VI) schedule, a time interval must elapse before the first response is reinforced, and the particular time interval varies around an average value. A tandem VI/FR schedule has an additional ratio requirement attached to the interval. For example, with a tandem VI 30 s/FR5 schedule, the animal is reinforced for the fifth response after the interval elapses, rather than the first. In this way, one can vary the ratio requirement of a schedule while keeping the programmed time intervals the same. Research employing tandem VI/FR schedules with varying combinations (e.g., VI 30 s/FR5, VI 60 s/FR10, VI 120 s/FR10) has yielded a consistent pattern; accumbens DA depletions do not impair overall response output in rats responding on the conventional VI schedules (i.e., those requiring only one response after the interval), but do substantially reduce responding on the corresponding VI schedule with the higher ratio requirement attached (Correa et al., 2002 ; Mingote et al., 2005 ). These results are consistent with research showing that accumbens DA antagonism did not impair performance on a progressive interval task (Wakabayashi et al., 2004 ), and suggest that interval requirements per se do not pose a severe constraint to rats with compromised DA transmission in nucleus accumbens. This serves to underscore the critical importance of ratio requirements as providing a work-related challenge to rats with accumbens DA depletions or antagonism.
In summarizing these results, Salamone and Correa (2002) stated that nucleus accumbens DA depletions appear to have two major effects: (1) they reduce the response-enhancing effects that moderate-size ratio requirements have on operant responding (i.e., the ascending limb of the function relating ratio requirement to response output), and (2) they enhance the response-suppressing effects that very large ratios have on operant responding (i.e., the descending limb of the function, enhancing ratio strain). Furthermore, finer grained analyses of detailed patterns of responding reveal more insights into the behavioral manifestations of accumbens DA depletions. Accumbens DA depletions produce a slight reduction in the local rate of responding, as indicated by the distribution of inter-response times (Salamone et al., 1993b , 1999 ; Mingote et al., 2005 ). In addition, they enhance pauses in responding (Salamone et al., 1993b ; Mingote et al., 2005 ). The latter may indicate a fragmentation in the pattern of responding (Mingote et al., 2005 ), a reduction in the ability to sustain uninterrupted response output, or a lack of engagement in the task (Nicola, 2007 ). Recently, computational approaches have been used to analyze these effects of accumbens DA depletions on response rate (e.g., Niv et al., 2007 ; Phillips et al., 2007 ). This relation between response output and DA function has been interpreted to mean that DA release in nucleus accumbens could provide a window of opportunistic drive during which the threshold cost expenditure to obtain the reward is decreased (Phillips et al., 2007 ).
In discussing the effects of dopaminergic manipulations on ratio performance, it is useful to mention the term “reinforcement efficacy”, which is sometimes used to describe the effects of drug manipulations on progressive ratio performance. With progressive ratio schedules, the ratio requirement increases as successive ratios are completed, and the “break point” is said to occur at the point at which the animal essentially ceases to respond. One can operationally define reinforcement efficacy in terms of the break point in a progressive ratio schedule (and also by measuring ratio strain in rats responding across different FR schedules). The determination of reinforcement efficacy can be a very useful tool for characterizing some of the fundamental reinforcing actions of drugs that are self-administered, and for comparing self-administration behavior across different substances or classes of substances (e.g., Richardson and Roberts, 1996 ; Marinelli et al., 1998 ; Woolverton and Ranaldi, 2002 ). Used in this manner, reinforcement efficacy is essentially being employed as an empirical descriptor of a particular behavioral outcome. Nevertheless, given the terminological problems mentioned above, it is worth emphasizing that the term “reinforcement efficacy” should not be used simply as a replacement for “reward”, nor should progressive ratio breakpoints be viewed as necessarily providing some direct and unambiguous measure related to the subjective pleasure produced by the stimulus (Salamone, 2006) . Changes in progressive ratio break points can reflect more than just changes in the appetitive motivational properties of a reinforcing stimulus (Richardson and Roberts, 1996 ; Hamill et al., 1999 ). For example, changing the kinetic requirements of the instrumental response (e.g., increasing the height of the lever) was shown to decrease progressive ratio break points (Skjoldager et al., 1993 ; Schmelzeis and Mittleman, 1996 ). Although some researchers have maintained that the break point provides a direct measure of the appetitive motivational characteristics of a stimulus, it is, as explicitly stated in a classic review by Stewart (1974) , more directly a measure of how much work the organism will do in order to obtain that stimulus. Progressive ratio break points and measures of ratio strain are essentially outcomes that result from effort-related decision making processes. The animal is making a cost/benefit choice about whether or not to continue to respond, based partly on factors related to the reinforce itself, but also upon the work-related response costs and time constraints imposed by the ratio schedule. For these reasons, interpretations of the actions of drugs or lesions on progressive ratio break points should be done with caution, as should be the case for any individual task. A drug that alters the break point could do so for many different reasons; it may be affecting functions related to the processing of reward value, or alternatively it could be affecting exertion of effort, or decision making processes.

Response Allocation, Effort-Related Choice Behavior, and Nucleus Accumbens DA

The ability to exert effort, sustain work, overcome obstacles, and attain access to motivationally relevant stimuli is necessary for survival. But it is only part of the story. In a complex environment, which affords many opportunities for obtaining significant stimuli, and multiple paths for accessing them, organisms must make choices. The variables that need to be evaluated to make these decisions are complex and multidimensional, but among the most important are those involving cost/benefit assessments based upon effort and reinforcement value (Salamone and Correa, 2002 ; Salamone et al., 2003 , 2005 , 2007 ; van den Bos et al., 2006 ; Walton et al., 2006 ). Considerable evidence indicates that nucleus accumbens DA, along with other transmitters and structures, participates in the overall circuitry that regulates effort-based choice behavior (Salamone et al., 2003 , 2005 , 2007 ; Floresco et al., 2008a ; Hauber and Sommer, 2009 ).
One of the procedures that has been used to assess the contribution of accumbens DA to response allocation and effort-related choice behavior is a task that offers rats the option of either lever pressing to obtain a relatively preferred food (e.g., Bioserve pellets; usually obtained on a FR5 schedule), or approaching and consuming a less preferred food (lab chow) that is concurrently available in the chamber. Well trained rats under baseline conditions typically get most of their food by lever pressing, and consume only small quantities of chow (Salamone et al., 1991 ). Low-to-moderate doses of DA antagonists, which block either D1 or D2 family receptor subtypes, produce a substantial alteration of response allocation in rats performing on this task. The DA antagonists cis-flupenthixol, haloperidol, raclopride, eticlopride, SCH 23390, SKF83566, and ecopipam all decreased lever pressing for food but substantially increased intake of the concurrently available chow (Salamone et al., 1991 , 1996 , 2002 ; Cousins et al., 1994 ; Sink et al., 2008 ; Worden et al., 2009 ). The use of this task for assessing effort-related choice behavior has been validated in many ways. For example, the low dose of haloperidol that produced the shift from lever pressing to chow intake (0.1 mg/kg) did not affect total food intake or alter preference between these two specific foods in free-feeding choice tests (Salamone et al., 1991 ). Although DA antagonists have been shown to reduce FR5 lever pressing and increase chow intake, appetite suppressants from different classes, including amphetamine (Cousins et al., 1994 ), fenfluramine (Salamone et al., 2002 ), and cannabinoid CB1 antagonists (Sink et al., 2008 ), failed to increase chow intake at doses that suppressed lever pressing. Similarly, pre-feeding to reduce food motivation was shown to suppress both lever pressing and chow intake (Salamone et al., 1991 ). Furthermore, attachment of higher ratio requirements (up to FR20) caused animals that were not drug treated to shift from lever pressing to chow intake (Salamone et al., 1997 ), indicating that this task is sensitive to work load. Together with other results, these findings demonstrate that interference with DA transmission does not simply reduce appetite, but does act to alter response allocation between alternative sources of food that can be obtained through different instrumental responses.
The shift from lever pressing to chow intake in rats performing on this task is associated with DA depletions in nucleus accumbens, but not the neostriatum. Although it has been suggested that caudate/putamen DA may have some types of motivational functions related to feeding (Palmiter, 2007 ), DA depletions in anteroventromedial neostriatum, which is dorsal to nucleus accumbens, had no behavioral effect, while ventrolateral neostriatal DA depletions produced severe motor impairments that merely decreased both lever pressing and feeding (Cousins et al., 1993 ). In contrast, decreases in lever pressing and increases in chow intake occur as a result of accumbens DA depletions, as well as intra-accumbens injections of D1 or D2 antagonists (Salamone et al., 1991 ; Cousins et al., 1993 ; Cousins and Salamone, 1994 ; Sokolowski and Salamone, 1998 ; Koch et al., 2000 ; Nowend et al., 2001 ). The shift from lever pressing to chow intake on this task has been shown to occur in rats if D1 or D2 family antagonist are injected into the medial core, lateral core, or dorsal shell subregions of the accumbens (Salamone et al., 1991 ; Nowend et al., 2001 ). Thus, although lever pressing is decreased by accumbens DA antagonism or depletions, the rats show a compensatory reallocation of behavior and select a new path to an alternative food source. Consistent with these effects observed in rats that have impaired DA transmission, DA transporter knockdown mice, which have enhanced DA transmission, show increased selection of lever pressing relative to chow intake when tested with this task (Cagniard et al., 2006 ).
Salamone et al. (1994) also developed a T-maze procedure in order to assess the effects of DA antagonists and accumbens DA depletions on effort-related decision making. With this procedure, the two choice arms of the maze can have different reinforcement densities (e.g., four vs. two food pellets, or four vs. zero food pellets), and under some conditions a 44-cm barrier can be placed in the arm with the higher density of food reinforcement to present an effort-related challenge to the rat. When no barrier is placed in the arm with the high reinforcement density, rats mostly choose that arm, and neither haloperidol nor accumbens DA depletion alters their response choice (Salamone, 1994 ). When the arm with the barrier contained four pellets, but the other arm contained no pellets, rats with accumbens DA depletions were very slow, but still managed to choose the high density arm, climb the barrier, and consume the pellets (Cousins et al., 1996 ). Yet accumbens DA depletions dramatically altered choice behavior when the high density arm (four pellets) had the barrier in position, and the arm without the barrier contained an alternative food source (two pellets). In this case, DA depletions or antagonism decreased choice for the high density arm, and increased choice for the low density arm (Salamone, 1994 ; Cousins et al., 1996 ; Denk et al., 2005 ; Mott et al., 2009 ). Like the operant concurrent choice task, the T-maze task for measuring effort-based choice behavior also has undergone considerable behavioral validation and evaluation (Salamone, 1994 ; Cousins et al., 1996 ; van den Bos et al., 2006 ; Correa et al., 2009 ). For example, in a recent T-maze choice study with mice, it was confirmed that haloperidol reduced choice of the arm with the barrier, and it also was demonstrated that haloperidol had no effect on choice when both arms had a barrier in place (Correa et al., 2009 ). Thus, dopaminergic manipulations did not alter the preference for the high density of food reward over the low density, and did not affect discrimination or memory processes related to arm preference. Over the last several years, variants of this task have been used by several laboratories to characterize the effects of brain lesions or drug manipulations (Salamone, 1994 ; Walton et al., 2003 ; Denk et al., 2005 ; Schweimer and Hauber, 2005 ; van den Bos et al., 2006 ; Bardgett et al., 2009 ; Hauber and Sommer, 2009 ; Mott et al., 2009 ). The results of the T-maze studies in rodents, together with the findings from the operant concurrent choice studies reviewed above, indicate that low doses of DA antagonists and accumbens DA depletions cause animals to reallocate their instrumental response selection based upon the response requirements of the task, and select lower effort alternatives for obtaining rewards (see reviews by Salamone et al., 2003 , 2005 , 2007 ; Floresco et al., 2008a ).
Recent papers have used effort discounting procedures to study the effects of dopaminergic manipulations. Floresco et al. (2008b) investigated the effects of dopaminergic and glutamatergic drugs on both effort and delay discounting. The DA antagonist haloperidol altered effort discounting even when the effects of time delay were controlled for (Floresco et al., 2008b ). A T-maze effort discounting task was recently developed (Bardgett et al., 2009 ), in which the amount of food in the high density arm of the maze was diminished each trial on which the rats selected that arm (i.e., an “adjusting-amount” discounting variant of the T-maze procedures, which allows for the determination an indifference point for each rat). Administration of both the D1 family antagonist SCH23390 and the D2 family antagonist haloperidol altered effort discounting, making it more likely that rats would choose the arm with the smaller reward. Increasing DA transmission by administration of amphetamine blocked the effects of SCH23390 and haloperidol, and also biased rats toward choosing the high reward/high cost arm, which is consistent with operant choice studies using DA transporter knockdown mice (Cagniard et al., 2006 ). Together with other results, the findings reported by Bardgett et al. (2009) and Floresco et al. (2008b) support the suggestion that, across a variety of conditions, DA transmission exerts a bidirectional influence over effort-related decision making.

Interactions between DA and Adenosine

As reviewed above, considerable research has demonstrated that DA antagonists and accumbens DA depletions affect behavioral activation, instrumental response output, response allocation, and effort-related processes (Salamone et al., 1991 , 2007 ; Salamone and Correa, 2002 ; Phillips et al., 2007 ; Robbins and Everitt, 2007 ; Floresco et al., 2008a ). Clearly, DA does not participate in effort-related processes in isolation, and for that reason it is important to review how other brain areas and neurotransmitters interact with dopaminergic mechanisms. Within the last few years, considerable emphasis has been placed upon interactions between DA and adenosine. Caffeine and other methylxanthines, which act as minor stimulants, are non-selective adenosine antagonists (Ferré et al., 2008 ). Recently, there has been a rapid growth of research on adenosine receptor neurochemistry and pharmacology, particularly concerning the A2A subtype of adenosine receptor. DA-rich striatal areas, including both the caudate/putamen (neostriatum) and the nucleus accumbens, have a very high degree of adenosine A2A receptor expression (Schiffmann et al., 1991 ; DeMet and Chicz-DeMet, 2002 ; Ferré et al., 2004 ). There is considerable evidence of a functional interaction between striatal DA D2 and adenosine A2A receptors (Fink et al., 1992 ; Ferré, 1997 ; Hillion et al., 2002 ; Fuxe et al., 2003 ). This interaction frequently has been studied in regard to neostriatal motor functions that are related to parkinsonism (Ferré et al., 1997 , 2001 ; Hauber and Munkel, 1997 ; Svenningsson et al., 1999 ; Hauber et al., 2001 ; Wardas et al., 2001 ; Morelli and Pinna, 2002 ; Correa et al., 2004 ; Jenner, 2005 ; Pinna et al., 2005 ; Ishiwari et al., 2007 ; Salamone et al., 2008a ,b ). Several reports also have characterized aspects of adenosine A2A receptor function related to cognitive processes (Takahashi et al., 2008 ), anxiety (Correa and Font, 2008 ), and motivation (Salamone et al., 2007 ; Mingote et al., 2008 ).
Adenosine A2A receptors also are involved in aspects of behavioral activation and effort-related processes (Farrar et al., 2007 ; Font et al., 2008 ; Mingote et al., 2008 ; Mott et al., 2009 ; Worden et al., 2009 ). Injections of the adenosine A2A agonist CGS 21680 directly into the accumbens can produce effects that resemble those of accumbens DA depletions or antagonism. Intra-accumbens injections of CGS 21680 were shown to reduce locomotor activity (Barraco et al., 1993 ). Local infusions of CGS 21680 into the accumbens reduced responding on a VI 60 s schedule with a FR10 requirement attached, but did not impair performance on a conventional VI 60 s schedule (Mingote et al., 2008 ); this pattern is similar to that previously shown with accumbens DA depletions (Mingote et al., 2005 ). In rats responding on the operant FR5/chow feeding concurrent choice procedure, injections of CGS 21680 into the accumbens decreased lever pressing and increased chow intake (Font et al., 2008 ), a pattern of effects similar to that produced by accumbens DA depletions and antagonism. Consistent with the observation that an adenosine A2A agonist could produce actions similar to those resulting from DA depletion or blockade, it also has been reported the locomotor suppression induced by the DA antagonist haloperidol was reduced by injections of the adenosine A2A antagonist MSX-3 into nucleus accumbens core, but not into the shell or the ventrolateral neostriatum (Ishiwari et al., 2007 ). Furthermore, it has been demonstrated that adenosine A2A receptor antagonists can reverse the effects of DA D2 antagonists on both the operant concurrent choice task (Farrar et al., 2007 ; Salamone et al., 2009 ; Worden et al., 2009 ) and the T-maze choice procedure (Correa et al., 2009 ; Mott et al., 2009 ). Recently, studies with intracranial injections revealed that systemic or intra-accumbens injections of the adenosine A2A antagonist MSX-3 were able to block the effects of intra-accumbens injections of the D2 antagonist eticlopride in rats responding on the operant concurrent choice task (Farrar, 2009 , unpublished doctoral dissertation, University of Connecticut).
These studies afford an interesting opportunity to assess the overall interaction between DA and adenosine receptor subtypes. Adenosine A2A receptor antagonists MSX-3 and KW 6002 reliably attenuate the effects of D2 antagonists such as haloperidol and eticlopride in rats responding on the operant concurrent choice procedure (Farrar et al., 2007 ; Salamone et al., 2009 ; Worden et al., 2009 ). In contrast, MSX-3 was relatively ineffective at reducing the effects of the D1 antagonist ecopipam (SCH 39166; Worden et al., 2009 ) on this task. Although the non-selective adenosine antagonist caffeine was able to partially reverse the effects of haloperidol on the concurrent choice task, DPCPX, which is highly selective for the adenosine A1 receptor subtype, was ineffective (Salamone et al., 2009 ). Similar results were obtained with rats and mice responding on the T-maze barrier choice task. Although MSX-3 was able to reverse the effect of haloperidol on selection of the arm with the barrier, the A1 antagonists DPCPX and CPT were not (Correa et al., 2009 ; Mott et al., 2009 ).
The results described above demonstrate that there is a relatively selective interaction between DA D2 and adenosine A2A receptor subtypes (Table 1 ). Based upon anatomical studies, it appears that this is likely to be due to the pattern of cellular localization of adenosine A1 and A2A receptors in striatal areas, including the nucleus accumbens (Ferré, 1997 ). Adenosine A2A receptors are typically co-localized on striatal and accumbens enkephalin-positive medium spiny neurons with DA D2 family receptors, and these receptors converge onto the same signal transduction pathways and show the capacity for forming heteromeric complexes (Fink et al., 1992 ; Ferré, 1997 ; Svenningsson et al., 1999 ; Hillion et al., 2002 ; Fuxe et al., 2003 ). Thus, adenosine A2A receptor antagonists appear to be so effective in reversing the effort-related actions of D2 antagonists because of direct interactions between DA D2 and adenosine A2A receptors located on the same neurons (Figure 2 ). On the other hand, DA D1 receptors are more likely to be co-localized with adenosine A1 receptors (Ferré, 1997 ), which could help to explain why it is more difficult for adenosine A1 receptor antagonists to reverse the effects of D2 receptor blockade. Interestingly, despite the fact that D1 and A1 receptors tend to be co-localized on the same neurons, the A1 antagonists DPCPX and CPT were unable to reverse the effects of the D1 antagonist ecopipam in rats responding on the concurrent choice operant procedure (Nunes et al., 2009 ). This suggests that A2A antagonists exert an overall greater effect than A1 antagonists on effort-related functions of nucleus accumbens.
yes
Figure 2. Anatomical diagram depicting the pattern of DA and adenosine receptor localization in nucleus accumbens. See text for details (see also Ferré, 1997 ; Hillion et al., 2002 ; Fuxe et al., 2003 ). mGP, medial globus pallidus; epn, entopeduncular nucleus; s. nigra, substantia nigra; VTA, ventral tegmental area.

Behavioral Theory and Analyses: Further Evaluation of Effort-Related Processes

Research on the brain mechanisms involved in effort-related processes may lead to new ways of thinking about behavioral analysis and theory in behavioral economics. One of the contributions that behavioral neuroscience can make to behavioral theory is to use manipulations (e.g., drugs, lesions) that dissociate complex behavioral processes into component parts (Salamone et al., 2007 ). In this regard, it is useful to consider that a given parameter that is generated from curve-fitting analyses, when viewed in terms of its biological characteristics, has many factors that contribute to it. A good example of this is the ED50, which is used in pharmacology to provide a measure of the potency of a drug based upon dose-response analysis. Empirically, the ED50 is the dose that produces an effect that is 50% of the maximal effect. Although the ED50 is expressed as one number, that simplicity is deceptive because many biochemical factors contribute to it, including the affinity of a drug for a receptor, duration of action, drug metabolism, and penetration into the target tissue. A useful example of this principle from the behavioral neuroscience literature is the progressive ratio break point; as discussed above, this measure also has many factors that can contribute to it. Another case in which this point is important to consider is threshold measures used in intracranial self-stimulation studies. Such measures often are viewed as providing “rate-free” indices of reinforcement value, nevertheless, they are influenced by lever pressing ratio requirements as well as the electrical current level (Fouriezos et al., 1990 ).
Some research related to behavioral economics, reinforcer value, and the functions of DA systems has used response-reinforcement matching methods (e.g., Heyman and Monaghan, 1987 ; Aparicio, 2007 ). Matching equations have been employed to describe the results of studies with both conventional and concurrent VI schedules, and one of the parameters (Re) can be used to represent reinforcement value (e.g., Herrnstein, 1974 ; see equation below for single-lever conventional VI schedules, in which B represents response rate, R represents reinforcement density, k is the constant for maximal responding, and Re represents the reinforcement level that generates 50% of maximum responding).
B = k R/(R + Re)
However, used in this way, Re does not selectively represent only the reinforcement value of food per se; actually, it reflects the relative value of lever pressing for and consuming the food reinforcer compared to the reinforcing value of all other stimuli and responses available (Salamone et al., 1997 ). Several factors can contribute to this composite measure, which is one of the reasons why other matching equations have been developed that account for deviations from matching by allowing for estimates of reinforcer sensitivity, as well as response preference or bias (Baum, 1974 ; Williams, 1988 ; Aparicio, 2001 ). Clearly, a drug or lesion manipulation could yield apparent effects on “reinforcement value” that actually reflect changes in response bias (Salamone, 1987 ; Salamone et al., 1997 ).
For these reasons, it may be useful to think more deeply about how terms such as value are used in neuroeconomics research. The aggregate reinforcement value of an instrumental activity (e.g., lever pressing for and consuming food) should perhaps be viewed as a composite measure that includes both the reinforcing value of the reinforcer itself, plus any net value or costs associated with the instrumental response that is required to obtain the reinforcer. Viewed in this way, the effects of dopaminergic manipulations on effort-related choice behavior could be described in terms of actions upon the response costs associated with the particular reinforcer, rather than the reinforcing value of the food stimulus itself. Although the effects of haloperidol on bias may be minimal when two levers that are relatively similar are used (e.g., Aparicio, 2007 ), they may be much larger when very different responses are compared (e.g., lever pressing vs. sniffing; lever pressing vs. unrestricted access to food; barrier climbing vs. locomotion). Future research will determine if measures of bias based upon the matching equations, or some other type of mathematical formulation, would be the best way to capture these drug effects quantitatively.

Summary and Conclusions

In summary, DA and adenosine in the nucleus accumbens interact to regulate effort-related functions. Additional research has shown that a number of components of the cortico-striato-pallidal loop system also are involved (Walton et al., 2006 ; Floresco and Ghods-Sharifi, 2007 ; Farrar et al., 2008 ; Mingote et al., 2008 ; Hauber and Sommer, 2009 ). Disconnection studies have revealed that serial connections between basolateral amygdala, anterior cingulate cortex, nucleus accumbens, and ventral pallidum are involved in the exertion of effort and effort-related choice behavior (Floresco and Ghods-Sharifi, 2007 ; Farrar et al., 2008 ; Mingote et al., 2008 ; Hauber and Sommer, 2009 ). Within the last few years, there has been considerable progress in characterizing the functional anatomy underlying this important aspect of motivation and decision making. Several transmitters across multiple brain regions are involved in effort-related functions, and researchers are only beginning to piece together the complex puzzle of all the potential brain systems that are involved. Presently, the specific way in which each structure contributes to the overall function of the system is unclear. It is uncertain which brain areas are involved in the exertion of effort, or the perception of effort, vs. the actual decision making process itself. For example, it is possible that nucleus accumbens is involved in the actual decision making processes, but it also is possible that it is mainly involved in regulating energy output, or setting effort-related constraints or feedback that in turn influences decisions made at other levels in the system. If the latter is true, then it is possible that the decision making effects of drug or lesion manipulations of nucleus accumbens are an outcome reflecting the constraints that are set after compromised DA function in accumbens, rather than a direct effect upon decision making processes per se. Future research will be necessary to tease apart these distinct aspects of effort-related function.
In addition to providing insights into aspects of animal behavior and natural motivation, research on effort-related processes also has clinical implications. Within the last few years, there has been a greater emphasis upon effort-related functions involved in drug self-administration (e.g., Vezina et al., 2002 ). Drug seeking behavior in humans involves many psychological processes, including effort. Addicts will go to great lengths to obtain their preferred drug, overcoming numerous obstacles and constraints, both behavioral and economic. Furthermore, addiction is characterized not only by a re-organization of the preference structure of the person, but also by a dramatic change in the allocation of behavioral resources toward the addictive substance; there is a heightened emphasis upon drug seeking and drug taking, typically at the expense of other motivational activities. As well as being related to aspects of drug taking and addiction, research on behavioral activation and effort has implications for understanding the neural basis of psychiatric symptoms such as psychomotor slowing, anergia, fatigue and apathy, which are seen in depression as well as other psychiatric or neurological conditions (Salamone et al., 2006 , 2007 ). These motivational symptoms, which can have devastating behavioral manifestations (Stahl, 2002 ; Demyttenaere et al., 2005 ), represent impairments in aspects of behavioral activation and effort that can lead to problems in the workplace, as well as limitations in terms of life function, interaction with the environment, and responsiveness to treatment. There is considerable overlap between the neural circuitry involved in effort-related functions in animals and the brain systems that have been implicated in psychomotor slowing and anergia in depression (Salamone et al., 2006 ). Thus, research on effort-related behavioral processes, and their neural regulation, could have substantial impact on clinical research related to addiction, depression, and other disorders.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Much of the work cited in this review was supported by a grant to John D. Salamone from the US NIH/NIMH (MH078023), and to Merce Correa from C. Sanitat G.V. AP04108.

References

Aberman, J. E., and Salamone, J. D. (1999). Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement. Neuroscience 92, 545–552.
Allison, J. (1981). Economics and operant conditioning. In Predictability, Correlation and Contiguity, P. Harzem and M. D. Zeiler, eds (New York, John Wiley and Sons), pp 321–353.
Allison, J. (1993). Response deprivation, reinforcement, and economics. J. Exp. Anal. Behav. 60, 129–140.
Anstrom, K. K., and Woodward, D. J. (2005). Restraint increases dopaminergic burst firing in awake rats. Neuropsychopharmacology 30, 1832–1840.
Aparicio, C. F. (2001). Overmatching in rats: the barrier choice paradigm. J. Exp. Anal. Behav. 75:93–106.
Aparicio, C. F. (2007). Haloperidol, dynamics of choice, and the parameters of the matching law. Behav. Processes 75, 206–212.
Bakshi, V. P., and Kelley, A. E. (1991). Dopaminergic regulation of feeding behavior: I. Differential effects of haloperidol microinjection in three striatal subregions. Psychobiology 19, 223–232.
Baldo, B. A., and Kelley, A. E. (2007). Discrete neurochemical coding of distinguishable motivational processes: insights from nucleus accumbens control of feeding. Psychopharmacology (Berl.) 191, 439–459.
Baldo, B. A., Sadeghian, K., Basso, A. M., and Kelley, A. E. (2002). Effects of selective dopamine D1 or D2 receptor blockade within nucleus accumbens subregions on ingestive behavior and associated motor activity. Behav. Brain Res. 137, 165–177.
Barbano, M. F., and Cador, M. (2007). Opioids for hedonic experience and dopamine to get ready for it. Psychopharmacology 191, 497–506.
Bardgett, M. E., Depenbrock, M., Downs, N., Points, M., and Green, L. (2009). Dopamine modulates effort-based decision making in rats. Behav. Neurosci. 123, 242–251.
Barraco, R. A., Martens, K. A., Parizon, M., and Normile, H. J. (1993). Adenosine A2a receptors in the nucleus accumbens mediate locomotor depression. Brain Res. Bull. 31, 397–404.
Baum, W. M. (1974). On two types of deviation from the matching law: bias and undermatching. J. Exp. Anal. Behav. 22, 231–242.
Berridge, K. C. (2007). The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology 191, 391–431.
Berridge, K. C., and Kringlebach, M. L. (2008). Affective neuroscience of pleasure: reward in humans and animals. Psychopharmacology 199, 457–480.
Bickel, W. K., Marsch, L. A., and Carroll, M. E. (2000). Deconstructing relative reinforcing efficacy and situating the measures of pharmacological reinforcement with behavioral economics: a theoretical proposal. Psychopharmacology 153, 44–56.
Blazquez, P. M., Fujii, N., Kojima, J., and Graybiel, A. M. (2002). A network representation of response probability in the striatum. Neuron 33, 973–982.
Brauer, L. H., and De Wit, H. (1997). High dose pimozide does not block amphetamine-induced euphoria in normal volunteers. Pharmacol. Biochem. Behav. 56, 265–272.
Brischoux, F., Chakraborty, S., Brierley, D. I., and Ungless, M. A. (2009). Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc. Natl. Acad. Sci. USA 106, 4894–4899.
Broom, S. L., and Yamamoto, B. K. (2005). Effects of subchronic methamphetamine exposure on basal dopamine and stress-induced dopamine release in the nucleus accumbens shell of rats. Psychopharmacology 181, 467–476.
Cagniard, B., Balsam, P. D., Brunner, D., and Zhuang, X. (2006). Mice with chronically elevated dopamine exhibit enhanced motivation, but not learning, for a food reward. Neuropsychopharmacology 31, 1362–1370.
Cannon, C. M., and Bseikri, M. R. (2004). Is dopamine required for natural reward? Physiol. Behav. 81, 741–748.
Caul, W. F., and Brindle, N. A. (2001). Schedule-dependent effects of haloperidol and amphetamine: multiple-schedule task shows within-subject effects. Pharmacol. Biochem. Behav. 68, 53–63.
Cofer, C. N., and Appley, M. H. (1964). Motivation: Theory and Research. New York, John Wiley and Sons.
Collier, G. H., and Jennings, W. (1969). Work as a determinant of instrumental performance. J. Comp. Physiol. Psychol. 68, 659–662.
Correa, M., Carlson, B. B., Wisniecki, A., and Salamone, J. D. (2002). Nucleus accumbens dopamine and work requirements on interval schedules. Behav. Brain Res. 137, 179–187.
Correa, M., and Font, L. (2008). Is there a major role for adenosine A2A receptors in anxiety? Front. Biosci. 13, 4058–4070.
Correa, M., Pardo, M., Hockemeyer, J., Müller, C. E., and Salamone, J. D. (2009). Dopamine D2 Antagonism Increases Selection of Less Effortful Food-seeking Behavior in a T-maze Procedure in Mice: Reversal of the Motivational Impairment with Adenosine Antagonists. Neuroscience Meeting Planner. Washington, DC, Society for Neuroscience (online).
Correa, M., Wisniecki, A., Betz, A., Dobson, D. R., O’Neill, M. F., O’Neill, M. J., and Salamone, J. D. (2004). The adenosine A2A antagonist KF17837 reverses the locomotor suppression and tremulous jaw movements induced by haloperidol in rats: possible relevance to parkinsonism. Behav. Brain Res. 148, 47–54.
Cousins, M. S., Atherton, A., Turner, L., and Salamone, J. D. (1996). Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task. Behav. Brain Res. 74, 189–197.
Cousins, M. S., and Salamone, J. D. (1994). Nucleus accumbens dopamine depletions in rats affect relative response allocation in a novel cost/benefit procedure. Pharmacol. Biochem. Behav. 49, 85–91.
Cousins, M. S., Sokolowski, J. D., and Salamone, J. D. (1993). Different effects of nucleus accumbens and ventrolateral striatal dopamine depletions on instrumental response selection in the rat. Pharmacol. Biochem. Behav. 46, 943–951.
Cousins, M. S., Wei, W., and Salamone, J. D. (1994). Pharmacological characterization of performance on a concurrent lever pressing/feeding choice procedure: effects of dopamine antagonist, cholinomimetic, sedative and stimulant drugs. Psychopharmacology 116, 529–537.
Delgado, M. R., Li, J., Schiller, D., and Phelps, E. A. (2008). The role of the striatum in aversive learning and aversive prediction errors. Philos. Trans. R. Soc. Lond., B, Biol. Sci. 363, 3787–3800.
DeMet, E. M., and Chicz-DeMet, A. (2002). Localization of adenosine A2A-receptors in rat brain with [3H]ZM-241385. Naunyn Schmiedebergs Arch. Pharmacol. 366, 478–481.
Demyttenaere, K., De Fruyt, J., and Stahl, S. M. (2005). The many faces of fatigue in major depressive disorder. Int. J. Neuropsychopharmacol. 8, 93–105.
Denk, F., Walton, M. E., Jennings, K. A., Sharp, T., Rushworth, M. F., and Bannerman, D. M. (2005). Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology 179, 587–596.
Dunnett, S. B., and Iversen, S. D. (1982). Regulatory impairments following selective 6-OHDA lesions of the neostriatum. Behav. Brain Res. 4, 195–202.
Ettenberg, A., Koob, G. F., and Bloom, F. E. (1981). Response artifact in the measurement of neuroleptic-induced anhedonia. Science 213, 357–359.
Everitt, B. J., and Robbins, T. W. (2005). Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 8, 1481–1489.
Farrar, A. M., Font, L., Pereira, M., Mingote, S. M., Bunce, J. G., Chrobak, J. J., and Salamone, J. D. (2008). Forebrain circuitry involved in effort-related choice: injections of the GABAA agonist muscimol into ventral pallidum alters response allocation in food-seeking behavior. Neuroscience 152, 321–330.
Farrar, A. M., Pereira, M., Velasco, F., Hockemeyer, J., Muller, C. E., and Salamone, J. D. (2007). Adenosine A(2A) receptor antagonism reverses the effects of dopamine receptor antagonism on instrumental output and effort-related choice in the rat: implications for studies of psychomotor slowing. Psychopharmacology 191, 579–586.
Faure, A., Reynolds, S. M., Richard, J. M., and Berridge, K. C. (2008). Mesolimbic dopamine in desire and dread: enabling motivation to be generated by localized glutamate disruptions in nucleus accumbens. J. Neurosci. 28, 7184–7192.
Ferré, S. (1997). Adenosine–dopamine interactions in the ventral striatum. Implications for the treatment of schizophrenia. Psychopharmacology, 133, 107–120.
Ferré, S., Ciruela, F., Borycz, J., Solinas, M., Quarta, D., Antoniou, K., Quiroz, C., Justinova, Z., Lluis, C., Franco, R., and Goldberg, S. R. (2008). Adenosine A1-A2A receptor heteromers: new targets for caffeine in the brain. Front. Biosci. 13, 2391–2399.
Ferré, S., Ciruela, F., Canals, M., Marcellino, D., Burgueno, J., Casado, V., Hillion, J., Torvinen, M., Fanelli, F., Benedetti, P. P., Goldberg, S. R., Bouvier, M., Fuxe, K., Agnati, L. F., Lluis, C., Franco, R., and Woods. A. (2004). Adenosine A2A-dopamine D2 receptor–receptor heteromers. Targets for neuro-psychiatric disorders. Parkinsonism Relat. Disord. 10, 265–271.
Ferré, S., Fredholm, B. B., Morelli, M., Popoli, P., and Fuxe, K. (1997). Adenosine–dopamine receptor–receptor interactions as an integrative mechanism in the basal ganglia. Trends Neurosci. 20, 482–487.
Ferré, S., Popoli, P., Giménez-Llort, L., Rimondini, R., Müller, C. E., Strömberg, I., ögren, S. O., and Fuxe, K. (2001). Adenosine/dopamine interaction: implications for the treatment of Parkinson’s disease. Parkinsonism Relat. Disord. 7, 235–241.
Fink, J. S., Weaver, D. R., Rivkees, S. A., Peterfreund, R. A., Pollack, A. E., Adler, E. M., and Reppert, S. M. (1992). Molecular cloning of the rat A2A adenosine receptor: selective co-expression with D2 dopamine receptors in rat striatum. Brain Res. Mol. Brain Res. 14, 186–195.
Floresco, S. B., and Ghods-Sharifi, S. (2007). Amygdala-prefrontal cortical circuitry regulates effort-based decision making. Cereb. Cortex 17, 251–260.
Floresco, S. B., St Onge, J. R., Ghods-Sharifi, S., and Winstanley, C. A. (2008a). Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cogn. Affect. Behav. Neurosci. 8, 375–389.
Floresco, S. B., Tse, M. T., and Ghods-Sharifi, S. (2008b). Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology 33, 1966–1979.
Foltin, R. W. (1991). An economic analysis of “demand” for food in baboons. J. Exp. Anal. Behav. 56, 445–454.
Font, L., Mingote, S., Farrar, A. M., Pereira, M., Worden, L., Stopper, C., Port, R. G., and Salamone, J. D. (2008). Intra-accumbens injections of the adenosine A(2A) agonist CGS 21680 affect effort-related choice behavior in rats. Psychopharmacology 199, 515–526.
Fouriezos, G., Bielajew, C., and Pagotto, W. (1990). Task difficulty increases thresholds of rewarding brain stimulation. Behav. Brain Res. 37, 1–7.
Fuxe, K., Agnati, L. F., Jacobsen, K., Hillion, J., Canals, M., Torvinen, M., Tinner-Staines, B., Staines, W., Rosin, D., Terasmaa, A., Popoli, P., Leo, G., Vergoni, V., Lluis, C., Ciruela, F., Franco, R., and Ferré, S. (2003). Receptor heteromerization in adenosine A2A receptor signaling: relevance for striatal function and Parkinson’s disease. Neurology 61, S19–S23.
Gawin, F. H. (1986). Neuroleptic reduction of cocaine-induced paranoia but not euphoria? Psychopharmacology 90, 142–143.
Guarraci, F. A., and Kapp, B. S. (1999). An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential pavlovian fear conditioning in the awake rabbit. Behav. Brain Res. 99, 169–179.
Hamill, S., Trevitt, J. T., Nowend, K. L., Carlson, B. B., and Salamone, J. D. (1999). Nucleus accumbens dopamine depletions and time-constrained progressive ratio performance: effects of different ratio requirements. Pharmacol. Biochem. Behav. 64, 21–27.
Haney, M., Ward, A. S., Foltin, R. W., and Fischman, M. W. (2001). Effects of ecopipam, a selective dopamine D1 antagonist, on smoked cocaine self-administration by humans. Psychopharmacology 155, 330–337.
Hauber, W., and Munkel, M. (1997). Motor depressant effects mediated by dopamine D2 and adenosine A2A receptors in the nucleus accumbens and the caudate-putamen. Eur. J. Pharmacol. 323, 127–131.
Hauber, W., Neuscheler, P., Nagel, J., and Muller, C. E. (2001). Catalepsy induced by a blockade of dopamine D1 or D2 receptors was reversed by a concomitant blockade of adenosine A2A receptors in the caudate putamen of rats. Eur. J. Neurosci. 14, 1287–1293.
Hauber, W., and Sommer, S. (2009). Prefrontostriatal circuitry regulates effort-related decision making. Cereb. Cortex [Epub ahead of print]. doi: 10.1093/cercor/bhn241.
Hengeveld, G. M., van Langevelde, F., Groen, T. A., and de Knegt, H. J. (2009). Optimal foraging for multiple resources in several food species. Am. Nat. 17, 102–110.
Herrnstein, R. J. (1974). Formal properties of the matching law. J. Exp. Anal. Behav. 21, 159–164.
Heyman, G. M., and Monaghan, M. M. (1987). The effect of changes in the response requirement and deprivation on the parameters of the matching law equation: new data and review. J. Exp. Psych. An. Behav. Proc. 13, 384–394.
Hillion, J., Canals, M., Torvinen, M., Casado, V., Scott, R., Terasmaa, A., Hansson, A., Watson, S., Olah, M. E., Mallol, J., Canela, E. I., Zoli, M., Agnati, L. F., Ibañez, C. F., Lluis, C., Franco, R., Ferré, S., and Fuxe, K. (2002). Coaggregation, cointernalization, and codesensitization of adenosine A2A receptors and dopamine D2 receptors. J. Biol. Chem. 277, 18091–18097.
Hursh, S. R., Raslear, T. G., Shurtleff, D., Bauman, R., and Simmons, L. (1988). A cost-benefit analysis of demand for food. J. Exp. Anal. Behav. 50, 419–440.
Ishiwari, K., Madson, L. J., Farrar, A. M., Mingote, S. M., Valenta, J. P., DiGianvittorio, M. D., Frank, L. E., Correa, M., Hockemeyer, J., Muller, C., and Salamone, J. D. (2007). Injections of the selective adenosine A2A antagonist MSX-3 into the nucleus accumbens core attenuate the locomotor suppression induced by haloperidol in rats. Behav. Brain Res. 178, 190–199.
Ishiwari, K., Weber, S. M., Mingote, S., Correa, M., and Salamone, J. D. (2004). Accumbens dopamine and the regulation of effort in food-seeking behavior: modulation of work output by different ratio or force requirements. Behav. Brain Res. 151, 83–91.
Jenner, P. (2005). Istradefylline, a novel adenosine A2A receptor antagonist, for the treatment of Parkinson’s disease. Expert Opin. Investig. Drugs 14, 729–738.
Jensen, J., McIntosh, A. R., Crawley, A. P., Mikulis, D. J., Remington, G., and Kapur, S. (2003). Direct activation of the ventral striatum in anticipation of aversive stimuli. Neuron 40, 1251–1257.
Johnson, D. F., and Collier, G. H. (1987). Caloric regulation and patterns of food choice in a patchy environment: the value and cost of alternative foods. Physiol. Behav. 39, 351–359.
Kaufman, L. W. (1980). Foraging costs and meal patterns in ferrets. Physiol. Behav. 25, 139–141.
Kaufman, L. W., Collier, G., Hill, W. L., and Collins, K. (1980). Meal cost and meal patterns in an uncaged domestic cat. Physiol. Behav. 25, 135–137.
Kelley, A. E., Baldo, B. A., Pratt, W. E., and Will, M. J. (2005). Corticostriatal-hypothalamic circuitry and food motivation: integration of energy, action and reward. Physiol. Behav. 86, 773–795.
Koch, M., Schmid, A., and Schnitzler, H. U. (2000). Role of nucleus accumbens dopamine D1 and D2 receptors in instrumental and Pavlovian paradigms of conditioned reward. Psychopharmacology 152, 67–73.
Koob, G. F., Riley, S. J., Smith, S. C., and Robbins, T. W. (1978). Effects of 6-hydroxydopamine lesions of the nucleus accumbens septi and olfactory tubercle on feeding, locomotor activity, and amphetamine anorexia in the rat. J. Comp. Physiol. Psychol. 92, 917–927.
Krebs, J. R. (1977). Optimal foraging: theory and experiment. Nature 268, 583–584.
Lapish, C. C., Kroener, S., Durstewitz, D., Lavin, A., Seamans, J. K. (2007). The ability of the mesocortical dopamine system to operate in distinct temporal modes. Psychopharmacology 191, 609–625.
Lea, S. E. G. (1978). The psychology and economics of demand. Psychol. Bull. 85, 441–466.
Levita, L., Hare, T. A., Voss, H. U., Glover, G., Ballon, D. J., and Casey, B. J. (2009). The bivalent side of the nucleus accumbens. Neuroimage 44, 1178–1187.
Liberzon, I., Taylor, S. F., Amdur, R., Jung, T. D., Chamberlain, K. R., Minoshima, S., Koeppe, R. A., and Fig, L. M. (1999). Brain activation in PTSD in response to trauma-related stimuli. Biol. Psychiatry 45, 817–826.
Marinelli, M., Barrot, M., Simon, H., Oberlander, C., Dekeyne, A., Le Moal, M., and Piazza, P. V. (1998). Pharmacological stimuli decreasing nucleus accumbens dopamine can act as positive reinforcers but have a low addictive potential. Eur. J. Neurosci. 10, 3269–3275.
Marinelli, S., Pascucci, T., Bernardi, G., Puglisi-Allegra, S., and Mercuri, N. B. (2005). Activation of TRPV1 in the VTA excites dopaminergic neurons and increases chemical- and noxious-induced dopamine release in the nucleus accumbens. Neuropsychopharmacology 30, 864–875.
Martinez, R. C. R., Oliveira, A. R., Macedo, C. E., Molina, V. A., and Brandao, M. L. (2008). Involvement of dopaminergic mechanisms in the nucleus accumbens core and shell subregions in the expression of fear conditioning. Neurosci. Lett. 446, 112–116.
McCullough, L. D., and Salamone, J. D. (1992) Anxiogenic drugs beta-CCE and FG 7142 increase extracellular dopamine levels in nucleus accumbens. Psychopharmacology 109, 379–382.
McCullough, L. D., Sokolowski, J. D., and Salamone, J. D. (1993). A neurochemical and behavioral investigation of the involvement of nucleus accumbens dopamine in instrumental avoidance. Neuroscience 52, 919–925.
Mekarski, J. E. (1988). Main effects of current and pimozide on prepared and learned self-stimulation behaviors are on performance not reward. Pharmacol. Biochem. Behav. 31, 845–853.
Mingote, S., Font, L., Farrar, A. M., Vontell, R., Worden, L. T., Stopper, C. M., Port, R. G., Sink, K. S., Bunce, J. G., Chrobak, J. J., and Salamone, J. D. (2008). Nucleus accumbens adenosine A2A receptors regulate exertion of effort by acting on the ventral striatopallidal pathway. J. Neurosci. 28, 9037–9046.
Mingote, S., Weber, S. M., Ishiwari, K., Correa, M., and Salamone, J. D. (2005). Ratio and time requirements on operant schedules: effort-related effects of nucleus accumbens dopamine depletions. Eur. J. Neurosci 21, 1749–1757.
Mogenson, G. J., Jones, D. L., and Yim, C. Y. (1980). From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 14, 69–97.
Morelli, M., and Pinna, A. (2002). Interaction between dopamine and adenosine A2A receptors as a basis for the treatment of Parkinson’s disease. Neurol. Sci. 22, 71–72.
Mott, A. M, Nunes, E. J., Collins, L. E., Port, R. G., Sink, K. S., Hockemeyer, J., Müller, C. E., and Salamone, J. D. (2009). The adenosine A2A antagonist MSX-3 reverses the effects of the dopamine antagonist haloperidol on effort-related decision making in a T-maze cost/benefit procedure. Psychopharmacology 204, 103–112.
Munro, L. J., and Kokkinidis, L. (1997). Infusion of quinpirole and muscimol into the ventral tegmental area inhibits fear-potentiated startle: implications for the role of dopamine in fear expression. Brain Res. 746, 231–238.
Nann-Vernotica, E., Donny, E. C., Bigelow, G. E., and Walsh, S. L. (2001). Repeated administration of the D1/5 antagonist ecopipam fails to attenuate the subjective effects of cocaine. Psychopharmacology 155, 338–347.
Neill, D. B., and Justice, J. B. (1981). An hypothesis for a behavioral function of dopaminergic transmission in nucleus accumbens. In The Neurobiology of Nucleus Accumbens, R. B. Chronister and J. F. Defrance, eds (Brunswick, Huer Institute).
Nicola, S. M. (2007). The nucleus accumbens as part of a basal ganglia action selection circuit. Psychopharmacology 191, 521–550.
Niv, Y., Daw, N. D., Joel, D., and Dayan, P. (2007). Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology 191, 507–520.
Nowend, K. L., Arizzi, M., Carlson, B. B., and Salamone, J. D. (2001). D1 or D2 antagonism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consumption. Pharmacol. Biochem. Behav. 69, 373–382.
Nunes, E. J., Farrar, A. M., Font, L., Patel, V., Schlar, D. E., Collins, L. E., Sager, T. N., Correa, M., and Salamone, J. D. (2009). Differential Actions of Adenosine A1 and A2A Antagonists on the Effort-related Effects of Dopamine Antagonists. Neuroscience Meeting Planner. Washington, DC, Society for Neuroscience (online).
Palmiter, R. D. (2007). Is dopamine a physiologically relevant mediator of feeding behavior? Trends Neurosci. 30, 375–381.
Pavic, L. (2003). Alterations in brain activation in posttraumatic stress disorder patients with severe hyperarousal symptoms and impulsive aggressiveness. Eur. Arch. Psychiatry Clin. Neurosci. 253, 80–83.
Pezze, M. A., and Feldon, J. (2004). Mesolimbic dopaminergic pathways in fear conditioning. Prog. Neurobiol. 74, 301–320.
Phan, K. L., Taylor, S. F., Welsh, R. C., Ho, S. H., Britton, J. C., and Liberzon, I. (2004). Neural correlates of individual ratings of emotional salience: a trial-related fMRI study. Neuroimage 21, 768–780.
Phillips, P. E., Walton, M. E., and Jhou, T. C. (2007). Calculating utility: preclinical evidence for cost-benefit analysis by mesolimbic dopamine. Psychopharmacology 191, 483–495.
Pinna, A., Wardas, J., Simola, N., and Morelli, M. (2005). New therapies for the treatment of Parkinson’s disease: adenosine A2A receptor antagonists. Life Sci. 77, 3259–3267.
Pruessner, J. C., Champagne, F., Meaney, M. J., and Dagher, A. (2004). Dopamine release in response to a psychological stress in humans and its relationship to early life maternal care: a positron emission tomography study using [11C]raclopride. J. Neurosci. 24, 2825–2831.
Rachlin, H. (2003). Economic concepts in the behavioral study of addiction. In Choice, Behavioral Economics and Addiction, R. E. Vuchinich and N. Heather, eds (Oxford, Elsevier), pp. 129–149.
Redgrave, P., Gurney, K., and Reynolds, J. (2008). What is reinforced by phasic dopamine signals? Brain Res. Rev. 58, 322–339.
Richardson, N. R., and Roberts, D. C. (1996). Progressive ratio schedules in drug self-administration studies in rats: a method to evaluate reinforcing efficacy. J. Neurosci. Methods 66, 1–11.
Rick, J. H., Horvitz, J. C., and Balsam, P. D. (2006). Dopamine receptor blockade and extinction differentially affect behavioral variability. Behav. Neurosci. 120, 488–492.
Robbins, T. W., and Everitt, B. J. (2007). A role for mesencephalic dopamine in activation: commentary on Berridge (2006). Psychopharmacology 191, 433–437.
Robbins, T. W., and Koob, G. F. (1980). Selective disruption of displacement behaviour by lesions of the mesolimbic dopamine system. Nature 285, 409–412.
Roitman, M. F., Stuber, G. D., Phillips, P. E., Wightman, R. M., and Carelli, R. M. (2004). Dopamine operates as a subsecond modulator of food seeking. J. Neurosci. 24, 1265–1271.
Salamone, J. D. (1986). Different effects of haloperidol and extinction on instrumental behaviours. Psychopharmacology 88, 18–23.
Salamone, J. D. (1987). The actions of neuroleptic drugs on appetitive instrumental behaviors. In Handbook of Psychopharmacology, L. L. Iversen, S. D. Iversen and S. H. Snyder, eds (New York, Plenum Press), pp. 575–608.
Salamone, J. D. (1988). Dopaminergic involvement in activational aspects of motivation: effects of haloperidol on schedule induced activity, feeding and foraging in rats. Psychobiology 16, 196–206.
Salamone, J. D. (1994). The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav. Brain Res. 61, 117–133.
Salamone, J. D. (1996). The behavioral neurochemistry of motivation: methodological and conceptual issues in studies of the dynamic activity of nucleus accumbens dopamine. J. Neurosci. Methods 64, 137–149.
Salamone, J. D. (in press). Involvement of nucleus accumbens dopamine in behavioral activation and effort-related functions. In Dopamine Handbook, L. L. Iversen, S. D. Iversen, S. B., Dunnett and A. Bjorkland, eds (Oxford, Oxford University Press).
Salamone, J. D. (2006). Will the last person who uses the term ‘reward’ please turn out the lights? Comments on processes related to reinforcement, learning, motivation and effort. Addict. Biol.11, 43–44.
Salamone, J. D., Aberman, J. E., Sokolowski, J. D., and Cousins, M. S. (1999). Nucleus accumbens dopamine and rate of responding: neurochemical and behavioral studies. Psychobiology 27, 236–247.
Salamone, J. D., Arizzi, M., Sandoval, M. D., Cervone, K. M., and Aberman, J. E. (2002). Dopamine antagonists alter response allocation but do not suppress appetite for food in rats: contrast between the effects of SKF 83566, raclopride and fenfluramine on a concurrent choice task. Psychopharmacology 160, 371–380.
Salamone, J. D., Betz, A. J., Ishiwari, K., Felsted, J., Madson, L., Mirante, B., Clark, K., Font, L., Korbey, S., Sager, T. N., Hockemeyer, J., and Muller, C. E. (2008a). Tremorolytic effects of adenosine A2A antagonists: implications for parkinsonism. Front. Biosci. 13, 3594–3605.
Salamone, J. D., Ishiwari, K., Betz, A. J., Farrar, A. M., Mingote, S. M., Font, L., Hockemeyer, J., Müller, C. E., and Correa, M. (2008b). Dopamine/adenosine interactions related to locomotion and tremor in animal models: possible relevance to parkinsonism. Parkinsonism Relat. Disord. 14, S130–S134.
Salamone, J. D., and Correa, M. (2002). Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav. Brain Res. 137, 3–25.
Salamone, J. D., Correa, M., Farrar, A., and Mingote, S. M. (2007). Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology 191, 461–482.
Salamone, J. D., Correa, M., Mingote, S., and Weber, S. M. (2003). Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: implications for studies of natural motivation, psychiatry, and drug abuse. J. Pharmacol. Exp. Ther. 305, 1–8.
Salamone, J. D., Correa, M., Mingote, S. M., and Weber, S. M. (2005). Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr. Opin. Pharmacol. 5, 34–41.
Salamone, J. D., Correa, M., Mingote, S. M., Weber, S. M., and Farrar, A. M. (2006). Nucleus accumbens dopamine and the forebrain circuitry involved in behavioral activation and effort-related decision making: implications for understanding anergia and psychomotor slowing in depression. Curr. Psychiatry Rev. 2, 267–280.
Salamone, J. D., Cousins, M. S., and Bucher, S. (1994). Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behav. Brain Res. 65, 221–229.
Salamone, J. D., Cousins, M. S., Maio, C., Champion, M., Turski, T., and Kovach, J. (1996). Different behavioral effects of haloperidol, clozapine and thioridazine in a concurrent lever pressing and feeding procedure. Psychopharmacology 125, 105–112.
Salamone, J. D., Cousins, M. S., and Snyder, B. J. (1997). Behavioral functions of nucleus accumbens dopamine: empirical and conceptual problems with the anhedonia hypothesis. Neurosci. Biobehav. Rev. 21, 341–359.
Salamone, J. D., Farrar, A. M., Font, L., Patel, V., Schlar, D. E., Nunes, E. J., Collins, L. E., and Sager, T. N. (2009). Differential actions of adenosine A1 and A2A antagonists on the effort-related effects of dopamine D2 antagonism. Behav. Brain Res. 201, 216–222.
Salamone, J. D., Kurth, P., McCullough, L D., and Sokolowski, J. D. (1995). The effects of nucleus accumbens dopamine depletions on continuously reinforced operant responding: contrasts with the effects of extinction. Pharmacol. Biochem. Behav. 50, 437–443.
Salamone, J. D., Mahan, K., and Rogers, S. (1993a). Ventrolateral striatal dopamine depletions impair feeding and food handling in rats. Pharmacol. Biochem. Behav. 44, 605–610.
Salamone, J. D., Kurth, P. A., McCullough, L. D., Sokolowski, J. D., and Cousins, M. S. (1993b). The role of brain dopamine in response initiation: effects of haloperidol and regionally specific dopamine depletions on the local rate of instrumental responding. Brain Res. 628, 218–226.
Salamone, J. D., Steinpreis, R. E., McCullough, L. D., Smith, P., Grebel, D., and Mahan, K. (1991). Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure. Psychopharmacology 104, 515–521.
Salamone, J. D., Wisniecki, A., Carlson, B. B., and Correa, M. (2001). Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement. Neuroscience 105, 863–870.
Sanchis-Segura, C., and Spanagel, R. (2006). Behavioural assessment of drug reinforcement and addictive features in rodents: an overview. Addict. Biol. 11, 2–38.
Sarchiapone, M., Carli, V., Camardese, G., Cuomo, C., Di Guida, D., Calgagni, M. L., Focacci, C., and De Riso, S. (2006). Dopamine transporter binding in depressed patients with anhedonia. Psychiatry Res. Neuroimag. 147, 243–248.
Schiffmann, S. N., Jacobs, O., and Vanderhaeghen, J. J. (1991). Striatal restricted adenosine A2A receptor (RDC8) is expressed by enkephalin but not by substance P neurons: an in situ hybridization histochemistry study. J. Neurochem. 57, 1062–1071.
Schmelzeis, M. C., and Mittleman, G. (1996). The hippocampus and reward: effects of hippocampal lesions on progressive-ratio responding. Behav. Neurosci. 110, 1049–1066.
Schultz, W. (2007a). Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 30, 259–288.
Schultz, W. (2007b). Behavioral dopamine signals. Trends Neurosci. 30, 203–210.
Schweimer, J., and Hauber, W. (2005). Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy. Learn. Mem. 12, 334–342.
Sink, K. S., Vemuri, V. K., Olszewska, T., Makriyannis, A., and Salamone, J. D. (2008). Cannabinoid CB1 antagonists and dopamine antagonists produce different effects on a task involving response allocation and effort-related choice in food-seeking behavior. Psychopharmacology 196, 565–574.
Skjoldager, P., Pierre, P. J., and Mittlman, G. (1993). Reinforcer magnitude and progressive ratio responding: effects of increased effort, prefeeding and extinction. Learn. Motiv. 24, 303–343.
Sokolowski, J. D., and Salamone, J. D. (1998). The role of accumbens dopamine in lever pressing and response allocation: effects of 6-OHDA injected into core and dorsomedial shell. Pharmacol. Biochem. Behav. 59, 557–566.
Staddon, J. E. R. (1979). Operant behavior as adaptation to constraint. J. Exp. Psychol. Gen. 108, 48–67.
Stahl, S. M. (2002). The psychopharmacology of energy and fatigue. J. Clin. Psychiatry 63, 7–8.
Stewart, W. J. (1974). Progressive reinforcement schedules: a review and evaluation. Aust. J. Psychol. 27, 9–22.
Svenningsson, P., Le Moine, C., Fisone, G., and Fredholm, B. B. (1999). Distribution, biochemistry and function of striatal adenosine A2A receptors. Prog. Neurobiol. 59, 355–396.
Szczypka, M. S., Kwok, K., Brot, M. D., Marck, B. T., Matsumoto, A. M., Donahue, B. A., and Palmiter, R. D. (2001). Dopamine production in the caudate putamen restores feeding in dopamine-deficient mice. Neuron 30, 819–828.
Takahashi, R. N., Pamplona, F. A., and Prediger, R. D. (2008). Adenosine receptor antagonists for cognitive dysfunction: a review of animal studies. Front. Biosci. 13, 2614–2632.
Ungerstedt, U. (1971). Adipsia and aphagia after 6-hydroxydopamine induced degeneration of the nigro-striatal dopamine system. Acta Physiol. Scand. Suppl. 367, 95–122.
van den Bos, R., van der Harst, J., Jonkman, S., Schilders, M., and Spruijt, B. (2006). Rats assess costs and benefits according to an internal standard. Behav. Brain Res. 171, 350–354.
Vezina, P., Lorrain, D. S., Arnold, G. M., Austin, J. D., and Suto, N. (2002). Sensitization of midbrain dopamine neuron reactivity promotes the pursuit of amphetamine. J. Neurosci. 22, 4654–4662.
Vuchinich, R. E., and Heather, N. (2003). Introduction: Overview of behavioural economic perspectives on substance use and addiction. In Choice, Behavioral Economics and Addiction, R. E. Vuchinich and N. Heather, eds (Oxford, Elsevier), pp. 1–31.
Wachtel, S. R., Ortengren, A., and de Wit, H. (2002). The effects of acute haloperidol or risperidone on subjective responses to methamphetamine in healthy volunteers. Drug Alcohol Depend. 68, 23–33.
Wacker, J., Dillon, D. G., and Pizzagalli, D. A. (2009). The role of the nucleus accumbens and rostral anterior cingulate cortex in anhedonia: integration of resting EEG, fMRI, and volumetric techniques. Neuroimage 46, 327–337.
Wakabayashi, K. T., Fields, H. L., and Nicola, S. M. (2004). Dissociation of the role of nucleus accumbens dopamine in responding to reward-predictive cues and waiting for reward. Behav. Brain Res. 154, 19–30.
Wallace, M., Singer, G., Finlay, J., and Gibson, S. (1983). The effect of 6-OHDA lesions of the nucleus accumbens septum on schedule-induced drinking, wheelrunning and corticosterone levels in the rat. Pharmacol. Biochem. Behav. 18, 129–136.
Walton, M. E., Bannerman, D. M., Alterescu, K., and Rushworth, M. F. S. (2003). Functional specialization within medial frontal cortex of the anterior cingulated for evaluating effort-related decisions. J. Neurosci. 23, 6475–6479.
Walton, M. E., Kennerley, S. W., Bannerman, D. M., Phillips, P. E., and Rushworth, M. F. (2006). Weighing up the benefits of work: behavioral and neural analyses of effort-related decision making. Neural Netw. 19, 1302–1314.
Wardas, J., Konieczny, J., and Lorenc-Koci, E. (2001). SCH 58261, an A2A adenosine receptor antagonist, counteracts parkinsonian-like muscle rigidity in rats. Synapse 41, 160–171.
Williams, B. A. (1988). Reinforcement, choice, and response strength. In Stevens’ Handbook of Experimental Psychology Vol. 2, R. C. Atkinson, R. J. Herrnstein, G. Lindsey and R. D. Luce, eds (New York, John Wiley and Sons), pp. 167–174.
Wise, R. A. (2004). Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494.
Woolverton, W. L., and Ranaldi, R. (2002). Comparison of the reinforcing efficacy of two dopamine D2-like receptor agonists in rhesus monkeys using a progressive-ratio schedule of reinforcement. Pharmacol. Biochem. Behav. 72, 803–809.
Worden, L. T., Shahriari, M., Farrar, A. M., Sink, K. S., Hockemeyer. J., Müller, C., Salamone, JD. (2009). The adenosine A2A antagonist MSX-3 reverses the effort-related effects of dopamine blockade: differential interaction with D1 and D2 family antagonists. Psychopharmacology 203, 489–499.
Yin, H. H., Ostlund, S. B., and Balleine, B. W. (2008). Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. Eur. J. Neurosci. 28, 1437–1448.
Young, A. M. (2004). Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. J. Neurosci. Methods 138, 57–63.
Keywords:
nucleus accumbens, motivation, reward, reinforcement, activation, anergia, psychomotor slowing, depression
Citation:
Salamone JD, Correa M, Farrar AM, Nunes EJ and Pardo M (2009). Dopamine, behavioral economics, and effort. Front. Behav. Neurosci. 3:13. doi: 10.3389/neuro.08.013.2009
Received:
09 June 2009;
 Paper pending published:
17 June 2009;
Accepted:
21 July 2009;
 Published online:
07 September 2009.

Edited by:

Paul Phillips, University of Washington, USA

Reviewed by:

Daeyeol Lee, Yale University, USA
Kent Berridge, University of Michigan, USA
Copyright:
© 2009 Salamone, Correa, Farrar, Nunes and Pardo. This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence:
John D. Salamone, Board of Trustees Distinguished Professor, Division of Behavioral Neuroscience, Department of Psychology, University of Connecticut, Storrs, CT 06269-1020, USA. e-mail: john.salamone@uconn.edu

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.