Department of Psychology, Stony Brook University, Stony Brook, NY, USA
Decisions frequently have consequences that play out over time and these temporal factors can exert strong influences on behavior. For example, decision-makers exhibit delay discounting, behaving as though immediately consumable goods are more valuable than those available only after some delay. With the use of functional magnetic resonance imaging, we are now beginning to characterize the physiological bases of such behavior in humans and to link work on this topic from neuroscience, psychology, and economics. Here we review recent neurocognitive investigations of temporal decision-making and outline the theoretical picture that is beginning to take shape. Taken as a whole, this body of work illustrates the progress made in understanding temporal choice behavior. However, we also note several questions that remain unresolved and areas where future work is needed.
Decision-makers are frequently faced with choices that differ in the timing of their consequences. Such intertemporal choices require shrewd decision-makers to consider, not only what they want, but when they want it. For example, when asked to deliver a guest lecture, your response is likely to depend strongly on whether the lecture is to be delivered relatively soon or in the more distant future. More gravely, decisions about whether to refinance one’s mortgage (Harding, 2000
) and about whether governments should spend money to protect the environment (Dasgupta, 2008
; Hardisty and Weber, 2009
) can be characterized as intertemporal choices. Furthermore, abnormalities in intertemporal choice behavior have been associated with an array of undesirable behavior including drug addiction (Kirby and Petry, 2004
; Rossow, 2008
). Given the relevance of intertemporal choice, it is clear that we have much to gain by understanding how intertemporal choices are made, what factors influence intertemporal choices, and what is responsible for aberrations in intertemporal choice in some patient populations.
Like much of decision-making, intertemporal choice has long been the province of economics. Work from this field has provided both normative guidelines for intertemporal choice and the theoretical tools to evaluate observed behavior. Empirical support has come primarily from psychology and has, as it often does, focused on decision-makers’ deviations from the prescriptions of economics. With the recent interest in utilizing neurobiological techniques to understand decision making behavior (Glimcher, 2003
; Glimcher et al., 2008
), particularly functional magnetic resonance imaging (fMRI), we are in the position to observe the operation of the processes responsible for intertemporal decisions, processes that are extremely difficult to evaluate using behavior alone. Here, we review recent work on intertemporal choice with a focus on studies involving humans, the majority of which have utilized a combination of behavior, fMRI, and quantitative economic theory.
The majority of intertemporal choice studies have been designed to explore delay discounting, the robust finding that animals, including humans, behave as though immediately consumable goods are more valuable than those only available after some delay. This phenomenon is so powerful that decision-makers frequently forgo delayed rewards in favor of immediate rewards even when the delayed rewards are objectively more valuable. For example, a decision-maker might choose $100 delivered immediately over $200 to be delivered in 3 years. Such a choice is said reflect the subjective value of the $200 option, discounted according to the associated 3-year delay. The sway of negative events is similarly blunted by delay. The idea of working on your taxes next month seems less unpleasant than the prospect of working on them tonight.
Economics has viewed delay discounting from within the framework of discounted utility theory (Samuelson, 1937
) according to which the subjective value of goods drops by a fixed percentage (frequently referred to as the discount rate) for each unit of time that those goods are delayed. If a decision-maker discounts the future at a rate of 10% annually, then $100 available in a year is only worth $90 right now. That same reward offered in 5 years is only worth $59. If this drop in subjective value is plotted over time, the resulting discounting curve is exponential in shape.
Behavioral work on delay discounting has primarily focused on two major facets of the phenomenon. First, it appears that animals, including humans, do not discount exponentially. Given that such behavior is arguably non-normative, this possibility has generated a large body of behavioral data (Ainslie and Herrnstein, 1981
; Loewenstein and Thaler, 1989
; Ainslie, 1992
; Green et al., 1994a
; Kirby and Herrnstein, 1995
; Rachlin, 1995
; Kirby, 1997
) nearly all of which demonstrates that decision-makers behave as though their discount rate declines as rewards are pushed further into the future. Waiting 2 years for a reward might be worth 10% less than waiting 1 year, but waiting 4 years for a reward might be worth only 5% less than waiting 3 years. Such discounting is referred to as hyperbolic or quasi-hyperbolic and is blamed for a variety of unwanted behavior (Ainslie, 2001
) all stemming from the fact that hyperbolic discounters make one set of choices about rewards in the distant future only to reverse their preferences as those same rewards draw near.
Second, work has focused on the rate of discounting itself and has found that discounting rates vary across individuals and contexts, and are sometimes unreasonably extreme. For example, in two of the more well-cited studies (Hausman, 1979
; Gately, 1980
), discount rates were estimated based on the purchase price and operating costs of home appliances. The estimated rates were shown to be significantly greater than typically assumed by economists (anywhere from 25 to 300% per year which is obviously well above the rates at which consumers borrow and invest). Thus, there has been a significant effort to characterize the rate at which various populations discount rewards. For example, children (Green et al., 1994b
; Scheres et al., 2006
), including those with attention deficit hyperactivity disorder (ADHD, Barkley et al., 2001
), alcoholics (Vuchinich and Simpson, 1998
; Petry, 2001
), smokers (Bickel et al., 1999
; Reynolds et al., 2004
), cocaine and heroin addicts (Coffey et al., 2003
; Kirby and Petry, 2004
), and compulsive gamblers (Holt et al., 2003
) all discount at a faster rate than healthy adults; they exhibit a relative inability to wait for rewards. In contrast, older adults (Green et al., 1994b
) and those with a higher IQ (Shamosh and Gray, 2008
) have been shown to discount at a slower rate; they exhibit relative patience.
Utilizing classic intertemporal choice tasks, recent work in cognitive neuroscience has begun to address the neural mechanisms associated with delay discounting. One basic question that this field is uniquely suited to address is what distinguishes those occasions on which decision-makers choose to wait from those occasions on which they choose immediate rewards. That is, what leads to patient and impatient choices? Wittmann et al. (2007
) utilized fMRI and a standard intertemporal choice task (though with completely hypothetical rewards). Based on subjects’ choices, the magnitude of the immediate option was adjusted incrementally to find the point at which that particular decision-maker would be indifferent between the immediate and delayed options. Trials on which the delayed option was chosen (patient choices) were then compared to trials on which the immediate option was chosen (impatient choices). This contrast yielded a network of brain regions that included bilateral posterior insular cortex, left posterior cingulate, as well as temporal and parietal regions. Interestingly, no regions appeared to exhibit greater activity when choosing the immediate reward. This study also observed higher levels of activity in the striatum when subjects were asked about rewards to be delivered in the near future (<1 year) than when they were asked about delayed rewards in the distant future (≥1 year).
These findings, particularly the involvement of the insula, extends previous work (Tanaka et al., 2004
) on reward-based learning that has shown a delay-related gradient running from anterior to posterior insular cortex. When subjects learn to make sequences of actions to acquire monetary rewards, anterior and inferior portions of insular cortex appear to be differentially involved in producing reward-prediction error signals related to immediate rewards. In contrast, posterior and superior portions of insular cortex appear to serve this same function when learning about more delayed rewards. Taken together, these studies suggest that decisions involving increased delay are associated with activity in the posterior insula.
Though insular cortex has been implicated in a variety of sensory, cognitive, and emotional processes, there are intriguing intersections between these decision-related findings and previous work on pain. Mirroring the delay-related gradient in insular cortex, work on pain perception has found a similar differentiation along the anterior–posterior axis with more posterior portions associated with the more sensory aspects of pain processing and the more anterior portions associated with the more cognitive or emotional aspects of pain (Singer et al., 2004
). For example, the anticipation of impending pain elicits activity in more anterior portions of the insula than the subsequent pain experience itself (Ploghaus et al., 1999
). More generally, insular cortex has been associated with drug addiction, a condition marked by, and presumably maintained by, pronounced difficulties in weighing short-term gains (e.g., drug-use) against long-term outcomes (e.g., jail, health). For example, cocaine addicts exhibit structural abnormalities in insular cortex including white matter legions (Bartzokis et al., 1999
) and a reduction in gray matter (Franklin et al., 2002
). In particular, insular activity appears to be closely related to drug craving (Garavan et al., 2000
; Kilts et al., 2001
; Schneider et al., 2001
; Bonson et al., 2002
; Brody et al., 2002
; Myrick et al., 2004
; Wang et al., 2007
) and relapse (Paulus et al., 2005
; Naqvi et al., 2007
). Abnormal insular activation has also been found in individuals with ADHD (Ernst et al., 2003
; Rubia et al., 2009
, but see Scheres et al., 2006
) and conduct disorder (Rubia et al., 2009
) who, like addicts, exhibit diminished patience in delay discounting tasks (Barkley et al., 2001
).
Other work has sought to explore what neural features distinguish patient individuals from impatient individuals. Activity in the striatum has been shown (Hariri et al., 2006
) to predict discounting rates across individuals such that larger but less discriminative reward prediction errors are associated with diminished patience. This same pattern of striatal activity was recently shown (Forbes et al., 2009
) to be associated with genetic variation in genotypes thought to influence the release, availability, and signaling strength of dopamine (DAT1, DRD2, and DRD4). There also appears (Boettiger et al., 2007
) to be a relationship between polymorphic variation of the catechol-O-methyltransferase (COMT) gene and delay discounting with the 158Val/Val genotype being associated with diminished patience and hyperactivity in dorso-lateral prefrontal cortex (DLPFC) and posterior parietal cortex (with no apparent effects in the striatum). The 158Val/Val genotype has also been linked to perseverative errors during reinforcement learning tasks which have been attributed to reduced levels of dopamine in prefrontal cortex (Egan et al., 2001
; Frank et al., 2007
). Lastly, there is intriguing evidence (Yacubian et al., 2007
) that variation of COMT and DAT may interact to modulate complex patterns of activity in the striatum during reward processing.
More recent neurocognitive work has explored delay discounting using more fine-grained analytical methods. For example, an fMRI study by Kable and Glimcher (2007)
utilized what they refer to as a “neurometric” approach in order to explore brain regions whose activity varied with the subjective value of various monetary rewards. In their study, decision-makers completed a standard intertemporal choice task in which they chose between pairs of rewards that varied both in their magnitude and in when they would be delivered. For example, a subject might choose between $20 to be delivered that day and $40 to be delivered 30 days later. By observing how changes in delay and reward magnitude modulated behavioral choices, the discounting curves underlying subjects’ choices could be reconstructed (Myerson and Green, 1995
). These reconstructed curves could then be used to compute the idiosyncratic subjective value of any arbitrary reward–delay combination. To explore the neural representation of subjective value, these authors investigated what, if any, brain regions exhibited activity that corresponded to these subjective value functions. The results indicated that the ventral striatum, medial prefrontal cortex, and posterior cingulate cortex exhibited such a pattern of activity. Variation in these regions’ activity was better predicted by subjective value than by several related quantities (e.g., delay, reward magnitude, choice) and closely mirrored individual differences in subjects’ discounting rates.
One problem in relating the neurobiological work on human intertemporal choice with the currently larger literature on non-human animals (Cardinal, 2006
) is that the delays typically utilized in human tasks (e.g., days, months, years) are significantly longer than those used with other animals (e.g., less than a minute). In an attempt to bridge this gap, recent work (Gregorios-Pippas et al., 2009
) has investigated human delay discounting utilizing relatively short delays. Subjects completed a delay discounting task involving delays ranging from 4 to 14 s. Unlike other studies, subjects did not choose between rewards. Instead, subjects were presented with visual cues about impending, temporally delayed rewards with the identity of the cue reliably signaling the length of the delay (although rewards were only paid out at the conclusion of the study). The results reveal that the visual cues elicited graded increases in the ventral striatum (the focus of this study) such that cues associated with shorter delays (thus indicating more subjectively valuable rewards) elicited greater striatal activity. Furthermore, these neural responses mirrored individual subjects’ patterns of choice in a separate behavioral choice task. Intriguingly, these cue-induced neural responses tended to decrease as subjects’ total accumulated reward increased, suggesting a potential neural analog of diminishing marginal utility (Edwards, 1954
). Taken together with the work of Kable and Glimcher (2007)
, these results suggest that activity in the ventral striatum, along with portions of anterior and posterior medial cortex, exhibits a graded signal that represents the subjective value of delayed rewards. This, along with related pharmacological work demonstrating the role of dopamine in delay discounting (Montague and Berns, 2002
; Kheramin et al., 2004
; Winstanley et al., 2004
; Phillips et al., 2007
; Moustafa et al., 2008
) suggests that striatal–cortical circuitry is likely to be a key player in the valuation of delayed rewards and a target for therapeutic work on disorders characterized by impulsive behavior (Rahman et al., 2001
).
Despite the large and growing literature describing the neural signals that represent the idiosyncratic, subjective value of delayed rewards, we ultimately wish to understand the origin of these value signals, their variation across healthy individuals, and their aberrations in clinical populations. If the subjective value of delayed rewards underlies impatient choices occur when, it seems reasonable to ask why they are not valued more strongly. With a better understanding of how subjective value is computed, we would be in a much better position to design both diagnostic instruments and treatments.
Theorizing in psychology has emphasized the idea that choices between delayed rewards (as well as other types of choices) involve a competition between “the passions” and reason (Ainslie, 1975
, 2001
; Schelling, 1984
; Loewenstein, 1996
; Soman et al., 2005
). Some (Metcalfe and Mischel, 1999
) suggested that this competition is between rational, cognitive processes and irrational, emotional processes. Others (Thaler and Sheffrin, 1981
; Ainslie, 1992
; McClure et al., 2004
) have suggested a competition between a prudent, far-sighted process concerned with overall welfare and a greedy, myopic process more concerned with immediate gains. Regardless of the details, what is common across these accounts is the belief that the relative value of waiting and immediate gratification results from a struggle between mutually incompatible drives. If the prudent, rational, cognitive system is able to suppress the greedy, myopic, emotional system, then the decision-maker will see the wisdom of waiting and exhibit relative patience. Otherwise, the emotional system will dominate, producing a strong aversion to waiting and relative impatience.
Several broad literatures have yielded data in support of this general proposal, though it is predominantly indirect in nature. For example, there appear to be large inter-species differences in delay discounting, though the comparison is plagued by methodological differences which make interpretation difficult. Compared to humans, non-human animals exhibit greater impatience for delayed rewards (Logue et al., 1986
). Even monkeys, which exhibit relative cognitive sophistication, will choose immediate rewards over significantly large delayed rewards even when the delay is only several seconds (Kim et al., 2008
; Hwang et al., 2009
). For pigeons, the situation is even more dramatic, with immediate rewards losing approximately 50% of their value when delayed by a single second (Mazur, 1984
). To the extent that one associates prudent, rational control of behavior with frontal lobe function (and to the extent that species differences are not a methodological artifact), these differences across species suggestively mirror the phylogenetic development of frontal cortex (Fuster, 2002
). Similarly, delay discounting behavior appears to follow a systematic trajectory over the course of the human lifespan (Green et al., 1994b
, 1999b
). Relative to young adults, children exhibit significantly less patience for delayed rewards. Here again, this developmental trend is generally consistent with the ontogenetic changes taking place in frontal cortex (Sowell et al., 1999
; Fuster, 2002
). A related and growing literature has also demonstrated a strong relationship between overall intellectual ability and patience (Mischel et al., 1989
; Burks et al., 2009
). Indeed, a recent meta-analysis of 24 relevant delay discounting studies ultimately concluded that higher IQ is reliably associated with greater patience (Shamosh and Gray, 2008
).
Two related studies by McClure and colleagues (McClure et al., 2004
; McClure et al., 2007
) provide the first neural evidence to support the idea that delay discounting involves a dual-process competition. Specifically, this group tested Laibson’s beta-delta account of discounting (Laibson, 1997
) which posits two components: one concerned with immediate rewards (beta) and one concerned with delayed rewards (delta). Using a traditional delay discounting task, decision-makers were asked to choose between pairs of rewards of varying sizes to be delivered at various points in the future. To isolate neural activity associated with the beta component, trials involving an immediate reward were compared with trials that involved only delayed rewards. This comparison revealed several brain regions that exhibited greater activity when faced with an immediate reward. These regions included ventral striatum, medial prefrontal cortex, and posterior cingulate cortex. To isolate the delta component, brain regions that were activated by the task, but that did not distinguish between the different delays were selected. This resulted in a broad network of regions including dorsolateral and ventrolateral portions of prefrontal cortex as well as lateral orbital frontal cortex.
In isolation, these contrasts are relatively coarse, especially given how well-specified the theory being tested is. Critically, however, further analyses demonstrated that the relative activity in these two networks was predictive of subjects’ choices. When faced with a choice between an immediate reward and a delayed reward, choosing the immediate reward was associated with increased activity in the beta network and decreased activity in the delta network. Choices for the delayed reward were associated with the opposite pattern. A recent replication of this study generalized these findings to decisions involving primary rewards (juice and water) and shorter delays (up to 20 min). Contrasts revealed similar networks of brain regions associated with the beta and delta components. Furthermore, choices were again found to be predicted by the relative activity in the two networks, this time utilizing more rigorous regression analyses.
It is interesting to note that the anatomical details of the beta and delta networks grossly mirror the psychologist’s conceptualization. The greedy, irrational, myopic drive is embodied by portions of the evolutionarily older limbic system whereas the rational, patient drive is embodied by the relatively recent frontal cortex (particularly DLPFC). The relationship between activity in these networks and choice behavior also matches the expected competition. To the extent that DLPFC can suppress the relatively insolent limbic system, the decision-maker will make choices that are beneficial in the long-run. If the passionate limbic system can overcome the DLPFC’s control, the decision-maker makes impatient choices.
Along with a fairly well-entrenched theoretical story, investigations into the mechanisms underlying delay discounting face another hurdle; such investigations are simply difficult to conduct. The above investigation of the beta and delta networks is illustrative. Though these findings are consistent with the theoretical framework proposed by its authors, this interpretation has been criticized (Kable and Glimcher, 2007
) as being consistent with alternative formulations. Recall the investigations into the neural representation of subjective value reviewed above. These studies found that activity in a highly similar set of regions was related to both delay and reward magnitude (and their combination, subjective value). Thus, it is possible that the ostensible beta network exhibited greater activity for immediate rewards simply because the immediate reward represented an option with a large subjective value. Furthermore, if choices are made on the basis of subjective value, then it is not surprising that activity in the beta should be related to choice behavior (see regression analyses, McClure et al., 2007
). Below we outline other obstacles.
The work reviewed above illustrates that neuroscientists have done much to shed light on what distinguishes patient from impatient choices and individuals and have even begun to gain insight to the cognitive and neural processes that govern decisions about delayed rewards. However, there is an even more basic issue that has been largely ignored. Why are delayed rewards discounted at all? Why are small, immediate rewards ever tempting enough to eclipse larger, delayed rewards? Why would rational decision-makers not always wait for larger rewards, regardless of the associated delay?
Again, one likely explanation may come from a long history of theorizing in economics (Yaari, 1965
; Benzion et al., 1989
; Prelec and Loewenstein, 1991
; Sozou, 1998
; Dasgupta and Maskin, 2005
), ecology (Kacelnik, 2003
), and psychology (Mischel, 1966
; Stevenson, 1986
; Mazur, 1989
; Rachlin et al., 1991
; Mazur, 1995
, 1997
) which suggests that delay exerts its influence on choices via the perceived risk associated with waiting; a suggestion that has been referred to as the implicit risk hypothesis (Benzion et al., 1989
). If a decision-maker believes that the probability of acquiring a promised reward is uncertain simply by virtue of being delayed, then that decision-maker is justified in reducing the subjective value of the reward. For example, a bird waiting for fruit to ripen might choose to eat some immediately if it believed the fruit’s future availability was not guaranteed (i.e., it could be eaten by a competitor, it might rot, etc.). Furthermore, decision-makers might believe that the probability of receiving a promised reward generally decreases with time which would give rise to the monotonic decreases in subjective value that occur with increases in delay.
In one sense, the implicit risk hypothesis is attractive because it has the potential to entirely eliminate the phenomenon of delay discounting by translating time, the processing of which we are just beginning to grapple with (Mauk and Buonomano, 2004
), into probability and uncertainty, concepts that are relatively well understood. In another sense, however, this hypothesis creates ambiguity when attempting to interpret previous delay discounting results (both behavioral and neural). For example, according to the implicit risk hypothesis, comparisons between immediate and delayed rewards are actually comparisons between high and low probability rewards. Thus, any results from such comparisons (e.g., contrasts in fMRI analyses) could reflect temporal processing or the processing of implicit probability or both. Similarly, one can reconceptualize the computation of subjective value as reflecting implicit probability instead of delay and the same can be done for dual-process accounts of choice.
Because of this potential ambiguity, it is instructive to briefly compare the temporal decision-making results reviewed above with work on choice under risk and uncertainty. Just as with the delay discounting work reviewed above, insular cortex has been implicated in risky decision-making. For example, insular cortex exhibits greater activity when decisions-makers chose a low probability reward than when decisions-makers chose a high probability reward (Paulus et al., 2003
). Furthermore, insular activity predicts the likelihood of choosing low probability rewards. Reward probability also modulates activity in orbitofrontal and ventromedial frontal cortices (Critchley et al., 2001
; Smith et al., 2002
; Clark et al., 2008
; Xue et al., 2009
) as well as in the striatium (Hsu et al., 2005
; Xue et al., 2009
) and activity in the striatum is correlated with the subjective value of risky rewards (Hsu et al., 2005
; Knutson et al., 2005
; Yacubian et al., 2006
; Tobler et al., 2007
; Yacubian et al., 2007
). The overlap between these findings and those from investigations of ostensibly temporal decision-making suggest that there is at least a reasonable possibility that the implicit risk hypothesis is correct. To be clear, reducing temporal decision-making to risky choice in no way trivializes the work on temporal decision-making. Indeed, substantiating the neural equivalence of delay and reward probability would be a major step forward, helping to unify two, currently separate, processes and to validate long-standing theory.
Unfortunately, not all of the empirical evidence for the implicit risk hypothesis is as straightforward. For example, manipulations of probability and delay appear to elicit different patterns of choices (Ostaszewski et al., 1998
; Holt et al., 2003
; Green and Myerson, 2004
; Chapman and Weber, 2006
). For example, as reward magnitudes increase, probability appears to have more influence on behavior, whereas delay appears to have less influence (Green et al., 1999a
). Temporal decisions appear to depend on whether the relevant rewards are immediately consumable (e.g., candy) or not (e.g., money) whereas discounting over probability does not (Estle et al., 2007
). Furthermore, some authors (Green et al., 1999a
) have noted that the lay concept of “impulsivity” seems to best describe an increased preference for low probability rewards (e.g., the temptation to play the lottery) but a decreased preference for delayed rewards (e.g., the temptation to take out a payday loan). Indeed, even with large samples, choice behavior in delay and probability discounting tasks is only weakly correlated within individual subjects (Myerson et al., 2003
). Lastly, there are results showing that delay can have behavioral consequences even when probability is held constant. Work on what is referred to as the temporal resolution of uncertainty (Chew and Ho, 1994
; Arai, 1997
) has found people exhibit strong preferences between gambles in which reward delivery time is fixed and only differ in when the outcome of the gamble is revealed.
The partial dissociation of risky and temporal decision-making implies that the neural basis of temporal decision-making is significantly less clear than it might appear. Without appropriate comparisons, it remains ambiguous as to whether delay discounting results are being driven by delay, the risk implied by delay, or both. Recent work has begun to tackle this issue directly.
The first direct test of the implicit risk hypothesis to utilize fMRI (or any other physiological measure) was recently carried out (Weber and Huettel, 2008
). Subjects in this study were asked to make two sorts of choices. First, subjects performed a classic risky choice task, choosing between rewards that varied in both magnitude and probability (e.g., a 50% chance of $13.50 or a 100% chance of $7). Second, subjects performed a traditional delay discounting task, choosing between rewards that varied in both magnitude and delay (e.g., $6.25 today or $9.25 in 1 month). According to the implicit risk hypothesis, these two conditions should be essentially identical because the stated delays are only influencing choices via the risk they imply. In contrast, this study revealed a variety of brain regions that were differentially engaged by the two tasks. Risky choice elicited greater involvement of posterior portions of parietal cortex, anterior cingulate, and anterior portions of the insula whereas the delay discounting task elicited greater involvement of DLPFC, posterior cingulate, and the caudate. Unfortunately, this comparison was complicated by the fact that subjects exhibited strikingly different patterns of choice in the risky and delayed choice tasks. Thus, it remains somewhat unclear whether the neural dissociation of risk and delay was driven by the task dimensions or other factors.
A recent fMRI study from our lab (Luhmann et al., 2008
) has taken a slightly different approach to this same question. Rather than comparing risky and delay tasks, we instead choose to compare a risky task with a temporal resolution of uncertainty task that involved both risk and delay. Doing so allowed us to exert considerable control over the decision variables and to thus isolate behavioral and physiological results specifically tied to the temporal dimension. Subjects choose between pairs small rewards (10 and 20 cents) that were delivered with varying probabilities (39–100%). In the immediate condition, the uncertainty associated with subjects’ choices was resolved immediately; subjects’ learned whether they would or would not be receiving their chosen reward as soon as they made their choice. In the delay condition, the uncertainty associated with subjects’ choices was resolved only after some variable delay. The delays were constructed such that lower probability rewards were resolved after a longer delay and higher probability rewards were resolved after a shorter delay. Specifically, the probabilities were such that the delay period embodied a constant hazard rate, a pattern that has been theorized to underlie normative delay discounting (Sozou, 1998
; Dasgupta and Maskin, 2005
). Comparing the two conditions, we found that both risk and delay exerted influence on subjects’ choices. Subjects were significantly more likely to choose the larger, less probable reward when the outcomes were revealed immediately, despite the fact that the probabilities were identical. Neurally, the delay condition elicited greater activity in the posterior cingulate than did the immediate condition. Furthermore, we observed parametric effects in the parahippocampal gyri, the anterior cingulate and the portions of superior parietal cortex such that activity in these regions increased as the delay associated with chosen rewards increased. Lastly, we found that differences in individuals’ attitudes toward the delay component of our task were mirrored by activity in a region of frontopolar cortex.
These two studies appear to contradict the implicit risk hypothesis and may begin to shed some light on how delay and risk exert dissociable influences on choice. We have noted that the specific brain regions implicated in temporal processing in our study have also been implicated in the process of prospection, the imagining of events in one’s future (Okuda et al., 2003
; Addis et al., 2007
; Buckner and Carroll, 2007
; Hassabis et al., 2007
; Szpunar et al., 2007
; Addis and Schacter, 2008
). Thus, we suggested that one way in which temporal decisions might differ from similar, risky decisions, is that deliberation about temporal choices is likely to involve evaluating both the reward and its value, but also the experience of waiting itself. Our subjects’ choice behavior implied that delaying the resolution of uncertainty decreased the subjective value of options (much like when reward delivery is delayed). This very well may be due to the fact that the delay interval itself evoked a negative subjective experience. Decision-makers able to foresee such experiences as they deliberated their choices would be in a much better position to make superior choices.
This possibility highlights the true complexity of temporal decision-making. Risky choice involves presenting a choice to the decision-maker, allowing a choice to be made, and resolving the outcome, all of which can and usually does happen rather quickly. Temporal decisions, on the other hand, are made at one point in time, but produce consequences that are subsequently stretched out over time. Decision-makers have subjective experiences as they attempt to make decision, while waiting, when uncertainty is resolved, and when receiving (or not receiving) the reward itself. To the extent that any or all of these experiences can be forecast before choices are made (Wilson and Gilbert, 2005
), they can presumably exert some influence on decision-makers’ behavior (Loewenstein, 1987
; Loewenstein et al., 2001
).
Indeed, we already know something about the physiology of anticipation itself. For example, in a particularly elegant fMRI study (Berns et al., 2006
), human subjects were shown cues followed by a variable delay and then an electric shock. The identity of the cue signaled the duration of the delay interval, so subjects knew in advance how long they would have to wait. As subjects waited for the shock delivery, a network of brain regions exhibited a complex and theoretically interesting pattern of activity. Regions that respond to pain, including somatosensory cortex, insular cortex, and the anterior cingulate, exhibited activity that reflected both the anticipation of the impending shock as well as dread, the negative, subjective experience associated with the waiting itself. Furthermore, the neural patterns exhibited during this delay period were associated with preferences in a behavioral decision-making task performed separately. Those subjects that exhibited the strongest neural effects of dread were more likely to choose stronger, immediate shocks over weaker, but delayed shocks.
We also know that delay period activity is modulated by factors such as risk. In one study (Critchley et al., 2001
), decision-makers made risky choices and had the outcomes of their choices withheld for 8 s. Across trials, the probability of winning was varied to investigate anticipatory processes. Several regions, including the anterior cingulate and orbital frontal cortex, and anterior insula exhibited delay period activity that reflected the amount of uncertainty subjects were facing.
These findings suggest that temporal decisions pose a formidable challenge for the savvy decision-maker. Despite the relative simple descriptions temporal outcomes can take (e.g., $100 in 12 months), they actual embody a complex sequence of events including emotional and cognitive events, each of which can potentially influence the subjective value of a choice. Delay periods can elicit negative emotional reactions (e.g., dread) and thus decrease the value of delayed outcomes, but waiting can also elicit positive emotional reaction (Loewenstein, 1987
; Chew and Ho, 1994
) and thus act to increase value. This lability, coupled with the finding that people are not necessarily adept at predicting their future emotional reactions (Wilson and Gilbert, 2005
), begins to make temporal choice look even more difficult. Furthermore, not only do these factors complicate temporal decision-making itself, they also complicate our attempts to study it; attempts to fully control and dissociate each of the relevant influences are unlikely to be feasible. Nonetheless, studies that acknowledge and take these factors into account certainly have the potential to help overcome some of the ambiguities noted above and to paint a much richer picture of temporal decision-making.
The work reviewed above suggests that we are just beginning to gain insight into the nature temporal decision-making. Much of this work has sought to explore the phenomenon of delay discounting; the finding that waiting for rewards decreases their attractiveness. As a first step, this work has begun to characterize the difference between patient and impatient choices within a single individual as well as between patient individuals and impatient individuals. While critically informative, we ultimately need to understand the processes that operate to produce temporal decisions. Fortunately, the study of decision-making has a wealth of theoretical tools available from economics. More recent work has attempted to leverage economic theories to better understand the neural patterns observed during decision-making. The results have been provocative, but there are many questions left unanswered.
We have pointed to several places where there is currently ambiguity in the treatment of temporal decision-making: correlates of patience vs. subjective value signals, delay vs. risk, etc. Furthermore, we have tried to point to a small number of relatively unexplored dimensions that are likely to be relevant for temporal decision-making: affective forecasting, anticipation, etc. This is certainly not an exhaustive list, as others (Berns et al., 2007
; Wittmann and Paulus, 2008
) have noted additional relevant factors. At this point, it appears that, despite complicating our experimental designs, investigating the interactions between these factors would be most valuable in illuminating the subtleties of temporal decision-making. Given the clinical and public policy implications of temporal decision-making as well as the sheer scientific potential in psychology, neuroscience, and economics, the benefits of such an approach seem to outweigh the costs.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
We are grateful to Heather Moore and Marla Krukowski for comments and feedback on drafts of this manuscript.
Bartzokis, G., Goldstein, I. B., Hance, D. B., Beckson, M., Shapiro, D., Lu,P. H., Edwards, N., Mintz, J., and Bridge, P. (1999). The incidence of T2-weighted MR imaging signal abnormalities in the brain of cocaine-dependent patients is age-related and region-specific. Am. J. Neuroradiol. 20, 1628–1635.
Rubia, K., Smith, A. B., Halari, R., Matsukura, F., Mohammad, M., Taylor, E., and Brammer, M. J. (2009). Disorder-specific dissociation of orbitofrontal dysfunction in boys with pure conduct disorder during reward and ventrolateral prefrontal dysfunction in boys with pure ADHD during sustained attention. Am. J. Psychiatry 166, 83–94.