Decisions from experience: adaptive information search and choice in younger and older adults

In real-world decision making, choice outcomes, and their probabilities are often not known a priori but must be learned from experience. The dopamine hypothesis of cognitive aging predicts that component processes of experience-based decision making (information search and stimulus–reward association learning) decline with age. Many existing studies in this domain have used complex neuropsychological tasks that are not optimal for testing predictions about specific cognitive processes. Here we used an experimental sampling paradigm with real monetary payoffs that provided separate measures of information search and choice for gains and losses. Compared with younger adults, older adults sought less information about uncertain risky options. However, like younger adults, older participants also showed evidence of adaptive decision making. When the desirable outcome of the risky option was rare (p = 0.10 or 0.20), both age groups engaged in more information search and made fewer risky choices, compared with when the desirable outcome of the risky option was frequent (p = 0.80 or 0.90). Furthermore, loss options elicited more sampling and greater modulation of risk taking, compared with gain options. Overall, these findings support predictions of the dopamine hypothesis of cognitive aging, but they also highlight the need for additional research into the interaction of age and valence (gain vs. loss) on experience-based choice.

Introduction

Decision making in everyday life often involves choosing among options that vary with respect to potential payoffs and outcome probabilities. Typically, payoffs and risks are not known a priori but must be inferred from past experience (e.g., Hertwig et al., 2004). For example, a commuter may have to choose between alternate routes to work. One route may be more direct (high payoff) but carry a larger risk of delays, whereas another route may be longer (low payoff) but carry no risk of delays. In this scenario, a poor decision may cost the decision maker no more than a few minutes of commuting time. In high-stakes domains such as healthcare, financial planning, and consumer choice, however, the costs associated with maladaptive decisions can be considerable.

What characterizes adaptive decision making in the context of experience-based risky choice? One aspect is predecisional information search, or the extent to which the decision maker explores the available options prior to making a choice. In the commuting example, the decision maker’s chances of choosing the optimal route are likely to increase as a function of the number of times that she has used each route before. A second aspect is stimulus–reward learning, or the ability to learn from positive and negative feedback. If the decision maker is insensitive to successes (short commutes) and failures (long commutes), her choices are unlikely to change as a function of past experience. A third aspect of adaptive decision making is adaptive choice. This refers to the degree to which the decision maker modulates her choices according to the level of risk, risk being defined by the probability or the variability of the possible outcomes of a choice (Glimcher, 2008). For example, other things being equal, routes with a small probability of delays should be chosen more often than routes with a high probability of delays.

The effects of aging on decision making have recently moved to the forefront of research in psychology and neuroeconomics. This trend has been partly motivated by evidence of age-related decline in dopaminergic neurotransmission in the brain (e.g., Wang et al., 1998; Kaasinen et al., 2000). According to the “dopamine hypothesis” of cognitive aging (e.g., Li et al., 2001; Braver and Barch, 2002; Bäckman et al., 2006, 2010), deficient dopaminergic neuromodulation is one of the causes of age-related cognitive decline. In younger adults, dopaminergically innervated brain structures such as the ventral striatum and the ventromedial prefrontal cortex have been linked to decision-relevant functions such as information search and exploration (e.g., Düzel et al., 2010), stimulus–reward learning (e.g., Schultz, 2000), as well as the coding of uncertainty (e.g., Fiorillo et al., 2003; Fiorillo, 2011). According to the dopamine hypothesis of aging, each of these aspects of experience-based choice should therefore show age-related decline. However, the evidence from extant behavioral and neuroimaging studies is mixed.

Behavioral studies in domains such as consumer choice and medical decision making suggest that compared with younger adults, older adults prefer to have fewer choice options (Reed et al., 2008), seek less variety (Novak and Mather, 2007), and, critically, seek less information about choice options (for reviews, see Mather, 2006; Mata and Nunes, 2010). At least two studies indicate, however, that age-related decline in information seeking may be limited to situations in which choice options have negative features (Mather et al., 2005; Lockenhoff and Carstensen, 2007). This finding has been explained in terms of socioemotional selectivity theory (Carstensen et al., 1999), which postulates that age-related reductions in time perspective lead older adults to prioritize emotion-regulation goals at the expense of information-seeking goals. By this account, older adults may avoid seeking out negative information about choice options in order to protect their emotional well-being.

Stimulus–reward learning in younger and older adults has been investigated with a number of different neuropsychological and neuroimaging tasks. Studies using variants of the Probabilistic Object Reversal Task (Heekeren et al., 2007), which requires flexible adjustment to changes in stimulus–reward contingencies, have revealed age-related decline in learning from positive feedback (Mell et al., 2005), as well as under-activation of the ventral striatum in response to reward cues (Mell et al., 2009). In the Iowa gambling task (IGT; Bechara et al., 1994), participants make repeated draws from decks of cards that differ with respect to their expected value. To maximize their scores, participants must learn to choose “good decks” over “bad decks,” an ability that is impaired in patients with damage in the ventromedial prefrontal cortex (e.g., Bechara et al., 2000). Successful performance requires learning of outcome contingencies, but likely also taps other cognitive–affective processes (e.g., Wood et al., 2005). Findings with healthy older adults have been mixed, with some studies showing age-related deficits (Denburg et al., 2005; Fein et al., 2007; Zamarian et al., 2008), and others reporting no age differences (Kovalchik et al., 2005; Wood et al., 2005; see also Hosseini et al., 2010). There is also no consistent evidence for age-by-valence effects on learning (but see Wood et al., 2005; Denburg et al., 2006). In the Probabilistic Selection Task (Frank et al., 2004), participants have to acquire positive and negative outcome contingencies, and there are separate learning measures for both. One study using this task with “younger-old” and “older-old” participants suggested that negative feedback may be more effective than positive feedback in the second group, possibly due to low tonic dopamine levels in advanced old age (Frank and Kong, 2008). In a more recent study with younger and older adults (Hämmerer et al., 2011), there was evidence of age-related decline in stimulus–reward learning, as well as a negative learning bias in older adults. There are also data suggesting that the proportion of individuals with a negative learning bias may increase with age (Simon et al., 2010). However, Samanez-Larkin et al., 2007, supplementary materials) found no age-by-valence interaction in a probabilistic learning task. Finally, neuroimaging studies with the monetary incentive delay (MID) task (Knutson et al., 2001), designed to separate brain activations related to anticipation and receipt of monetary gains and losses, have shown age-related reductions in anticipatory brain signals (Schott et al., 2007; Dreher et al., 2008). These findings have added to the evidence that reward-based learning is impaired in older adults. However, in one study with the MID task, older adults showed anticipatory responses similar to those observed in younger adults, at least during gain anticipation (Samanez-Larkin et al., 2007). Striatal activity associated with outcome processing also appears to be normal in older adults (Cox et al., 2008). In summary, the literature on stimulus–reward learning is mixed. There is a fair amount of evidence for an age-related learning deficit, but to what extent this deficit is modulated by valence is as yet unclear.

A final aspect of experience-based decision making is risk preference. Given two options of identical expected value, a risk-averse decision maker prefers low-risk to high-risk options (for a full definition of the term, see Glimcher, 2008). The idea that older adults are more risk-averse than younger adults has a long history in psychology (for an early review, see Okun, 1976), but empirical support for it is surprisingly scant. Studies of risky choice in real-world domains such as financial planning and gambling have often failed to show age-related increases in risk aversion (for a review, see Mather, 2006). A recent quantitative meta-analysis of the experimental literature (Mata et al., 2011) indicates that the size and direction of age differences in risk preference depends strongly on the task, with no evidence for task-general age-related changes. This heterogeneity is also illustrated by the two existing neuroimaging studies of aging and risky choice. In one of these studies (Lee et al., 2008), older adults made more risk-averse gambling decisions. They also showed greater activation in the right insula when choosing risky options, perhaps reflecting a stronger negative anticipatory response to risk. However, in another study (Samanez-Larkin et al., 2010) using a financial investment task, older adults were more risk-seeking than younger adults. Interestingly, this effect was shown to be associated with increased variability in subcortical brain activity, consistent with Li et al.’s (2001) proposal that age-related dopaminergic decline leads to increased “neural noise” and reduced sensitivity to outcome probabilities. Overall, there is no clear evidence for a systematic age-related shift in risk preferences. Instead, age differences in task-specific risk preferences may reflect differences in the learning demands of the tasks (Mata et al., 2011).

In summary, the dopamine hypothesis of cognitive aging predicts that critical aspects of experience-based decision making (information search and stimulus–reward learning) undergo age-related decline. While this prediction has received support in some studies, there are many inconsistent findings, perhaps partly due to the diversity and complexity of the paradigms used (e.g., the IGT). Few studies have systematically compared younger and older adults on multiple components of experience-based risky choice within the same experimental setting (but see Deakin et al., 2004). Furthermore, only about half of the existing studies have used monetary incentives (Mata et al., 2011). There is a dearth of studies comparing gain- and loss-related decisions (Mata et al., 2011), even though several lines of evidence suggest that aging may affect the two types of choices differently (e.g., Mather et al., 2005; Samanez-Larkin et al., 2007; Frank and Kong, 2008). Finally, we noted a lack of theoretical integration between the neuropsychological and aging literatures, on the one hand, and the growing field on experience-based choice research in the behavioral economics literature, on the other (for reviews, see Hertwig and Erev, 2009; Rakow and Newell, 2010). Together, these observations provided the rationale for the current study.

Younger and older adults completed a computerized sampling task (Hau et al., 2010). Participants were presented with a series of financial choice problems requiring a choice between a certain option (e.g., $3 for sure) and a risky option (e.g., $4 with a probability of 0.80, else nothing). The certain option was described explicitly, whereas the risky option had to be explored through active sampling. Because each choice problem features only one uncertain option, the task is less complex than the otherwise similar IGT, in which participants must track outcomes for multiple decks of cards. The sampling task provides separate measures of information search and choice, and it allows for a direct comparison of gain-related and loss-related decisions. It also involves manipulation of two aspects of risk: outcome probability and payoff variability (i.e., the SD of the outcomes of the risky option; Hau et al., 2010), neither of which has been systematically investigated in the aging literature.

We tested the following hypotheses:

1. In line with the idea that age-related dopaminergic decline leads to reduced information seeking (Düzel et al., 2010), we hypothesized that older adults would sample less than younger adults before making choices.

2. We predicted that participants would show adaptive decision making by modulating their choices according to objective outcome probabilities (“adaptive choice”; see also Deakin et al., 2004). On the assumption that information search and choice behavior reflect similar dopaminergic influences (Düzel et al., 2010), we made the novel prediction that sampling would also be sensitive to variations in outcome probabilities (“adaptive sampling”).

3. Based on the idea that age-related dopaminergic decline results in noisy memory representations (Li et al., 2001; Samanez-Larkin et al., 2010), as well as prior evidence of impaired stimulus–reward learning in older adults (for a review, see Mohr et al., 2010), we hypothesized that adaptive sampling and adaptive choice would show age-related decline.

Additional questions for which the prior literature provided no clear hypotheses were (1) how the valence of the choice options (gain vs. loss) would affect age differences in experience-based decision making; (2) whether younger and older adults would differ with respect to overall risk preference; and (3) how payoff variability would affect age differences in experience-based decision making.

Materials and Methods

Participants

All participants gave written informed consent for the study, which was approved by the Research Ethics Board at Ryerson University. Participants in the final sample included 40 younger adults (35 women) who were students at Ryerson University and received course credit for their participation, as well as 41 community-dwelling older adults (32 women) who received a $10 travel reimbursement. Participants in both groups also had the opportunity to win monetary rewards for their performance in the experimental task. Eight additional younger adults were excluded for failing to meet one or more of the criteria for inclusion: absence of major health problems (e.g., history of neurological or psychiatric illness, cancer, cardiovascular disease), normal, or corrected-to-normal vision and hearing, and a score of 27 or higher on the Mini-Mental Status Examination (Folstein et al., 1975). Two additional older adults were excluded for failing to follow the experimental instructions. Characteristics of the final sample are shown in Table 1. Compared with younger adults, older adults had significantly more years of education, t(79) = 5.86, p < 0.01 and scored higher on the Mill-Hill Vocabulary Scale (Raven, 1982), t(77) = 8.59, p < 0.01. Scores on a numeracy questionnaire (Reyna and Brainerd, 2008) showed no significant age difference.

TABLE 1

Table 1. Participant characteristics, by age group.

Design

The study employed a mixed factorial design that included the between-subjects factor age group (younger, older) and two within-subjects factors: (1) valence (gain, loss), (2) the probability of the desirable outcome of the risky option (p_desirable: 0.10, 0.20, 0.80, 0.90), and (3) the payoff variability of the risky option ($1.6, $4.5, $9.6). Dependent variables were the sampling frequency, which provided a measure of information search, and the proportion of risky choices, which provided an index of risk taking.

Stimuli and Apparatus

We used the 12 choice problems (Hau et al., 2010; Experiment 1) shown in Table 2. In each problem, participants chose between a risky option (winning X with probability p_non-zero, or 0 with probability 1 − p_non-zero) and a certain option (winning Y with probability 1.0). Y was either slightly below or slightly above the expected value of the risky option. For example, in Problem 1, the expected value of the risky option was $0.53, and Y was either $0.30 or $0.70. For each participant, half of the problems used the smaller value of Y and half used the larger value of Y. The assignment of specific problems to the first group (small Y) and the second group (large Y), respectively, was counterbalanced across participants.

TABLE 2

Table 2. Twelve choice problems (Hau et al., 2010).

In Hau et al.’s (2010) protocol, all problems involved choosing between risky and certain gains (i.e., X and Y were always positive numbers). In contrast, we presented participants with both gain and loss problems. To this end, we created a loss version of each problem. For example, the loss version of Problem 1 required participants to choose between a 10% chance of losing $5.30 (risky option) and a 100% chance of losing $0.30/$0.70 (certain option).

For the purpose of data analysis, the variable p_non-zero (probability of the non-zero outcome) was recoded into p_desirable (probability of the desirable outcome), with the following rationale. Adaptive decision making involves maximizing desirable outcomes. On gain trials, it is adaptive to choose the risky option when p_non-zero (the probability of a gain) is high. On loss trials, it is adaptive to choose the risky option when p_non-zero (the probability of a loss) is low. To conduct meaningful comparisons between gain and loss trials, p_desirable is therefore more useful than p_non-zero. For gain trials, p_desirable equaled p_non-zero. For loss trials, p_desirable was the probability of a non-loss, or 1 − p_non-zero.

E-Prime 2.0 (Psychology Software Tools, Inc.) was used for stimulus presentation and response collection on an Intel Core 2 Duo 2.40 GHz laptop with 4 GB of RAM and a 16.0″ LCD display running 32-bit Windows 7 Enterprise Edition. Viewing distance was approximately 50 cm. All text appeared in black 18-point Calibri font against a white background. Participants pressed the “X” and “,” keys with their left and right index fingers to give their responses.

Procedure

Participants were tested individually in a quiet testing room. After signing a consent form, participants received instructions for the choice task and completed four practice problems. After the practice, the experimenter repeated the instructions and provided clarification if necessary. Participants were informed that they would start with a balance of $0, and that the computer would keep track of their gains and losses throughout the study. Participants were also told that they would receive their final balance in cash, if it was greater than 0. Older adults were reassured that the experimental rewards would be paid in addition to the compensation they would receive for participating in the study.

The 24 trials of the choice task included 12 gain and 12 loss problems, presented in random order. At the beginning of each trial, an on-screen message informed participants of the number of the upcoming trial (1–24). The message also indicated whether the upcoming trial was a gain trial (“you should try to maximize your gains”) or a loss trial (“you should try to minimize your losses”). The risky option and the certain option were then presented side-by-side on the screen, separated by a central black vertical line (Figure 1). The left/right assignment of the two options was counterbalanced across trials for each participant. The outcome of the certain option was presented explicitly, whereas the risky option (symbolized by the playing card with a question mark) had to be explored through sampling. Participants were told that their balance would not be affected by the sample outcomes, and that they should sample the risky option until they felt that they knew which option they preferred. Participants sampled by pressing a key and immediately saw the outcome of the sample superimposed on the playing card. Each sample outcome was a random draw from the probability distribution of the risky option. After each sample, participants indicated, again via button press, whether they wished to continue sampling or to make their final choice. After they had finished sampling and made their final choice of either the certain or the risky option, a feedback screen indicated the trial outcome. If the participant had chosen the risky option, the outcome was again determined by a random draw from the probability distribution of the risky option. All aspects of the trials were participant-paced, and there was no upper limit on the number of samples drawn on a given trial. After the last trial, the final balance was shown on the screen.

FIGURE 1

Figure 1. Illustration of the trial sequence. The example shows the loss version of Problem 4 (see Table 2). The certain option (losing $3.40) is presented on the left, the risky option (losing $4 with probability 0.80, losing nothing with probability 0.20) is on the right. The participant samples the risky option four times and experiences both outcomes twice. The participant’s final choice is the risky option, resulting in a $4 loss.

At the end of the session, participants completed a set of paper-and-pencil questionnaires, including the Numeracy scale (Reyna and Brainerd, 2008), an 11-item questionnaire measuring proficiency with fractions, proportions, decimals, and percentages, and the Mill-Hill Vocabulary Scale, a 33-item vocabulary test in multiple-choice format (Raven, 1982). The experimenter also administered the Mini-Mental State Exam (Folstein et al., 1975). Afterward, participant were paid (if applicable) and debriefed about the goals of the study.

Results

The analyses focused on two aspects of experience-based decision making: information search, operationalized as the sampling frequency, and choice, operationalized as the proportion of risky final choices. Analyzing the effects of all independent variables in a single step was not possible due to the small item set (see Table 2). We thus conducted two sets of analyses for each dependent variable. Because of the significant age-group difference in educational attainment (see Participants), the variable “years of education” was mean-centered and included as a covariate in the analyses below (Delaney and Maxwell, 1981). Vocabulary, which also showed an age difference, was not included as an additional covariate because it was significantly correlated with education (r = 0.43, p < 0.01).

Information Search: Sampling Frequency

In the first analysis, we collapsed the sampling frequencies across the levels of the payoff variability factor (Figure 2) and conducted a 2 (age group: younger, older) × 2 (valence: gain, loss) × 4 (p_desirable: 0.10, 0.20, 0.80, 0.90) mixed analysis of covariance (ANCOVA) with education as a covariate. Consistent with Hypothesis 1 (age-related reduction in overall sampling frequency), the main effect of age group was significant, F(1, 78) = 5.08, p = 0.03, partial η² = 0.06, with older adults (M = 6.84) sampling less than younger adults (M = 10.73). The main effect of valence was significant, F(1, 78) = 4.11, p = 0.05, partial η² = 0.05, reflecting the fact that loss trials elicited more sampling (M = 9.09) than gain trials (M = 8.47). A significant main effect of p_desirable, F(3, 234) = 19.37, p < 0.01, partial η² = 0.20, was qualified by a significant interaction of age group and p_desirable, F(3, 234) = 4.72, p < 0.01, partial η² = 0.06. Consistent with Hypothesis 2 (adaptive sampling), planned linear contrasts indicated that younger adults sampled more as p_desirable decreased, F(1, 38) = 14.03, p < 0.01, partial η² = 0.27. Consistent with Hypothesis 3 (age-related reduction in adaptive sampling), the effect was only marginally significant in older adults, F(1, 39) = 3.63, p = 0.06, partial η² = 0.09. There was no significant interaction of age and valence on sampling frequencies.

FIGURE 2

Figure 2. Sampling frequency as a function of valence and the probability of the desirable outcome (p_desirable), for younger and older adults. Adjusted means for the average level of the covariate (years of education) are shown. Error bars indicate SEM.

In the second analysis, we collapsed the sampling frequencies across the levels of p_desirable (Figure 3) and conducted a 2 (age group) × 2 (valence) × 3 (payoff variability: 1.6, $4.5, $9.6) mixed ANCOVA with education as a covariate. In addition to the significant main effects of age group and valence, reported in the previous analysis, there was a significant interaction of valence and payoff variability, F(2, 156) = 3.18, p = 0.04, partial η² = 0.04. Follow-up linear contrasts indicated that payoff variability had no effect on sampling frequency for gain trials. For loss trials, increases in payoff variability were associated with increased sampling, F(1, 78) = 6.47, p = 0.01, partial η² = 0.08.

FIGURE 3

Figure 3. Sampling frequency as a function of valence and payoff variability, for younger and older adults. Adjusted means for the average level of the covariate (years of education) are shown. Error bars indicate SEM.

Risk Taking: Proportion of Risky Choices

In the first analysis, we collapsed the proportions of risky choices across the levels of the payoff variability factor (Figure 4) and conducted a 2 (age group) × 2 (valence) × 4 (p_desirable) mixed ANCOVA with education as a covariate. The main effect of p_desirable was significant, F(3, 234) = 49.06, p < 0.01, partial η² = 0.39. It was qualified by a significant Valence × p_desirable interaction, F(3, 234) = 6.84, p < 0.01, partial η² = 0.08. Consistent with Hypothesis 2 (adaptive risk taking), planned linear contrasts indicated that risk taking increased as p_desirable increased, suggesting that both age groups engaged in adaptive risk taking. This was the case for both gain trials, F(1, 78) = 24.16, p < 0.01, partial η² = 0.24, and loss trials, F(1, 78) = 78.81, p < 0.01, partial η² = 0.50, but the effect was more pronounced for loss trials than for gain trials (Figure 4). There was no significant interaction of age group and p_desirable, contrary to Hypothesis 3 (age-related reduction in adaptive risk taking). Age group and valence also did not interact significantly.

FIGURE 4

Figure 4. The proportion of risky choices as a function of valence and the probability of the desirable outcome (p_desirable), for younger and older adults. Adjusted means for the average level of the covariate (years of education) are shown. Error bars show the SEM.

In the second analysis, we collapsed the proportions of risky choices across the levels of p_desirable (Figure 5) and conducted a 2 (age group) × 2 (valence) × 3 (payoff variability) mixed ANCOVA with education as a covariate. The only significant effect was the interaction of valence and payoff variability, F(2, 156) = 8.03, p < 0.01, partial η² = 0.09. Follow-up contrasts showed a significant quadratic effect of payoff variability on the proportion of risky choices for gain trials, F(1, 78) = 5.94, p = 0.02, partial η² = 0.07, with the medium level of payoff variability producing the highest proportion of risky choices. For loss trials, there was a significant linear effect of payoff variability, F(1, 78) = 15.82, p < 0.01, partial η² = 0.17, reflecting a linear increase in the proportion of risky choices with increasing payoff variability.

FIGURE 5

Figure 5. The proportion of risky choices as a function of valence and the payoff variability, for younger and older adults. Adjusted means for the average level of the covariate (years of education) are shown. Error bars show the SEM.

Discussion

In this experiment, younger and older adults completed a series of decision problems involving choices between certain and risky financial options. In each problem, the risky option was not explicitly described, but had to be explored through sampling. To test a set of predictions derived from the literature on dopamine, aging, and cognition (Li et al., 2001; Frank and Kong, 2008; Düzel et al., 2010), we analyzed the effects of age, valence, the probability of the desirable outcome in the risky option, and payoff variability on two dependent measures: sampling frequency, which provided an index of information search, and the proportion of risky choices, which provided an index of risk taking.

As predicted (Hypothesis 1), older adults overall sampled less than younger adults. This finding is consistent with the proposal that aging is associated with reduced exploratory drive due to dopaminergic decline (Düzel et al., 2010). Alternatively, it is possible that sampling frequency is constrained by working-memory capacity (e.g., Rakow et al., 2010), and that the reduced sampling in older adults is a consequence of age-related decline in working memory (e.g., Hasher and Zacks, 1988). Working memory is closely tied to the dorsolateral prefrontal cortex (e.g., D’Esposito et al., 1995; Cohen et al., 1997), a region that receives dopaminergic input and undergoes significant structural change in aging (e.g., Grady et al., 1994; Raz et al., 2005). We did not obtain a measure of working-memory capacity and thus could not test this hypothesis directly. However, the young-adult literature suggests that access to explicit memory representations may not be critical for experience-based choice. In one study, memory demands were lifted entirely by providing participants with a visual record of their sampling histories (Hau et al., 2010, Experiment 1). Choice patterns in the visual-record condition did not differ significantly from those in the standard sampling condition – even when participants were forced to sample as many as 50 times. Furthermore, lower animals (e.g., worker bees) show experience-based choice patterns that resemble those of humans (Weber et al., 2004). These findings suggest that implicit, rather than explicit, memory representations may drive experience-based choice, and that working memory may not play a major role. Indeed, in one recent study, decision quality in an experience-based investment task was unaffected by the addition of a secondary task, for both younger and older adults (Samanez-Larkin et al., 2011). Even so, it is possible that older adults use working-memory load as a metacognitive heuristic for terminating their information search. One strategy for testing these possibilities in future research would be to adopt the visual-record method of Hau et al. (2010; see also Samanez-Larkin et al., 2011). If the age-related difference in sampling frequency results from working-memory limitations rather than from reduced exploratory drive, then providing participants with a visual record of their sampling histories should eliminate age differences in sampling.

In line with Hypothesis 2, both younger and older adults showed adaptive decision making by adjusting their sampling and choice behavior in response to variation in the probability of the desirable outcome of the risky option. Adaptive choice of this kind has been demonstrated previously (e.g., Deakin et al., 2004; Hau et al., 2010), but we are not aware of previous reports of adaptive sampling. At a mechanistic level, the effect can be parsimoniously explained in terms of dopaminergic modulation of both information search/exploration and risk taking. At the subjective level, the increased sampling when the desirable outcome is rare (p = 0.10 or 0.20) may be associated with increased curiosity (i.e., anticipation of epistemic reward; e.g., Kang et al., 2009). In future research, think-aloud protocols during the sampling phase could help to shed light on the association between the subjective experience of curiosity and predecisional information search.

Based on theoretical models of “noisy processing” due to reduced dopaminergic neuromodulation in older adults (Li et al., 2001), as well as empirical evidence for impaired feedback learning in older adults (for a review, see Mohr et al., 2010), we had predicted that adaptive decision making would show age-related decline (Hypothesis 3). This hypothesis was supported for sampling, where older adults showed significantly flatter functions than younger adults (Figure 2). However, there was no evidence for an age-related deficit in adaptive choice (Figure 4). This dissociation indicates that age-related dopaminergic decline may affect information search more strongly than choice. Incidentally, the dissociation also suggests that sampling frequency had little effect on choice, at least within the range of sampling frequencies that we observed. To further explore this issue, we examined the relationship between sampling frequencies and the proportions of risky choices in both age groups, and found no significant correlations. This is consistent with findings in the behavioral decision-making literature which suggest that the impact of additional experience on choice is modest, and that participants’ default strategy is to rely on small samples despite the fact that small samples are more susceptible to sampling error (e.g., Hau et al., 2010; Hertwig and Pleskac, 2010).

In addition to testing specific hypotheses, the study also allowed us to address a set of exploratory research questions. The first of these questions concerned valence effects on experience-based decision making. To our knowledge, the current study was the first to systematically compare experience-based choice for gains and losses using the same choice problems. Both age groups sampled more in the face of losses than they did in the face of gains. Regardless of age, loss anticipation thus appears to energize predecisional information search more strongly than gain anticipation, consistent with Kahneman and Tversky’s (1979) influential loss-aversion hypothesis. Both age groups also showed greater adaptive modulation of risk taking for losses than for gains. Overall, these findings suggest that experience-based choice is affected by the valence of the choice options, but they offer no evidence to suggest that the nature of this modulation changes with age.

The second exploratory question concerned age differences in overall risk preference. Across both the gain and loss domains, the average proportion of risky choices was similar for younger and older adults, contrary to the widely held notion that aging is associated with increased risk aversion. Mata et al. (2011) suggested that age differences in risk preference are a by-product of age-related learning impairments, which can manifest differently in different tasks. In the current study, there was no evidence for an age-related learning deficit, as both age groups showed similar sensitivity to variation in outcome probabilities. The lack of an age effect on overall risk preference is thus consistent with Mata et al.’s (2011) view.

The third exploratory question concerned payoff variability, an aspect of risk that combines the probability and magnitude of outcomes. Consistent with a previous study with younger adults (Hau et al., 2010, Experiment 1), the effects of payoff variability on experience-based decision making were relatively subtle. The most notable finding here was that, for both age groups, increased payoff variability led to modest linear increases in both sampling and risky choice on loss trials, but not on gain trials. This finding further highlights the impact of valence on experiential decision making.

In conclusion, the current findings add to the growing literature on aging and neuroeconomics by (a) providing novel empirical observations of age differences in experience-based choice, (b) on the basis of the dopamine hypothesis of cognitive aging, testing hypotheses about specific cognitive processes involved in experience-based choice, and (c) demonstrating the need for greater integration of research in aging, neuroeconomics, and behavioral economics. An obvious limitation of the study was that we used behavioral methods to test predictions derived from a neurobiological hypothesis. However, behavioral data are valuable, indeed necessary, for testing and constraining models of neurocognitive age-related change (e.g., Frank and Kong, 2008; Simon et al., 2010). In future work, a multimodal approach that combines behavioral assessment with measurement (or pharmacological manipulation) of dopamine biomarkers will be the method of choice for testing causal influences of dopamine on experience-based choice in younger and older adults.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This research was supported by funding awarded to Julia Spaniol by the National Sciences and Engineering and Research Council (DG 358797) and the Canadian Institutes of Health Research (IAP-107854). The authors thank Christian Hoverath for his assistance with the preparation of the experimental materials and pilot testing.

References

Bäckman, L., Lindenberger, U., Li, S. C., and Nyberg, L. (2010). Linking cognitive aging to alterations in dopamine neurotransmitter functioning: recent data and future avenues. Neurosci. Biobehav. Rev. 34, 670–677.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bäckman, L., Nyberg, L., Lindenberger, U., Li, S. C., and Farde, L. (2006). The correlative triad among aging, dopamine, and cognition: current status and future prospects. Neurosci. Biobehav. Rev. 30, 791–807.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bechara, A., Damasio, A. R., Damasio, H., and Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50, 7–15.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bechara, A., Tranel, D., and Damasio, H. (2000). Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions. Brain 123, 2189–2202.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Braver, T. S., and Barch, D. A. (2002). A theory of cognitive control, aging cognition, and neuromodulation. Neurosci. Biobehav. Rev. 26, 809–817.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Carstensen, L. L., Isaacowitz, D. M., and Charles, S. T. (1999). Taking time seriously – a theory of socioemotional selectivity. Am. Psychol. 54, 165–181.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Cohen, J. D., Perlstein, W. M., Braver, T. S., Nystrom, L. E., Noll, D. C., Jonides, J., and Smith, E. E. (1997). Temporal dynamics of brain activation during a working memory task. Nature 386, 604–608.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Cox, K. M., Aizenstein, H. J., and Fiez, J. A. (2008). Striatal outcome processing in healthy aging. Cogn. Affect. Behav. Neurosci. 8, 304–317.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Deakin, J., Aitken, M., Robbins, T., and Sahakian, B. J. (2004). Risk taking during decision-making in normal volunteers changes with age. J. Int. Neuropsychol. Soc. 10, 590–598.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Delaney, H. D., and Maxwell, S. E. (1981). On using analysis of covariance in repeated measures designs. Multivariate Behav. Res. 16, 105–123.

CrossRef Full Text

Denburg, N. L., Recknor, E. C., Bechara, A., and Tranel, D. (2006). Psychophysiological anticipation of positive outcomes promotes advantageous decision-making in normal older persons. Int. J. Psychophysiol. 61, 19–25.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Denburg, N. L., Tranel, D., and Bechara, A. (2005). The ability to decide advantageously declines prematurely in some normal older persons. Neuropsychologia 43, 1099–1106.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

D’Esposito, M., Detre, J. A., Alsop, D. C., Shin, R. K., Atlas, S., and Grossman, M. (1995). The neural basis of the central executive system of working-memory. Nature 378, 279–281.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Dreher, J. C., Meyer-Lindenberg, A., Kohn, P., and Berman, K. F. (2008). Age-related changes in midbrain dopaminergic regulation of the human reward system. Proc. Natl. Acad. Sci. U.S.A. 105, 15106–15111.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Düzel, E., Bunzeck, N., Guitart-Masip, M., and Düzel, S. (2010). Novelty-related motivation of anticipation and exploration by dopamine (NOMAD): implications for healthy aging. Neurosci. Biobehav. Rev. 34, 660–669.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fein, G., McGillivray, S., and Finn, P. (2007). Older adults make less advantageous decisions than younger adults: cognitive and psychological correlates. J. Int. Neuropsychol. Soc. 13, 480–489.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fiorillo, C. D. (2011). Transient activation of midbrain dopamine neurons by reward risk. Neuroscience 197, 162–171.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fiorillo, C. D., Tobler, P. N., and Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Folstein, M. F., Folstein, S. E., and McHugh, P. R. (1975). Mini-mental state – practical method for grading cognitive state of patients for clinician. J. Psychiatr. Res. 12, 189–198.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Frank, M. J., and Kong, L. (2008). Learning to avoid in older age. Psychol. Aging 23, 392–398.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Frank, M. J., Seeberger, L. C., and O’Reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in Parkinsonism. Science 306, 1940–1943.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Glimcher, P. W. (2008). Understanding risk: a guide for the perplexed. Cogn. Affect. Behav. Neurosci. 8, 348–354.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Grady, C. L., Maisog, J. M., Horwitz, B., Ungerleider, L. G., Mentis, M. J., Salerno, J. A., Pietrini, P., Wagner, E., and Haxby, J. V. (1994). Age-related changes in cortical blood flow activation during visual processing of faces and location. J. Neurosci. 14, 1450–1462.

Pubmed Abstract | Pubmed Full Text

Hämmerer, D., Li, S. C., Müller, V., and Lindenberger, U. (2011). Life span differences in electrophysiological correlates of monitoring gains and losses during probabilistic reinforcement learning. J. Cogn. Neurosci. 23, 579–592.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hasher, L., and Zacks, R. T. (1988). “Working memory, comprehension, and aging: a review and a new view,” in The Psychology of Learning and Motivation, ed. G. H. Bower (New York: Academic Press), 193–225.

Hau, R., Pleskac, T. J., and Hertwig, R. (2010). Decisions from experience and statistical probabilities: why they trigger different choices than a priori probabilities. J. Behav. Decis. Mak. 23, 48–68.

CrossRef Full Text

Heekeren, H. R., Wartenburger, I., Marschner, A., Mell, T., Villringer, A., and Reischies, F. M. (2007). Role of ventral striatum in reward-based decision making. Neuroreport 18, 951–955.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hertwig, R., Barron, G., Weber, E. U., and Erev, I. (2004). Decisions from experience and the effect of rare events in risky choice. Psychol. Sci. 15, 534–539.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hertwig, R., and Erev, I. (2009). The description-experience gap in risky choice. Trends Cogn. Sci. (Regul. Ed.) 13, 517–523.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hertwig, R., and Pleskac, T. J. (2010). Decisions from experience: why small samples? Cognition 115, 225–237.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hosseini, S. M. H., Rostami, M., Yomogida, Y., Takahashi, M., Tsukiura, T., and Kawashima, R. (2010). Aging and decision making under uncertainty: behavioral and neural evidence for the preservation of decision making in the absence of learning in old age. Neuroimage 52, 1514–1520.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kaasinen, V., Vilkman, H., Hietala, J., Nagren, K., Helenius, H., Olsson, H., Farde, L., and Rinne, J. O. (2000). Age-related dopamine D2/D3 receptor loss in extrastriatal regions of the human brain. Neurobiol. Aging 21, 683–688.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kahneman, D., and Tversky, A. (1979). Prospect theory – analysis of decision under risk. Econometrica 47, 263–291.

CrossRef Full Text

Kang, M. J., Hsu, M., Krajbich, I. M., Loewenstein, G., McClure, S. M., Wang, J. T. Y., and Camerer, C. F. (2009). The wick in the candle of learning: epistemic curiosity activates reward circuitry and enhances memory. Psychol. Sci. 20, 963–973.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Knutson, B., Fong, G. W., Adams, C. M., Varner, J. L., and Hommer, D. (2001). Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport 12, 3683–3687.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kovalchik, S., Camerer, C. F., Grether, D. M., Plott, C. R., and Allman, J. M. (2005). Aging and decision making: a comparison between neurologically healthy elderly and young individuals. J. Econ. Behav. Organ. 58, 79–94.

CrossRef Full Text

Lee, T. M. C., Leung, A. W. S., Fox, P. T., Gao, J. H., and Chan, C. C. H. (2008). Age-related differences in neural activities during risk taking as revealed by functional MRI. Soc. Cogn. Affect. Neurosci. 3, 7–15.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Li, S. C., Lindenberger, U., and Sikström, S. (2001). Aging cognition: from neuromodulation to representation. Trends Cogn. Sci. (Regul. Ed.) 5, 479–486.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lockenhoff, C. E., and Carstensen, L. L. (2007). Aging, emotion, and health-related decision strategies: motivational manipulations can reduce age differences. Psychol. Aging 22, 134–146.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mata, R., Josef, A. K., Samanez-Larkin, G. R., and Hertwig, R. (2011). Age differences in risky choice: a meta-analysis. Ann. N. Y. Acad. Sci. 1235, 18–29.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mata, R., and Nunes, L. (2010). When less is enough: cognitive aging, information search, and decision quality in consumer choice. Psychol. Aging 25, 289–298.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mather, M. (2006). “A review of decision making processes: weighing the risks and benefits of aging,” in When I’m 64, eds L. L. Carstensen and C. R. Hartel (Washington, DC: National Academies Press), 145–173.

Mather, M., Knight, M., and McCaffrey, M. (2005). The allure of the alignable: younger and older adults’ false memories of choice features. J. Exp. Psychol. Gen. 134, 38–51.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mell, T., Heekeren, H. R., Marschner, A., Wartenburger, I., Villringer, A., and Reischies, F. M. (2005). Effect of aging on stimulus-reward association learning. Neuropsychologia 43, 554–563.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mell, T., Wartenburger, I., Marschner, A., Villringer, A., Reischies, F. M., and Heekeren, H. R. (2009). Altered function of ventral striatum during reward-based decision making in old age. Front. Hum. Neurosci. 3:34. doi:10.3389/neuro.09.034.2009

CrossRef Full Text

Mohr, P. N., Li, S. C., and Heekeren, H. R. (2010). Neuroeconomics and aging: neuromodulation of economic decision making in old age. Neurosci. Biobehav. Rev. 34, 678–688.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Novak, D. L., and Mather, M. (2007). Aging and variety seeking. Psychol. Aging 22, 728–737.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Okun, M. A. (1976). Adult age and cautiousness in decision – review of literature. Hum. Dev. 19, 220–233.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Rakow, T., and Newell, B. R. (2010). Degrees of uncertainty: an overview and framework for future research on experience-based choice. J. Behav. Decis. Mak. 23, 1–14.

CrossRef Full Text

Rakow, T., Newell, B. R., and Zougkou, K. (2010). The role of working memory in information acquisition and decision making: lessons from the binary prediction task. Q. J. Exp. Psychol. 63, 1335–1360.

CrossRef Full Text

Raven, J. C. (1982). Revised Manual for Raven’s Progressive Matrices and Vocabulary Scale. Windsor: NFER Nelson.

Raz, N., Lindenberger, U., Rodrigue, K. M., Kennedy, K. M., Head, D., Williamson, A., Dahle, C., Gerstorf, D., and Acker, J. D. (2005). Regional brain changes in aging healthy adults: general trends, individual differences and modifiers. Cereb. Cortex 15, 1676–1689.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Reed, A. E., Mikels, J. A., and Simon, K. I. (2008). Older adults prefer less choice than young adults. Psychol. Aging 23, 671–675.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Reyna, V. F., and Brainerd, C. J. (2008). Numeracy, ratio bias, and denominator neglect in judgments of risk and probability. Learn. Individ. Differ. 18, 89–107.

CrossRef Full Text

Samanez-Larkin, G. R., Gibbs, S. E., Khanna, K., Nielsen, L., Carstensen, L. L., and Knutson, B. (2007). Anticipation of monetary gain but not loss in healthy older adults. Nat. Neurosci. 10, 787–791.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Samanez-Larkin, G. R., Kuhnen, C. M., Yoo, D. J., and Knutson, B. (2010). Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. J. Neurosci. 30, 1426–1434.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Samanez-Larkin, G. R., Wagner, A. D., and Knutson, B. (2011). Expected value information improves financial risk taking across the adult life span. Soc. Cogn. Affect. Neurosci. 6, 207–217.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schott, B. H., Niehaus, L., Wittmann, B. C., Schutze, H., Seidenbecher, C. I., Heinze, H. J., and Düzel, E. (2007). Ageing and early-stage Parkinson’s disease affect separable neural mechanisms of mesolimbic reward processing. Brain 130, 2412–2424.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schultz, W. (2000). Multiple reward signals in the brain. Nat. Rev. Neurosci. 1, 199–207.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Simon, J. R., Howard, J. H., and Howard, D. V. (2010). Adult age differences in learning from positive and negative probabilistic feedback. Neuropsychology 24, 534–541.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wang, Y., Chan, G. L. Y., Holden, J. E., Dobko, T., Mak, E., Schulzer, M., Huser, J. M., Snow, B. J., Ruth, T. J., Calne, D. B., and Stoessl, A. J. (1998). Age-dependent decline of dopamine D1 receptors in human brain: a PET study. Synapse 30, 56–61.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Weber, E. U., Shafir, S., and Blais, A. R. (2004). Predicting risk sensitivity in humans and lower animals: risk as variance or coefficient of variation. Psychol. Rev. 111, 430–445.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wood, S., Busemeyer, J., Koling, A., Cox, C. R., and Davis, H. (2005). Older adults as adaptive decision makers: evidence from the Iowa gambling task. Psychol. Aging 20, 220–225.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zamarian, L., Sinz, H., Bonatti, E., Gamboz, N., and Delazer, M. (2008). Normal aging affects decisions under ambiguity, but not decisions under risk. Neuropsychology 22, 645–657.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text