A Neuroeconomic Theory of Mental Time Travel

We propose a theoretical model that places attention at the center of mental time travel (MTT) ability. This theory predicts that attention promotes a memory-based process that encodes memories of unexpected events, facilitates accurate recollection of information of such events during MTT, and optimizes subsequent decision-making. This process coexists with a habitual process that governs all other events and treats them equally. Our theory demonstrates that the memory-based process is useful when the environment features novel experiences that are likely to be relevant in future decision-making, hence worth remembering accurately. By contrast, the habitual process is optimal in environments that either do not change significantly, or have a small chance of being repeated in the future. This may explain why the ability to mentally travel in time has developed differently in humans than in other species. Implications are discussed in the context of decision-making.

1. Introduction

According to the classic Tulving theory (Tulving, 1985a), memory has the distinctive mission of permitting humans to mentally travel backwards and forwards in time. Mental time travel (MTT) allows us to recall past events to avoid dangers and choose the best future courses of action. Central to MTT is episodic memory, which processes memories of personal events and experiences and makes it possible to use past experiences to simulate future events or alternative pasts. Episodic memory summarizes sensory, perceptual, and emotional information and features a visual representation component. It is part of the declarative memory system, which encodes precise information that can be verbalized, but also easily forgotten (Squire, 2004; Poldrack and Foerde, 2008)¹. The relationship between episodic memory components and MTT has been widely documented. From a developmental perspective, MTT develops as soon as children understand the concept of “yesterday” and “tomorrow,” and in conjunction with the development of episodic memory (Suddendorf and Busby, 2005; Hayne and Imuta, 2011). The MTT ability is also less developed in subjects with a poor visual imaginary or those who tend to suppress emotions (D'Argembeau and Van der Linden, 2006).

The neural correlates of episodic memory are well-understood. Episodic memory involves the medial temporal lobe (MTL), which hosts the hippocampus. Dysfunction of the latter (amnesia) has been associated with either no memory of personal events or a difficulty in forming new memories about personal events further associated with no ability to project oneself in the future (Tulving, 1985b; Klein et al., 2002; Hassabis et al., 2007). The MTL is also implicated in episodic future thinking –or the construct of future possible scenarii– and episodic counterfactual thinking –or the simulation of alternative pasts– (Schacter et al., 2015). Interestingly, episodic memory has been shown to interact with working memory (Balconi, 2013). In particular, increased activation of the dorsolateral prefrontal cortex (dlPFC) during the formation of episodic memories results in their enhancement. Working memory also plays a role in the episodic buffer, which allows the use of old memories to form new ones. The dlPFC has been shown to have a causal role in episodic memory formation and to be responsible for suppressing memories through suppressing hippocampal processing (Benoit et al., 2014). These findings taken together indicate that a neural circuit involving hippocampal regions and working memory regions are at the core of episodic memory. As such, we conjecture that an interplay between these structures affects the selection of memories worth being recalled in the future.

MTT is often believed to be uniquely human and to have been shaped through evolution (Suddendorf et al., 2009). However, episodic memory capacity depends on a fundamental neural circuit that is similar across mammalian and avian species, which suggests a shared underlying neural ancestry rather than a specific human evolutionary trajectory (Allen and Fortin, 2013). It is therefore still unclear to which extent this capacity supports MTT in animals (Suddendorf and Busby, 2003; Suddendorf and Corballis, 2007). Working memory capacity is also believed to have evolved differentially across species, and humans are thought to be unique in some of the uses they make of it (Carruthers, 2013). These differences in both MTT and working memory capacities between humans and non-humans may be related. Fitness depends on the environments that species encounter, and some environments require specific abilities to adapt. In theory, an environment requiring better prospective and processing capabilities may have triggered the evolution of modern human working memory and MTT abilities. Hence, the specific interplay between episodic memory and working memory might have been shaped by evolution differentially across species to adapt to different environments.

These observations taken together suggest that MTT ability is supported by episodic memory, which is involved in an interplay with the working memory system to encode memories of relevance as a function of the environment. Said differently, observations are compatible with the existence of a goal-directed (top down) memory management mechanism that allocates attentional resources to form memories with the objective of maximizing future rewards. In this article, we consider a stochastic environment that produces events relevant for decision-making. These can be memorized and later recalled to make decisions. We build a theoretical framework to identify the most efficient interplay between a system capable of forming memories at a cost (episodic memory) and a goal-directed mechanism that allocates attentional resources in order to best serve future decision-making in a given environment.

To identify such an efficient mechanism, we adopt an optimization approach. This approach is widely used in decision-making fields such as Economics, and has been applied to model behavior driven by interactions between brain systems (Bernheim and Rangel, 2004; Brocas and Carrillo, 2008, 2014). This approach consists of endowing brain systems (e.g., those involved in goal-directed attention) with the ability to optimize their behavior (e.g., allocate attention) conditional on the stimulus they receive (e.g., features of an event) and the goal to fulfill (e.g., the relevance of the event in future decisions)². This approach captures two critical features of brain processing. First, it recognizes brain modularity and the fact that each system is handling a specific task. Second, it envisions systems as being tuned to operate efficiently to interpret the information they receive and to formulate a response.

2. Remembering the Past to Predict the Future

2.1. The Basic Model

We consider a mathematical formulation of MTT, which extends and complements the simple two-memory system representation introduced by Brocas and Carrillo (2016). We use a deliberately simplistic car-parking example to illustrate this theory. First, an event occurs. It corresponds to the location in the parking lot where the individual (from now on “he") has parked in the morning. We denote this event by x. The event can be conceptualized as a draw from the random variable X which, in our example, is simply the distribution of parking spots that the person has been using in the past. We assume that X follows a normal distribution:

$X~{N}\left(\theta ,\frac{1}{p}\right).$

This means that the person usually parks around the spot θ. The parameter p, the precision (or inverse of the variance) of the random variable X, captures how much the draw varies from day-to-day. In our example, it represents how congested the parking lot is, and therefore how likely it is to find a spot near θ. By abuse of language we will refer to θ, the average of the distribution, as a “typical” realization of the event.

The individual needs to form a memory about the event. He recruits the episodic memory for this purpose. In our example, the memory represents the individual's recollection of the spot where he parked his car in the morning. To model imperfect recollection of events, we assume that when the individual travels back in time at a later date, his memory m of the true event x is given by:

$m=x+uwhereu~{N}\left(0,\frac{1}{e}\right).(1)$

The memory is a distorted representation of the true event. The size of the distortion is inversely related to the amount of attentional resources e invested when forming the memory. More precisely, the distortion takes the form of a noise u that follows a normal distribution with mean 0 and precision e. In expectation, the memory is correct, that is, there is no systematic bias in the distortion: E(m) = x. Under infinite attention (e = +∞), the individual will perfectly recall the state (m = x with certainty), and with no attention (e = 0), the recollection is infinitely vague. Given attentional resources are scarce, there is an opportunity cost of allocating them to this task. We assume that the cost of attention is increasing and convex, formally represented by a quadratic cost function c(e) = e²/2. Thus, the main difference relative to Brocas and Carrillo (2016) is that, in the present paper, the individual chooses between a continuum of possible levels of attention (e ∈ [0, +∞)) rather than only two levels (e₁ and e₂ with e₂ > e₁ > 0).

The recollected memory m is used to plan an action a. In the context of our example, the action of the individual is where to look for his car when it is time to go home. The individual obtains a payoff inversely related to the distance between the true location and the location where the individual looks. Again for simplicity, we model this payoff with a standard quadratic utility loss function:

$-l{(a-x)}^2$

with l > 0. According to this formulation, if the event (true location) x is recalled with exactitude (m = x), the individual's optimal action is:

$\tilde{a}(x)=\text{arg }\underset{a}{\text{max}}-l{(a-x)}^2\Rightarrow \tilde{a}(x)=x.$

This simply means that it is optimal to look for the car wherever it is located. Deviations from ã(x) –looking for the car in a different spot– implies a loss (e.g., a time delay or a longer walk) which increases with the absolute distance |a − x| and the sensitivity of the individual to losses (l). The timeline of the decision process is summarized in Figure 1.

FIGURE 1

Figure 1. Timing of the decision making problem.

This individual decision problem has standard multi-stage features. It is formally solved by backward induction.

2.2. Optimal Action Given Available Memory

If a finite amount of attention e has been allocated to the task, the true location will not be recalled with exactitude. Instead an (imperfect) memory m will be retrieved. Given that memory, the action that maximizes the expected payoff is:

$a^{*}(m,e)=\text{arg }\underset{a}{\text{max}}-{\displaystyle {\int }_{x}l}{(a-x)}^{2}d{F}_e(x|m)$

where F_e(x|m) is the revised cumulative distribution function of the event x given the attention exerted (e), and the memory (m). Given a quadratic objective function, the optimal decision satisfies the first-order condition and is given by:

$a^{*}(m,e)=E_e[X|m],(2)$

where E_e is the expectation operator. The optimal action coincides with the expected belief about the event given the memory retrieved. The normality assumption of the variables X and u implies that:

$m|x~{N}\left(x,\frac{1}{e}\right)\text{and}X|m~{N}\left(\frac{p}{e+p}\theta +\frac{e}{e+p}m,\frac{1}{e+p}\right)(3)$

According to (3), the memory is centered around the true event x but with a distortion that depends inversely on the attentional effort e. Once a memory is retrieved, the updated belief about the event (X|m) is a random variable. Its expected value is a convex combination of the prior θ and the memory m. Combining Equations (2, 3), we can rewrite the optimal action as:

$a^{*}(m,e)=\frac{p}{e+p}\theta +\frac{e}{e+p}m(4)$

Given limited attention, memories are known to be imperfect and therefore cannot be fully trusted. Hence, in our example, it is optimal for the individual to look for his car not where his memory tells him to (m) but, instead, somewhere between the typical location and where his memory tells him to (between θ and m). If more attention has been dedicated to memorize the event (e high), the memory is more reliable, so the action is closer to the memory. The action is also closer to the memory when the dispersion in choices is higher (p low) since, in that case, the knowledge about the typical location is less valuable. This is summarized as follows (proofs of all Propositions can be found in the Supplementary Material section).

Proposition 1. Suboptimal actions are the result of imperfect memories and they are modulated by attention. Individuals can trust their memories more if they are highly attentive (e high) and if their typical behavior is less reliable (p low).

2.3. Optimal Attention Anticipating the Future Action

We now use backward induction to determine the optimal attention e* in stage 2 given the anticipated optimal action a* to be exerted in stage 4 as determined in Equation (4) (see Figure 1).

After observing the event x, allocating attention e affects the distribution from which memories are drawn, which we denote by G_e(m|x). Given a memory m, the action will subsequently be $a^{*}(m,e)={\mathrm{\text{E}}}_e\left[X|m\right]$. Therefore, the expected payoff of allocating attention e, net of opportunity costs, is:

$\begin{array}{l}{V}_x(e)=-{{\displaystyle \int }}_{m}l{(a^{*}\left(m,e\right)-x)}^{2}d{G}_e(m|x)-\frac{e^2}{2}\hfill \\ =-l\left(\frac{e+p^2{(x-\theta )}^2}{{(e+p)}^2}\right)-\frac{e^2}{2}.\hfill \end{array}$

It is optimal to allocate resources that achieve the highest expected payoff, namely:

$e^{*}=\text{arg }\underset{e}{\text{max}}{V}_x(e),$

which leads to the following conclusion:

Proposition 2. There exists a range of events [x, x] containing the typical event (x < θ < x) such that it is optimal not to exert any attention (e* = 0) when the realization of the event falls in that range. Some level of attention is optimal (e* > 0) when the realization falls outside that range (x ∉ [x, x]). In that case, attention increases as the event moves away from the typical realization.

The idea behind this result is simple. The event is a stochastic variable centered around θ. When the current realization of the event is sufficiently close to this value (|x − θ| small), it is not worth remembering. Instead it is a better strategy to save on attentional resources, act as if the location is θ and incur a moderate loss −l(x − θ)². This case corresponds to a form of “habitual thinking,” that is a thinking process based on general prior information rather than accurate information. By contrast, when the realization is sufficiently far from it (|x − θ| large), it is worth exerting attention to remember the event with precision and avoid an action excessively far from the optimal one. In our example, if the individual has parked the car near the typical spot θ, it is worth not paying any attention and looking in that spot first, but if the individual has parked very far from it, then it is worth remembering the location with some accuracy.

Overall, when x is observed, the individual exerts an optimal level of attention e* and the average action he undertakes is given by:

$E[a^{*}|x]=\{\begin{array}{ll}\theta & \text{if }x\in [\underset{\_}{x},\overline{x}]\\ \frac{p}{e^{*}+p}\theta +\frac{e^{*}}{e^{*}+p}x& \text{if }x\notin [\underset{\_}{x},\overline{x}]& (5)\end{array}$

The optimal solution of the problem is summarized in Figure 2. The individual exerts more attention when the realized event is far from typical (Figure 2A) resulting in an action that is more congruent with the event (Figure 2B). Events that are close to typical (x ∈ [x, x]) do not require any attention to be memorized and they are followed by typical actions (θ).

FIGURE 2

Figure 2. Optimal mechanism. (A) No attention is allocated (e* = 0) when the event is close to typical (x close to θ) and attention increases as the event becomes more unexpected (x far from θ). (B) In the absence of memory limitations, actions should follow the 45 degree line. Under memory limitations, actions are based on prior knowledge (a* = θ) when no attention has been allocated (x ∈ [x, x]) and it is based on the memory signal when attention has been allocated. As events are more extreme, memory signals are less distorted, and actions are closer to optimal.

2.4. Modulating Attention and Choice

Attention and actions are modulated by features of the environment. We can perform comparative statics to measure how changes in specific features of the problem affect the amount of attention exerted and the subsequent action undertaken.

2.4.1. The Effect of Stakes

Stakes are captured in our setting with the parameter l, that measures the sensitivity to losses. Not surprisingly, the optimal level of attention is increasing in the sensitivity to losses. If the individual expects to incur a large disutility of not finding his car quickly, he should allocate more attention to the formation of a reliable memory. This in turn impacts the decision. With a higher sensitivity to losses comes greater attention, and therefore more reliable memories. These memories are trusted more, which is reflected in actions closer to the memory and farther away from the typical event. We represent this effect in Figure 3.

FIGURE 3

Figure 3. Effect of stake. Optimal attention (A) and decisions (B) under low stakes (full line) and high stakes (dashed lines). As stakes increase, attention increases and actions are on average closer to the event. Also, the range of events for which no attention is exerted shrinks.

Note that l can equally capture the likelihood that the information contained in the event will be useful in the future. Under this interpretation, our theory predicts that it is efficient to invest more attention when this likelihood increases.

Proposition 3. As stakes increase, attention increases, making memories more precise and reliable. As a result, actions rely more heavily on the memory and less on the typical event.

2.4.2. The Effect of the Environment

We can compute the expected payoff of the individual before the realization of the event and anticipating that he will allocate an optimal amount of attention. It is given by:

$\begin{array}{l}EV={\displaystyle {\int }_X-}l\left(\frac{e^{*}+p^2{(X-\theta )}^2}{{(e^{*}+p)}^2}\right)dF(X)-\frac{{(e^{*})}^2}{2}\\ =-l\left(\frac{1}{e^{*}+p}\right)-\frac{{(e^{*})}^2}{2}.\end{array}$

In terms of our example, it represents the expected utility of the individual when he is going to work and has not yet parked his car. Interestingly, the individual is better-off when the distribution from which the event occurs is concentrated around its mean (high p) or, in our example, when he frequently finds a parking spot close to the typical location. This means that, even though the individual optimally reacts to extreme events by exerting more attention, he still benefits when such events do not occur frequently. An environment that changes little offers a guarantee to obtain safe rewards at minimal cost.

Proposition 4. Environments in which events do not vary much (high p) are conducive of habitual thinking and yield high expected rewards.

Notice that by choosing attention over a continuum of options, we can determine the marginal effect of stakes and events on the optimal level of attention. Perhaps more importantly, this model delivers new results relative to Brocas and Carrillo (2016). In particular, we can show that no attention is sometimes optimal and that the individual is better-off when the environment does not change significantly. These results have behavioral and neurophysiological implications that are discussed in the next section.

3. Implications of the Theory

According to our theory, decision-making that requires knowledge of previous events is optimized through an interplay between the episodic memory system (with capabilities to form memories) and the attentional system (with capabilities to enhance memories). This interplay is orchestrated by goal-directed mechanisms that select relevant memories and optimize their collection to best serve decision-making. This overall process makes it possible to recall past events with more or less accuracy, and to simulate future rewards informed by past memories. Importantly, our model also shows that memories and behavior resulting from this process are modulated by a series of parameters describing the environment in which decisions are made.

3.1. Dual Process Theories

The central result of the theoretical section is the existence of two regimes, one in which events are not memorized and decisions are based on prior information and another in which events are memorized and (imperfectly) recollected to optimize decision-making. As such, our theory provides support for the coexistence of a habitual process (which operates in the limit, when attentional resources are too difficult to be recruited or not worth the cost) and a memory-based process (which is modulated by the amount of attentional resources allocated to the task). These correspond to events that fall inside and outside the range [x, x], respectively (see Figure 2).

The habitual process is optimal when events are close to typical (x close to θ), and when decisions are not very important (l small). It is an efficient process associated with simple strategies that rely on prior information about the environment. However, whenever stakes increase or events are rare, the costly memory-based process should be activated to help integrate the important specific features into decisions. This theoretical finding is consistent with the existing literature. Indeed, behavioral studies have suggested that we often use heuristics that rely on limited information about past events (Bröder and Schiffer, 2003, 2006). Recent neuroimaging analyses have also shown that processing more precise information requires a memory-based process that taxes regions involved in working memory, such as dlPFC (Khader et al., 2011, 2016). Our theory rationalizes these findings and identifies the conditions under which heuristic-based decision-making (which does not require precise memory or the ability to simulate the future based on the past) is efficient and when it should be replaced by a costly memory-based process. Moreover, the theory predicts that structures involved in working memory should be recruited when events are far from typical (or unexpected), which is consistent with evidence implicating the dlPFC in these types of events (Kapur et al., 1997; Fletcher et al., 2001).

3.2. Dysfunctions

Our theory can be extended to address the behavioral implications of dysfunctions of the two main systems involved in memory formation, namely working memory and episodic memory.

3.2.1. Working Memory Dysfunction

Critical to our results is the working memory system that makes it possible to form and recall episodic memories of relevance for decision making. In the context of our model, the disorder can be captured by rescaling the cost function. More precisely, the cost of attention can vary across individuals and be given by c(e) = αe²/2 where α represents the idiosyncratic cost of a unit of attention (in the basic model, α = 1). This modification affects the cost-benefit trade-off reported in section 2.4. Using the same methodology as in section 2, we notice that the optimal allocation of attention is decreasing in α (see the Appendix in Supplementary Material for the formal derivation). Said differently, this theory predicts that dysfunctions of the working memory system result in the allocation of fewer resources to remember events. As the cost of attention increases, the region [x, x] becomes larger. This, in turn, implies that the habitual process becomes more prevalent and decisions are more often based on the typical realization of events rather than on accurate information. Interestingly, working memory disorders (such as attention deficit hyperactivity disorders) have been associated with episodic memory deficits in particular in complex memory tasks (Felton et al., 1987; Quinlan and Brown, 2003). Working memory disorders generally relate to the inability to allocate attention to the task at hand or for the duration required to complete it. Individuals act as if allocating attentional resources is extremely costly and decision-making trade-offs turn to favor limited attention. This description is consistent with the predictions of the model: when working memory becomes taxing, it is optimal to rely on habitual processes.

3.2.2. Episodic Memory Dysfunction

Mental time travel abilities have been shown to not work properly in amnesic patients, who can neither remember episodic memories nor simulate future events (Klein et al., 2002). Amnesia refers to the inability to encode memories and is an extreme case of an episodic memory dysfunction. Such dysfunction can be accommodated in our model by assuming that after exerting an amount of attention e, the memory is actually encoded with probability q and it is not encoded with probability 1 − q. Our basic model would correspond to q = 1, whereas the case of an amnesic patient incapable of forming any memory would correspond to q = 0. This limited ability to form memories affects the optimal allocation of attention as well as subsequent decisions. Indeed, individuals who form memories only very rarely (low q) should exercise less attention because the latter is likely to be wasted. Said differently, the theory predicts that dysfunctions of the episodic memory system result in allocating fewer resources to remembering events. As the probability of actually encoding memories decreases, the region [x, x] becomes larger. Dysfunctions of episodic memories also yield choices closer to heuristic decision-making. This, as in the case of a working memory dysfunction, translates into decisions that rely heavily on the habitual process. In the limit case in which no memory can be formed (q = 0), the individual is not able to simulate past events to make decisions and relies exclusively on the habitual system ([x, x] takes the entire support).

In sum, dysfunctions of the working memory system and the episodic memory system will result in an increased difficulty to form new memories and simulate future events based on these memories. It will also imply an increased tendency to resort to the habitual memory system. Conditional on the dysfunction, this is the best response of the individual to his environment. However, from the perspective of an outside observer, behavior will appear to not reflect the true information contained in the events and to rely on inadequate heuristic rules. This prediction is consistent with the evidence reported in the case of clinical populations, in particular for patients suffering from amnesia (Klein et al., 2002), stress disorders (Brown et al., 2014), schizophrenia (Aleman et al., 1999) and other related dysfunctions of hippocampal regions. Similar memory disorders are also observed as a consequence of normal aging (Friedman, 2013) or neurodegenerative diseases such as Alzheimer's, Parkinson's and Huntington's diseases (Panegyres, 2004), that gradually cause episodic memory dysfunction or executive dysfunctions that affect working memory.

3.3. Value-Based Decision-Making

Recent literature in neuroscience has dedicated much attention to value-based decision-making. These decisions involve the choice between options of different values that need to be constructed and compared. This paradigm is used to describe neural correlates of economic choices between items in situations involving time delays, uncertainty or risk. The evidence supports the existence of a common reward system (Levy and Glimcher, 2012). It also shows that decisions may be modulated by activity in regions involved in working memory (such as the dlPFC) in situations requiring more detailed or higher-order information, such as food choices subject to self-control problems (Hare et al., 2009). It is plausible that this activity is related to the retrieval of information encoded in the past and relevant to the current decision. Recent research focuses on the interplay between episodic memory and value based decision-making (Weilbächer and Gluth, 2016), but our understanding of these relationships is still limited. Our theory provides a framework to understand how value is formed at the date of decision as a function of how prior information is encoded. We illustrate these concepts with two examples.

3.3.1. Self-Control in the Food Domain

Let us consider the standard paradigm used in the literature on value and self-control. Suppose that X represents the health characteristic of food items. The individual tries food item x and learns its health characteristic. This information may be used in a future consumption episode. Knowing the exact characteristic of the food yields optimal consumption decisions. Our model predicts that this information will not be memorized if it is close to typical. The habitual process will feed a habitual value formation and a habitual behavior, while a memory-based process will help the individual choose the behavior that best fits his needs. Furthermore, our theory also implies that the item will be treated as healthier than it really is if x < θ, and therefore is over-consumed. Similarly, the item will be treated as less healthy than it really is if x > θ, and therefore is under-consumed. Overall and as illustrated in Figure 4A, our theory argues that the efficient memory management process is at the core of biases in behavior.

FIGURE 4

Figure 4. Value-based decisions. (A) Items that are less healthy (healthier) than typical will be imperfectly memorized when they are unexpected and not memorized if they are close to typical. In both cases, they are over-consumed (under-consumed) in future episodes. (B) Due to imperfect memory, individuals over-invest (under-invest) in lottery tickets they have purchased in the past and turned out to be less (more) profitable than typical.

These biases will be exacerbated if the individual is impulsive and heavily discounts the future (l is small) or has a working memory deficit. Interestingly, associations between impulsivity, low discounting and disrupted episodic future thinking have been observed among both healthy and obese subjects (Daniel et al., 2013; Bromberg et al., 2015). Our model is consistent with these findings and suggests some mechanisms. A first possible cause might be an inherent attention deficit that prevents attentional resources to be allocated to the formation of memories. The episodic memory system itself may be intact, but not triggered when necessary. This hypothesis is appealing in the case of eating behavioral disorders, since research in this area has documented disruptions of the working memory system and in particular the dlPFC (Seymour et al., 2015). Another mechanism involves an impaired episodic system that does not form efficient memories and makes the allocation of attentional resources wasteful. Interestingly, eating disorders are also associated with a dysfunction of the mesolimbic regions implicated in processing rewards (Avena and Bocarsly, 2012) and these regions are critical for hippocampal memory formation (Wittmann et al., 2008).

3.3.2. Risk Perception

Suppose that X represents the expected return of a risky investment (e.g., a lottery ticket). In a given event, the individual purchases a ticket x and learns whether it was a good investment. This information may be used in the future to determine whether to purchase it again or not. Our model predicts that lottery tickets associated with returns close to those of typical tickets will be treated equally. In particular, within that region, lottery tickets that have smaller expected returns than typical will be purchased too often (see Figure 4B). Individuals will behave as if they take irrational risks or overestimate their chances of winning. Again, disruptions of the attentional system will exacerbate these inefficiencies. This idea is consistent with studies showing that pathologic gamblers exhibit disruption of this system (Fujimoto et al., 2017). Alternatively, dysfunction of the episodic memory, which is known to be sensitive to reward outcomes, may affect the formation of memories and result in inefficient behavior (Mason et al., 2017).

These results allow us to rationalize both the observed behavior and the neural correlates of behavior that have been reported in the literature. First, the observed interactions between episodic memory regions and attention related regions are a logical response to the need of optimizing memories in order to support decision-making. Second, the behavior resulting from these interactions is efficient. Biases in behavior are the by-product of attentional limitations imposed on information processing.

3.4. Time Perception Biases

The literature on time perception has reported numerous biases on the perception of objective time (Wearden and Lejeune, 2008; Grondin, 2010). For instance, individuals who are told in advance that they will have to make a time judgment later recall time differently than individuals who are not (Block and Zakay, 1997). Time perception can also be manipulated through emotional interventions, such as emotional sounds or pictures, which are likely to disrupt the encoding of relevant time-keeping information (Droit-Volet and Meck, 2007; Fayolle et al., 2015). Indeed, evidence suggests that emotions modulate arousal and attention causing variations in the subjective perception of time (Coull et al., 2004; Droit-Volet and Gil, 2009).

If we reinterpret x as the time at which the subject arrived at work this morning and a as the report he makes when asked later in the day (θ being his typical arrival time), our model predicts that these reports will vary as a function of the environment. If the individual anticipates that he will need to make a time judgment later and that his accuracy is important (high l), the level of attention exerted to memorize the true time will be high, resulting in an accurate memory and a report close to that memory. If he does not anticipate it, he will behave as if the event is not linked to a future reward (low l). He will then exert little or no attention and, as a consequence, will report an estimate at or around his usual arrival time. This suggests that the processes involved in retrospective and prospective time paradigms differ and are bound to produce different reports, reflecting implicit manipulations of reward functions.

Along the same lines, if the individual is distracted, overloaded or subject to emotions that impact the amount of attention he can allocate to the task, his behavior can be formally modeled in a similar way to a working memory dysfunction. Our theoretical prediction is that such an individual will be more likely to resort to the habitual system, and will exhibit a less accurate perception of time. Distractions and emotional manipulations will divert attentional resources and prevent the individual from forming correct memories of time. This is again consistent with evidence of increased inaccuracy in the case of mental workload and suggests that attention is central to temporal experience (Brown and Boltz, 2002).

3.5. A Theory of Evolution

If memories are relevant to future choices, then the role of memory is to reconstruct past events to optimize future decisions and collect future rewards. However, if these operations require costly attention, not all events should be memorized and reconstructed equally. Events that are close to typical should not be encoded, since the prior knowledge of the environment is sufficient to guide future decisions with reasonable precision. As events become more striking, different from usual and/or contain important information for future decision-making, it is more important to remember them with accuracy. Overall, the relevance of memories to future rewards should determine the quality of those memories. This strongly suggests that the role of memory and MTT are context dependent. From our analysis, two basic features of the environment are relevant for MTT abilities. First, MTT is necessary when the environment is changing, featuring novel episodes constantly (in our model, large shifts in x relative to θ). This suggests that we will resort to costly attention mechanisms to remember rare experiences, but not everyday casual episodes. Hence, we will display the ability to travel back in time to precisely recall these rare experiences. Second, MTT is also necessary when the environment is likely to be relevant in the future (high l). Costly attention mechanisms will be recruited for remembering episodes that we would like to repeat in the future, but not events that have no chance of occurring again.

The results obtained with our optimization approach are consistent with evolutionary explanations of the unique time travel human ability (Suddendorf et al., 2009). Given evolution selects features that allow species to adapt to their environment, different evolution patterns should emerge across species as a function of differences in their respective environments. Species that are subject to an environment where the set of events is limited can efficiently operate based on a fixed or heuristic rule, that corresponds to acting as if events were always typical. These decisions do not require goal-directed attention. In such environments, there is no specific need for remembering events, and therefore no specific role for episodic memory or an interplay between episodic memory and attentional processes. This of course does not mean that episodic memory or attention systems have not developed in such species. It simply suggests that they did not need to develop for the purpose of using past events to optimize future decision-making, or that they did not need to develop to the extent humans did.

4. Discussion

The objective of our study was to identify the most efficient interplay between a system capable of forming memories (episodic memory) and a goal-directed mechanism that allocates attentional resources in order to best serve future decision-making. We have shown that the optimal way to manage memory formation entails an attention-free habitual system that categorizes events as typical when they are close to what is expected, and a costly memory-based mechanism that optimizes the future recollection of events when they are rare or unexpected. The latter relies on an interplay between brain structures involved in attention and episodic memory regions. The habitual mechanism generates heuristic rules always resulting in the same decision, while the memory-based process allows decisions to be modulated by true events, though imperfectly. This dual system permits an alignment of future decisions with past information in the most efficient way but it results in inefficiencies that depend on the environment. Inefficiencies are larger when events are not predicted to be useful in the future or when relatively rare events are treated as typical.

The conclusion that extreme outcomes are remembered more vividly than typical outcomes (Proposition 2 and Figure 2) relies on the assumption that subjects consciously optimize attention when forming memories. If, in a certain situation, an absent-minded person does not allocate attention as a function of the event, the result would be the opposite, namely a more reliable memory of typical than non-typical outcomes. In our motivating example, such person would be unable to locate the car whenever he parks it in an unusual spot.

Interestingly, and as discussed in section 3, the predictions of the model rationalize existing evidence in terms of both behavior and neural correlates. This has two implications. First, our study supports the idea that the brain processes information efficiently. The existence of different channels (e.g., cognitive or habitual) giving rise to different behavior (e.g., rational or heuristic) relies on a fundamental trade-off between the costs and benefits of information processing. A costly system is triggered when it is worth engaging it. In the context of MTT, our theory suggests that MTT is an invaluable ability when the environment produces events worth remembering. Second, our theory provides a conceptual framework to understand the role of attention in the formation of episodic memories as a function of the environment in which memories are formed. This suggests that the theory can be used to formulate hypotheses in new experimental paradigms. For example, one could design simple memory paradigms that would include a preliminary description of the distribution of experimental events, and would then ask participants to memorize individual events and to later recall them. The main theoretical predictions (Propositions 1 and 2) could be tested by comparing accuracy between expected and unexpected events and by contrasting neural activity across these two types of events. Attentional manipulations could further help assess the causality between attention and accuracy. Moreover, our theory suggests that stakes play a critical role in the formation of memories (Proposition 3). We conjecture that variations in stakes should be associated with differences in neural responses in both episodic memory and working memory regions. This hypothesis could be tested by designing incentivized memory experiments and varying the magnitude of the rewards for remembering accurately. Differences in accuracy should be correlated with differences in activation within regions involved in attention. Last, by comparing the average payoffs of subjects asked to recall similar or different pieces of information over the course of an experiment and by contrasting the patterns of neural activity within attention regions, one could assess whether habitual thinking is more prevalent and results in higher payoffs in environments in which events are more recurrent (Proposition 4). More generally, the model outlined here may be the starting point for a more general, testable neuroeconomic theory of decision-making with imperfect recall.

Author Contributions

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

Funding

This work has received the support of the National Science Foundation, SES-1425062.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2018.00658/full#supplementary-material

Footnotes

1. ^It complements the procedural system that encodes information about skills, an information which is durable but not easy to verbalize.

2. ^ It is obviously an “as if” approach: even though systems do not intentionally optimize their responses, they are tuned to produce responses that mimic the results of an optimization.

References

Aleman, A., Hijman, R., de Haan, E. H., and Kahn, R. S. (1999). Memory impairment in schizophrenia: a meta-analysis. Am. J. Psychiatry 156, 1358–1366.

PubMed Abstract | Google Scholar

Allen, T. A., and Fortin, N. J. (2013). The evolution of episodic memory. Proc. Natl. Acad. Sci. U.S.A. 110(Suppl. 2), 10379–10386.

PubMed Abstract | Google Scholar

Avena, N. M., and Bocarsly, M. E. (2012). Dysregulation of brain reward systems in eating disorders: neurochemical information from animal models of binge eating, bulimia nervosa, and anorexia nervosa. Neuropharmacology 63, 87–96. doi: 10.1016/j.neuropharm.2011.11.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Balconi, M. (2013). Dorsolateral prefrontal cortex, working memory and episodic memory processes: insight through transcranial magnetic stimulation techniques. Neurosci. Bull. 29, 381–389. doi: 10.1007/s12264-013-1309-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Benoit, R. G., Hulbert, J. C., Huddleston, E., and Anderson, M. C. (2014). Adaptive top–down suppression of hippocampal activity and the purging of intrusive memories from consciousness. J. Cogn. Neurosci. 27, 96–111. doi: 10.1162/jocn_a_00696

PubMed Abstract | CrossRef Full Text | Google Scholar

Bernheim, B. D., and Rangel, A. (2004). Addiction and cue-triggered decision processes. Am. Econ. Rev. 94, 1558–1590. doi: 10.1257/0002828043052222

PubMed Abstract | CrossRef Full Text | Google Scholar

Block, R. A., and Zakay, D. (1997). Prospective and retrospective duration judgments: a meta-analytic review. Psychon. Bull. Rev. 4, 184–197.

PubMed Abstract | Google Scholar

Brocas, I., and Carrillo, J. D. (2008). The brain as a hierarchical organization. Am. Econ. Rev. 98, 1312–1346. doi: 10.1257/aer.98.4.1312

CrossRef Full Text | Google Scholar

Brocas, I., and Carrillo, J. D. (2014). Dual-process theories of decision-making: a selective survey. J. Econ. Psychol. 41, 45–54. doi: 10.1016/j.joep.2013.01.004

CrossRef Full Text | Google Scholar

Brocas, I., and Carrillo, J. D. (2016). A neuroeconomic theory of memory retrieval. J. Econ. Behav. Organ. 130, 198–205. doi: 10.1016/j.jebo.2016.07.009

CrossRef Full Text | Google Scholar

Bröder, A., and Schiffer, S. (2003). Take the best versus simultaneous feature matching: probabilistic inferences from memory and effects of reprensentation format. J. Exp. Psychol. Gen. 132, 277–293.

PubMed Abstract | Google Scholar

Bröder, A., and Schiffer, S. (2006). Stimulus format and working memory in fast and frugal strategy selection. J. Behav. Decis. Making 19, 361–380.

Google Scholar

Bromberg, U., Wiehler, A., and Peters, J. (2015). Episodic future thinking is related to impulsive decision making in healthy adolescents. Child Dev. 86, 1458–1468. doi: 10.1111/cdev.12390

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, A. D., Addis, D. R., Romano, T. A., Marmar, C. R., Bryant, R. A., Hirst, W., et al. (2014). Episodic and semantic components of autobiographical memories and imagined future events in post-traumatic stress disorder. Memory 22, 595–604. doi: 10.1080/09658211.2013.807842

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, S. W., and Boltz, M. G. (2002). Attentional processes in time perception: effects of mental workload and event structure. J. Exp. Psychol. Hum. Percept. Perform. 28, 600–615. doi: 10.1037/0096-1523.28.3.600

PubMed Abstract | CrossRef Full Text | Google Scholar

Carruthers, P. (2013). Evolution of working memory. Proc. Natl. Acad. Sci. U.S.A. 110(Suppl. 2), 10371–10378. doi: 10.1073/pnas.1301195110

PubMed Abstract | CrossRef Full Text | Google Scholar

Coull, J. T., Vidal, F., Nazarian, B., and Macar, F. (2004). Functional anatomy of the attentional modulation of time estimation. Science 303, 1506–1508. doi: 10.1126/science.1091573

PubMed Abstract | CrossRef Full Text | Google Scholar

Daniel, T. O., Stanton, C. M., and Epstein, L. H. (2013). The future is now: reducing impulsivity and energy intake using episodic future thinking. Psychol. Sci. 24, 2339–2342. doi: 10.1177/0956797613488780

PubMed Abstract | CrossRef Full Text | Google Scholar

D'Argembeau, A., and Van der Linden, M. (2006). Individual differences in the phenomenology of mental time travel: the effect of vivid visual imagery and emotion regulation strategies. Conscious. Cogn. 15, 342–350. doi: 10.1016/j.concog.2005.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Droit-Volet, S., and Gil, S. (2009). The time–emotion paradox. Philos. Trans. R. Soc. Lond. B Biol. Sci. 64, 1943–1953. doi: 10.1098/rstb.2009.0013

CrossRef Full Text | Google Scholar

Droit-Volet, S., and Meck, W. H. (2007). How emotions colour our perception of time. Trends Cogn. Sci. 11, 504–513. doi: 10.1016/j.tics.2007.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Fayolle, S., Gil, S., and Droit-Volet, S. (2015). Fear and time: fear speeds up the internal clock. Behav. Process. 120, 135–140. doi: 10.1016/j.beproc.2015.09.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Felton, R. H., Wood, F. B., Brown, I. S., Campbell, S. K., and Harter, M. R. (1987). Separate verbal memory and naming deficits in attention deficit disorder and reading disability. Brain Lang. 31, 171–184. doi: 10.1016/0093-934X(87)90067-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Fletcher, P. C., Anderson, J., Shanks, D., Honey, R., Carpenter, T. A., Donovan, T., et al. (2001). Responses of human frontal cortex to surprising events are predicted by formal associative learning theory. Nat. Neurosci. 4, 1043–1048. doi: 10.1038/nn733

PubMed Abstract | CrossRef Full Text | Google Scholar

Friedman, D. (2013). The cognitive aging of episodic memory: a view based on the event-related brain potential. Front. Behav. Neurosci. 7:111. doi: 10.3389/fnbeh.2013.00111

PubMed Abstract | CrossRef Full Text | Google Scholar

Fujimoto, A., Tsurumi, K., Kawada, R., Murao, T., Takeuchi, H., Murai, T., et al. (2017). Deficit of state-dependent risk attitude modulation in gambling disorder. Transl. Psychiatry 7:1085. doi: 10.1038/tp.2017.55

PubMed Abstract | CrossRef Full Text | Google Scholar

Grondin, S. (2010). Timing and time perception: a review of recent behavioral and neuroscience findings and theoretical directions. Attention Percept. Psychophys. 72, 561–582. doi: 10.3758/APP.72.3.561

PubMed Abstract | CrossRef Full Text | Google Scholar

Hare, T. A., Camerer, C. F., and Rangel, A. (2009). Self-control in decision-making involves modulation of the vmPFC valuation system. Science 324, 646–648.

PubMed Abstract | Google Scholar

Hassabis, D., Kumaran, D., Vann, S. D., and Maguire, E. A. (2007). Patients with hippocampal amnesia cannot imagine new experiences. Proc. Natl. Acad. Sci. U.S.A. 104, 1726–1731. doi: 10.1073/pnas.0610561104

PubMed Abstract | CrossRef Full Text | Google Scholar

Hayne, H., and Imuta, K. (2011). Episodic memory in 3-and 4-year-old children. Dev. Psychobiol. 53, 317–322. doi: 10.1002/dev.20527

PubMed Abstract | CrossRef Full Text | Google Scholar

Kapur, S., Craik, F., Tulving, E., and Houle, S. (1997). 29-4-role of the prefrontal cortex in human episodic memory: lessons from pet studies. Biol. Psychiatry 42, 75S–76S.

Google Scholar

Khader, P. H., Pachur, T., Meier, S., Bien, S., Jost, K., and Rösler, F. (2011). Memory-based decision-making with heuristics: evidence for a controlled activation of memory representations. J. Cogn. Neurosci. 23, 3540–3554.

PubMed Abstract | Google Scholar

Khader, P. H., Pachur, T., Weber, L. A., and Jost, K. (2016). Neural signatures of controlled and automatic retrieval processes in memory-based decision-making. J. Cogn. Neurosci. 28, 69–83. doi: 10.1162/jocn_a_00882

PubMed Abstract | CrossRef Full Text | Google Scholar

Klein, S. B., Loftus, J., and Kihlstrom, J. F. (2002). Memory and temporal experience: the effects of episodic memory loss on an amnesic patient's ability to remember the past and imagine the future. Soc. Cogn. 20, 353–379. doi: 10.1521/soco.20.5.353.21125

CrossRef Full Text | Google Scholar

Levy, D. J., and Glimcher, P. W. (2012). The root of all value: a neural common currency for choice. Curr. Opin. Neurobiol. 22, 1027–1038. doi: 10.1016/j.conb.2012.06.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Mason, A., Farrell, S., Howard-Jones, P., and Ludwig, C. J. (2017). The role of reward and reward uncertainty in episodic memory. J. Mem. Lang. 96, 62–77. doi: 10.1016/j.jml.2017.05.003

CrossRef Full Text | Google Scholar

Panegyres, P. (2004). The contribution of the study of neurodegenerative disorders to the understanding of human memory. QJM 97, 555–567. doi: 10.1093/qjmed/hch096

PubMed Abstract | CrossRef Full Text | Google Scholar

Poldrack, R. A., and Foerde, K. (2008). Category learning and the memory systems debate. Neurosci. Biobehav. Rev. 32, 197–205. doi: 10.1016/j.neubiorev.2007.07.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Quinlan, D. M., and Brown, T. E. (2003). Assessment of short-term verbal memory impairments in adolescents and adults with ADHD. J. Atten. Disord. 6, 143–152. doi: 10.1177/108705470300600401

PubMed Abstract | CrossRef Full Text | Google Scholar

Schacter, D. L., Benoit, R. G., De Brigard, F., and Szpunar, K. K. (2015). Episodic future thinking and episodic counterfactual thinking: intersections between memory and decisions. Neurobiol. Learn. Mem. 117, 14–21. doi: 10.1016/j.nlm.2013.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Seymour, K. E., Reinblatt, S. P., Benson, L., and Carnell, S. (2015). Overlapping neurobehavioral circuits in adhd, obesity, and binge eating:evidence from neuroimaging research. CNS Spectrums 20, 401–411. doi: 10.1017/S1092852915000383

PubMed Abstract | CrossRef Full Text | Google Scholar

Squire, L. R. (2004). Memory systems of the brain: a brief history and current perspective. Neurobiol. Learn. Mem. 82, 171–177. doi: 10.1016/j.nlm.2004.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Suddendorf, T., Addis, D. R., and Corballis, M. C. (2009). Mental time travel and the shaping of the human mind. Philos. Trans. R. Soc. Lond. B Biol. Sci. 64, 1317–1324. doi: 10.1098/rstb.2008.0301

CrossRef Full Text | Google Scholar

Suddendorf, T., and Busby, J. (2003). Mental time travel in animals? Trends Cogn. Sci. 7, 391–396.

PubMed Abstract | Google Scholar

Suddendorf, T., and Busby, J. (2005). Making decisions with the future in mind: developmental and comparative identification of mental time travel. Learn. Motiv. 36, 110–125. doi: 10.1016/j.lmot.2005.02.010

CrossRef Full Text | Google Scholar

Suddendorf, T., and Corballis, M. C. (2007). The evolution of foresight: what is mental time travel, and is it unique to humans? Behav. Brain Sci. 30, 299–313. doi: 10.1017/S0140525X07001975

PubMed Abstract | CrossRef Full Text | Google Scholar

Tulving, E. (1985a). Elements of Episodic Memory. Oxford, UK: Oxford University Press.

Google Scholar

Tulving, E. (1985b). Memory and consciousness. Can. Psychol. 26:1. doi: 10.1037/h0080017

CrossRef Full Text | Google Scholar

Wearden, J. H., and Lejeune, H. (2008). Scalar properties in human timing: conformity and violations. Q. J. Exp. Psychol. 61, 569–587. doi: 10.1080/17470210701282576

PubMed Abstract | CrossRef Full Text | Google Scholar

Weilbächer, R. A., and Gluth, S. (2016). The interplay of hippocampus and ventromedial prefrontal cortex in memory-based decision making. Brain Sci. 7:4. doi: 10.3390/brainsci7010004

PubMed Abstract | CrossRef Full Text | Google Scholar

Wittmann, B. C., Schiltz, K., Boehler, C. N., and Düzel, E. (2008). Mesolimbic interaction of emotional valence and reward improves memory formation. Neuropsychologia 46, 1000–1008. doi: 10.1016/j.neuropsychologia.2007.11.020

PubMed Abstract | CrossRef Full Text | Google Scholar