A temporal basis for Weber's law in value perception

Weber's law—the observation that the ability to perceive changes in magnitudes of stimuli is proportional to the magnitude—is a widely observed psychophysical phenomenon. It is also believed to underlie the perception of reward magnitudes and the passage of time. Since many ecological theories state that animals attempt to maximize reward rates, errors in the perception of reward magnitudes and delays must affect decision-making. Using an ecological theory of decision-making (TIMERR), we analyze the effect of multiple sources of noise (sensory noise, time estimation noise, and integration noise) on reward magnitude and subjective value perception. We show that the precision of reward magnitude perception is correlated with the precision of time perception and that Weber's law in time estimation can lead to Weber's law in value perception. The strength of this correlation is predicted to depend on the reward history of the animal. Subsequently, we show that sensory integration noise (either alone or in combination with time estimation noise) also leads to Weber's law in reward magnitude perception in an accumulator model, if it has balanced Poisson feedback. We then demonstrate that the noise in subjective value of a delayed reward, due to the combined effect of noise in both the perception of reward magnitude and delay, also abides by Weber's law. Thus, in our theory we prove analytically that the perception of reward magnitude, time, and subjective value change all approximately obey Weber's law.

Introduction

Weber's law, or approximate Weber's law, has been observed in the perception of stimulus features such as weight (Weber, 1978; Killeen et al., 1993), length (Dehaene and Brannon, 2011; Droit-Volet, 2013; Akre and Johnsen, 2014), brightness (Rovamo et al., 1995), number (Whalen et al., 1999; Cordes et al., 2001; Nieder and Miller, 2003; Cantlon and Brannon, 2006; Beran, 2007; Gallistel, 2011; Droit-Volet, 2013), reward magnitude (Killeen et al., 1993; Bateson et al., 1995; Kacelnik and Bateson, 1996), time (Gibbon, 1977; Gibbon et al., 1984; Matell and Meck, 2000; Buhusi and Meck, 2005), loudness (Forrest, 1994; Bee et al., 2012) etc. (Akre and Johnsen, 2014). It states that the ability to perceive a change in a quantity decreases in proportion to its magnitude. The fact that our ability to perceive a change in a stimulus often decreases as its magnitude increases is immediately recognized; for instance, it is more difficult to perceive an increase of 1 g if one is measuring 100 g as opposed to when measuring 2 g. Weber's law, however, states that this decrease in ability to assess magnitude is proportional to the magnitude of the stimulus, i.e., that it is 50 times more difficult to perceive a given change around 100 g than it is to perceive the same change around 2 g. Even though there is considerable experimental support for the law, its neural or evolutionary origin is unclear (Walsh, 2003; Bueti and Walsh, 2009; Akre and Johnsen, 2014). Further, since animals are often thought to make decisions so as to maximize reward rates (thus requiring perception of reward magnitude and delays) (Stephens and Krebs, 1986; Balci et al., 2011; Blanchard et al., 2013; Namboodiri et al., 2014b), Weber's law in the perception of reward magnitudes and delays must affect such decisions. The mathematical properties of such effects on the decisions of animals are, however, unclear.

Previously, we presented a theory of decision-making and time perception that postulates that the decision of animals regarding delayed outcomes is a consequence of reward rate maximization in a limited temporal window that includes a past integration interval (over which experienced reward rate is estimated) and the delay to a given reward (TIMERR) (Namboodiri et al., 2014b). We showed that the decision-making algorithm resulting from this postulate automatically includes an estimate of opportunity cost and an explicit cost of time. We further showed that it can explain the breadth of behavioral observations on intertemporal decision-making. The theory also postulates that time is represented subjectively such that the subjective reward rate equals the objective change in reward rate, i.e., a subject's estimate of the subjective value per unit subjective time accurately represents how much the reward rate of the current offer exceeds the experienced reward rate. Using this theory, we examine the origin of Weber's law in reward magnitude in this paper and show that the perception of reward magnitude is correlated with the perception of time, and that the subjective value change of a delayed reward should also approximately abide by Weber's law. We also present a novel accumulator model of sensory perception that predicts approximate Weber's law for quantities (such as reward magnitude) that are measured over finite sensory intervals.

Results

Our main aim in this paper is to study how errors in the subjective representation of an interval correspondingly affect the subjective value of that reward. To this end, we first express the subjective value of a delayed reward in terms of the subjective representation of the delay.

The subjective value of a reward with magnitude r delayed by an interval t as calculated in TIMERR (Figure 1) is:

$\begin{array}{ll}SV\left(r,t\right)=\frac{r-a_{est}t}{1+\frac{t}{T_{ime}}}\hfill & \left(1\right)\hfill \end{array}$

where T_ime represents the past integration interval, i.e., the interval over which the past reward rate (a_est) is estimated. Importantly, T_ime is not a perceived temporal interval, but is merely the effective interval over which the past reward rate is estimated (e.g., using an exponential memory filter as in Namboodiri et al., 2014a).

FIGURE 1

Figure 1. The subjective value of a delayed reward (r) is calculated as the immediate reward that produces the same total reward rate over a window including a past-integration interval (T_ime) (over which experienced reward rate is calculated, a_est) and the expected delay (t) to a future reward. The figure shows a ready means to visually depict the subjective value of a delayed reward, shown as the purple bar at time zero (“now”). Modified from Namboodiri et al. (2014a).

In the above equation, r can be thought of as the magnitude of an offered reward. But a more rigorous definition of r is the subjective value of an immediate offered reward, i.e., r = SV(r, 0).

Correspondingly, the subjective representation of the delay t as expressed in TIMERR is:

$\begin{array}{ll}ST\left(t\right)=\frac{t}{1+\frac{t}{T_{ime}}}\hfill & \left(2\right)\hfill \end{array}$

Thus, the subjective representation of time is a non-linear mapping and its non-linearity is controlled by the past integration interval. It is important to emphasize that the subjective representation of the delay expressed above is not the subjective (verbal) report of an interval; it can be thought of as the non-linear neural representation of an interval.

Equation (1) can now be re-expressed in terms of the subjective representation of time as shown in Equation (2) as

$\begin{array}{ll}\begin{array}{l}SV\left(r,t\right)=\frac{r}{1+\frac{t}{T_{ime}}}-a_{est}ST(t)=r\frac{ST(t)}{t}-a_{est}ST(t)\\ =r\frac{ST(t)}{\frac{ST(t)}{1-\frac{ST(t)}{T_{ime}}}}-a_{est}ST(t)\end{array}\hfill & \left(3\right)\hfill \end{array}$

Therefore,

$\begin{array}{ll}SV\left(r,t\right)=r-\left(\frac{r}{T_{ime}}+a_{est}\right)ST(t)\hfill & \left(4\right)\hfill \end{array}$

Thus, the discounting of a delayed reward is linear with respect to the subjective representation of that delay. We assume here that the subjective value of a delayed reward is calculated by first measuring the subjective representation of the delay and then linearly discounting using the form expressed in Equation (4). This linear discounting with respect to the subjective representation of time is a direct result of the postulate of our theory that animals maximize reward-rates over a limited temporal window including the past integration interval and the delay to future reward.

Contribution of Time Measurement Error to the Error in Subjective Value

From this relation, we can now calculate the error in subjective value of a delayed reward resulting from an error in the representation of subjective time (Figure 2). To this end, let us denote that the just-noticeable-difference (JND) in the subjective representation ST(t) of the delay t by δST(t), and that the error in the corresponding subjective value is denoted by δSV(r, t). For the purpose of this section, we assume that the measurement of the reward magnitude is noiseless. Then, as the subjective representation of the delay t increases by its JND, the subjective value will increase by the corresponding error. This can be expressed mathematically as:

$\begin{array}{ll}SV(r,t)+\delta SV(r,t)=r-\left(\frac{r}{T_{ime}}+a_{est}\right)(ST(t)+\delta ST(t))\hfill & \left(5\right)\hfill \end{array}$

FIGURE 2

Figure 2. Errors in measurement of the delay to a future reward results in a corresponding error in subjective value. If the delay to the reward is perceived as earlier by the just-noticeable difference (JND), the subjective value is perceived as being larger. This error in subjective value is shown in the red bar and is calculated analytically in Section Contribution of time measurement error to the error in subjective value.

From Equations (4, 5), we can now calculate the JND in subjective value as

$\begin{array}{ll}\delta SV(r,t)=-\left(\frac{r}{T_{ime}}+a_{est}\right)\delta ST(t)\hfill & \left(6\right)\hfill \end{array}$

The negative sign here implies that as the delay increases, the subjective value decreases, i.e., the value is discounted.

We have previously shown that the error in the subjective representation of time is approximately linearly related to the subjective representation of time in an accumulator model (Namboodiri et al., 2014b), i.e., δST(t) = kST(t) + c. The contribution due to the constant term c can be thought of a constant read-out error and is quite small except in the limit of ST(t) approaching zero. Substituting this relationship into Equation (6), we get

$\begin{array}{ll}\delta SV(r,t)=-\left(\frac{r}{T_{ime}}+a_{est}\right)(kST(t)+c)\hfill & \left(7\right)\hfill \end{array}$

Equation (7) can also be rewritten using Equation (4) as

$\begin{array}{ll}\delta SV(r,t)=-c\left(\frac{r}{T_{ime}}+a_{est}\right)-k(r-SV(r,t))\hfill & \left(8\right)\hfill \end{array}$

From the above equation, it can be seen that the error in subjective value of a delayed reward is linearly related to the drop in subjective value from time zero due to the passage of time. Hence, Weber's law applies for the reduction in subjective value of a delayed reward due to the delay, i.e., to r − SV(r, t). In other words, as the delay increases and the subjective value reduces, the error in the change of subjective value due to the delay is proportional to the change in subjective value. Henceforth, we refer to this as Weber's law in value perception.

Let us now examine the effect of reducing the delay to zero. Since the negative sign in the above equations only indicates the direction of change, we drop this sign from here on for the calculation of noise. Thus, when t = 0, both Equations (7, 8) become

$\begin{array}{ll}\delta SV(r,0)=c\left(\frac{r}{T_{ime}}+a_{est}\right)\hfill & \left(9\right)\hfill \end{array}$

Thus, the error in the subjective value of an immediate reward is proportional to the magnitude of the reward. This is Weber's law in magnitude perception resulting purely from an error in the perception of an infinitesimally small immediate delay rather than arising solely from magnitude measurement error as is commonly believed. Interestingly, as the past integration interval (T_ime) increases—leading to an increased accuracy of time perception (Namboodiri et al., 2014b)—so does the accuracy of reward magnitude perception. This is a novel, untested prediction of the account presented here. This temporal basis of Weber's law for the perception of reward magnitude also predicts that the accuracy of magnitude representation reduces when the past reward rate is high. This too, is a novel, testable prediction and is consistent with the notion that when reward rate is high, the need to represent rewards accurately (thus incurring greater metabolic costs) is reduced. The above two predictions regarding the dependence of errors in subjective value on the past integration interval and the past reward rate is depicted in Figure 3.

FIGURE 3

Figure 3. (A) The Weber fraction of error in subjective value decreases with an increase in the past integration interval. This is mathematically represented in Equation (8). Compared to Figure 2 (represented as low T_ime in the graph on the right), the past integration interval is larger in this panel, thus reducing the error in subjective value while increasing the subjective value. The Weber fraction is thus smaller. (B) The Weber fraction of error in subjective value increases with an increase in the past reward rate. This is mathematically represented in Equation (8). Compared to Figure 2 (represented as low a_est in the graph on the right), the past reward rate is larger in this panel, thus increasing the error in subjective value while decreasing the subjective value. The Weber fraction is thus larger.

The above treatment indicates that noise in time perception results in reward magnitude perception that abides by Weber's law. Yet note that, to calculate the error in subjective magnitude (resulting purely from the noise in the measurement of the infinitesimally-small delay to immediate reward), we heretofore have ignored the contribution of noise in the measurement of the reward magnitude itself. Since we do not know the relative contributions of these sources of noise, it is possible that the source related to time might contribute but minimally to the overall error in reward magnitude perception. Therefore, in the next section, we present a model of sensory perception for reward magnitude, and then calculate the resulting perceptual error.

Sensory Measurement Error of Reward Magnitude due to Evidence Accumulation

In this section, we calculate the sensory measurement error of reward magnitude (e.g., error in the measurement of the volume of a water reward). In some modalities, the sensory receptor is itself thought to produce scalar noise (Matthews et al., 1990; Donner et al., 1998; Nieder and Miller, 2003). While this is possible in the measurement of reward magnitude, we do not consider this simple solution here as neural elements in the central nervous system are typically considered to approximate Poisson processes, which have square-root noise and not linear noise (Rieke et al., 1999). Rather, we consider errors in ascribing value to a given reward magnitude as resulting from central and not peripheral processes. While there are other models for Weber's law in sensation (Treisman, 1966; Dehaene, 2003; Deco and Rolls, 2006; Shouval et al., 2013), ours is based on the fact that the measurement of any sensory quantity has to be carried out over time.

To this end, we assume that the sensory process for measuring the magnitude is carried out in time over a small temporal window of sensation. This sensory window is defined as the time over which there is a constant rate of sensory input. Hence, we assume that the net perceived reward magnitude is proportional to the time it takes to integrate the sensory input (e.g., when drinking water at a constant rate, the amount of water obtained is proportional to the duration of consumption). For an alternative model of sensory integration, see Appendix A1 in Supplementary Material. In order to evaluate the noise in measurement, we assume that this sensory integration can be described by an accumulator model similar to previous decision-making models used for evidence accumulation (e.g., Simen et al., 2011; Brunton et al., 2013). We further assume that the reward magnitude is represented linearly and does not undergo a logarithmic transformation, as has been suggested for number representation (Dehaene, 2003). In the rest of this section, we formalize this accumulator model using a stochastic differential equation, and then analytically calculate the time dependence of its mean and variance.

If the neural system carrying out this sensory integration were perfectly noiseless, we can describe the accumulator model by the following differential equation

$\begin{array}{ll}d{r}_t=adt;0\le t\le t_{sensory}\hfill & \left(10\right)\hfill \end{array}$

Here, r_t represents the integrated reward magnitude at a given time. Thus, the measured reward magnitude r will be the integrated magnitude at the end of the sensory window, t_sensory, i.e., r = r_tsensory. The rate of sensory input is denoted by a.

We now relax the assumption that the sensory integration is noiseless. Noise in such an accumulator system can result from two sources: noise in the sensory input and feedback noise in the accumulator. We assume that the feedback is a zero mean noise resulting from balanced excitatory/inhibitory connections, similar to many previous works (e.g., Simen et al., 2011; Brunton et al., 2013), and that the neurons performing these computations can all be described as Poisson point processes, i.e., the variance of each source of noise will be proportional to the corresponding signal. Thus, the variance of the sensory input will be proportional to the input (a) and the feedback noise will be proportional to r_t. We denote the proportionality constants as b and σ respectively.

For simplicity, we first assume that these two sources of noise are independent and additive. Since the variance of the sum of two independent sources sum up, the net variance can be expressed as σ²r_t + b²a. If we consider the variance of the noise term as constant throughout the integration, it can be represented by introducing an additional diffusive term that approximates a Brownian motion with infinitesimal variance of σ²r_t + b²a into Equation (10). Thus, the introduction of these noise sources can be formally described by the following stochastic differential equation

$\begin{array}{ll}d{r}_t=adt+\sqrt{{\sigma }^2{r}_t+b^{2}a}d{W}_t;0\le t\le t_{sensory}\hfill & \left(11\right)\hfill \end{array}$

W_t represents a standard Wiener process (Brownian motion).

We will analytically solve the time dependence for the first and second moments of the above accumulator [shown in Equation (11)] so as to calculate the mean and variance at the end of the sensory window.

Taking the expectation values on both sides of Equation (11), we get

$\begin{array}{ll}d<r_t>=adt;0\le t\le t_{sensory}\hfill & \left(12\right)\hfill \end{array}$

where < r_t > represents the expectation value of r_t. Since < r₀ > = 0, we can write the solution obtained by integrating from 0 to t as

$\begin{array}{ll}<r_t>=at\hfill & \left(13\right)\hfill \end{array}$

The time evolution equation for < r²_t > can similarly be calculated by applying Ito's product rule as

$\begin{array}{ll}\begin{array}{l}d{r}_t^2=2r_{t}d{r}_t+{(d{r}_t)}^2=2a{r}_{t}dt+2r_t\sqrt{{\sigma }^2{r}_t+b^{2}a}d{W}_t\\ +{\left(adt+\sqrt{{\sigma }^2{r}_t+b^{2}a}d{W}_t\right)}^{2}0\le t\le t_{sensory}\end{array}\hfill & \left(14\right)\hfill \end{array}$

Using dt² = 0, dW_tdt = 0, and dW²_t = dt and taking the expectations of both sides, we get

$\begin{array}{ll}\begin{array}{l}d<r_t^2>=\left(2<a{r}_t>+<\sqrt{{\sigma }^2{r}_t+b^{2}a}.\sqrt{{\sigma }^2{r}_t+b^{2}a}>\right)dt;\\ 0\le t\le t_{sensory}\end{array}\hfill & \left(15\right)\hfill \end{array}$

Simplifying, we get

$\begin{array}{ll}d<r_t^2>=\left(({\sigma }^2+\text{2}a)<r_t>+b^{2}a\right)dt;0\le t\le t_{sensory}\hfill & \left(16\right)\hfill \end{array}$

Substituting from Equation (13) and integrating from 0 to t with the boundary condition of < r²_{t = 0} > = 0, we get

$\begin{array}{ll}<r_t^2>=a({\sigma }^2+2a)\frac{t^2}{2}+b^{2}at;0\le t\le t_{sensory}\hfill & \left(17\right)\hfill \end{array}$

Thus, the variance of r_t can be calculated as

$\begin{array}{ll}\text{var}(r_t)=<r_t^2>-<r_t{>}^2=\frac{a{\sigma }^2{t}^2}{2}+b^{2}at;0\text{}\le t\text{}\le t_{sensory}\hfill & \left(18\right)\hfill \end{array}$

The coefficient of variation of r_t is thus

$\begin{array}{ll}CV(r_t)=\sqrt{\frac{{\sigma }^2}{2a}+\frac{b^2}{at}};0<t\le t_{sensory}\hfill & \left(19\right)\hfill \end{array}$

Since the measured reward magnitude is the integrated magnitude after the sensory window, the CV of the measurement can be written as

$\begin{array}{ll}CV(r)=\sqrt{\frac{{\sigma }^2}{2a}+\frac{b^2}{a{t}_{sensory}}}\hfill & \left(20\right)\hfill \end{array}$

or

$\begin{array}{ll}CV(r)=\sqrt{\frac{{\sigma }^2}{2a}+\frac{b^2}{r}}\hfill & \left(21\right)\hfill \end{array}$

If one assumes that the rate of sensory input is a constant, the above equation shows that except for low reward magnitudes, the CV is a constant, i.e., Weber's law holds approximately for reward magnitude perception. If σ²/a is large compared to b², the constant term will dominate and the CV would be almost exactly constant. These analytical results are confirmed in numerical simulations as shown in Figure 4.

FIGURE 4

Figure 4. Confirmatory simulations (see Methods) of the analytical solution of an accumulator model in which the sensory and feedback noise combine additively. The red line shows the result of the analytical calculation as expressed in Equation (21), wherein the sensory signal (a), magnitude of sensory noise (b), and the magnitude of feedback noise (σ) are varied. The black dots show the results of numerical simulation. The results approximate Weber's law well but for low reward magnitudes and high sensory noise (b).

The mathematics of the accumulator shown in Equation (11) is quite similar to Equation (9) in (Simen et al., 2011). But there are some significant differences in the meaning of the terms. First, our model is for reward magnitude perception, whereas theirs is for time interval production. Second, as a consequence, while in our model the rate of sensory input is assumed to be a constant, they assume that the rate of accumulation is tuned for the interval to be timed. For this reason, their model can produce scalar timing only for time interval production and not for time interval measurement/perception where the coefficient of variation decreases in inverse proportion to the square root of the interval [similar to the second term in Equation (21)].

Equations (11–21) assumed that the sensory input noise is additive with respect to the feedback noise. Instead, if this noise were in fact multiplicative, Equation (11) would change to

$\begin{array}{ll}d{r}_t=adt+\sigma \sqrt{b^{2}a{r}_t}d{W}_t;0\le t\le t_{sensory}\hfill & \left(22\right)\hfill \end{array}$

In this case, the CV can similarly be calculated as (shown in Appendix A2 in Supplementary Material)

$\begin{array}{ll}CV(r)=\frac{\sigma b}{\sqrt{2}}\hfill & \left(23\right)\hfill \end{array}$

Thus, when the sensory and feedback noises multiply, the coefficient of variation is independent of the magnitude of the sensory signal (a).

Again, we performed confirmatory numerical simulations of Equation (22), the results of which are shown in Figure 5. Therefore, if the sensory input noise is multiplicative, the coefficient of variation is exactly constant, thus making Weber's law exact. Instead, if the sensory input noise is additive, the coefficient of variation shows deviations from exact Weber's law at low reward magnitudes.

FIGURE 5

Figure 5. Confirmatory simulations (see Methods) of the analytical solution of an accumulator model in which the sensory and feedback noise combine multiplicatively. The red line shows the result of the analytical calculation as expressed in Equation (23) wherein the sensory signal (a), magnitude of sensory noise (b), and the magnitude of feedback noise (σ) are varied. The black dots show the results of numerical simulation. Here, Weber's law is exact.

The accumulator model considered above is similar to the one that we previously proposed for the representation of subjective time (Namboodiri et al., 2014b), with two differences. The most important difference is that whereas subjective time is assumed to be a non-linear transform of real time, subjective reward is assumed to be linearly proportional to the real reward. Due to this difference, the reward magnitude accumulator is analytically tractable, unlike the subjective time accumulator, for which the analytical solution was approximate (Namboodiri et al., 2014b). The other difference is that since the reward magnitude accumulator operates on a sensory input (unlike the subjective time accumulator), the contribution of this sensory noise has also been included.

Combined Error due to Time and Magnitude Measurements on Subjective Value

We now have all the elements to calculate the error in subjective value of a delayed reward resulting from errors in both magnitude and time measurements (Figure 6).

FIGURE 6

Figure 6. The error in subjective value is affected by errors in the measurement of both delay (as shown in Figure 2) and reward magnitude. This combined error is calculated analytically in Section Combined error due to time and magnitude measurements on subjective value.

Returning to Equation (4), if we consider the effect of adding the JND of both r and ST(t), we see that while adding the JND of r leads to an increase in the SV(r, t), adding the JND of ST(t) leads to a decrease (due to temporal discounting). Since we are only interested in the net error, so as to match the direction of change, we will consider the effect of error in both r and ST(t) by adding the JND of r and subtracting the JND of ST(t). Thus, we get the following equation

$\begin{array}{ll}\begin{array}{l}SV(r,t)+\delta SV(r,t)=r+\delta r-\left(\frac{r+\delta r}{T_{ime}}+a_{est}\right)(ST(t)\\ -\delta ST(t))\end{array}\hfill & \left(24\right)\hfill \end{array}$

Therefore, using Equation (4), the error in subjective value δSV(r, t) can be written as

$\begin{array}{ll}\begin{array}{l}\delta SV(r,t)=\delta r\left(1-\frac{ST(t)}{T_{ime}}\right)+\left(\frac{r}{T_{ime}}+a_{est}\right)\delta ST(t)\\ +\frac{\delta r\delta ST(t)}{T_{ime}}\end{array}\hfill & \left(25\right)\hfill \end{array}$

From Equation (4), $\left(1-\frac{ST(t)}{T_{ime}}\right)=\frac{SV(r,t)+a_{est}ST(t)}{r}$. Therefore, Equation (25) becomes

$\begin{array}{ll}\begin{array}{l}\delta SV(r,t)=\frac{\delta r}{r}\left(SV(r,t)+a_{est}ST(t)\right)+\left(\frac{r}{T_{ime}}+a_{est}\right)\delta ST(t)\\ +\frac{\delta r\delta ST(t)}{T_{ime}}\end{array}\hfill & \left(26\right)\hfill \end{array}$

For simplicity, we consider the exact form of Weber's law to hold for the sensory measurement of r. Therefore, we write $\frac{\delta r}{r}=l$, where l is the Weber fraction.

From Equations (6), (8), the second term in the R.H.S is equal to $c\left(\frac{r}{T_{ime}}+a_{est}\right)+k(r-SV(r,t))$, where δST(t) = kST(t) + c.

Before calculating the error in subjective value at any delay, we first calculate its value for an immediate reward, where t = 0 and ST(t) = 0. From Equation (26), this can be written as

$\begin{array}{ll}\delta SV(r,0)=lr+c\left(\frac{r}{T_{ime}}+a_{est}\right)+\frac{lcr}{T_{ime}}\hfill & \left(27\right)\hfill \end{array}$

Simplifying, we get

$\begin{array}{ll}\delta SV(r,0)=r\left(l\left(1+\frac{c}{T_{ime}}\right)+\frac{c}{T_{ime}}\right)+c{a}_{est}\hfill & \left(28\right)\hfill \end{array}$

The above equation obeys Weber's law for reward magnitude perception, resulting from errors in both the measurement of magnitude and the measurement of the infinitesimal delay to an immediate reward. As can be seen, the Weber fraction [slope of δSV(r, 0) with respect to r] depends on T_ime, the past integration interval. Thus, we predict that even within an individual, the Weber fraction in the perception of reward magnitude (subjective value of an immediate reward) can change depending on the context, as the past integration interval changes. The direction of this change will be such that the better the perception of time, the better the perception of reward magnitude. Further, as mentioned previously after Equation (9), the above equation also predicts that the larger the experienced reward rate, the larger the error in perception of reward magnitude. These are the strong falsifiable predictions of our account.

We now calculate the error in subjective value at a given delay t due to errors in both time and reward magnitude measurement. From Equation (26), we get

$\begin{array}{ll}\begin{array}{l}\delta SV(r,t)=l\left(SV(r,t)+a_{est}ST(t)\right)+c\left(\frac{r}{T_{ime}}+a_{est}\right)\\ +k(r-SV(r,t))+\frac{lr(kST(t)+c)}{T_{ime}}\end{array}\hfill & \left(29\right)\hfill \end{array}$

Simplifying, we get

$\begin{array}{ll}\begin{array}{l}\delta SV(r,t)=(l-k)SV(r,t)+\left(l{a}_{est}+\frac{lrk}{T_{ime}}\right)ST(t)\\ +r\left(\frac{(1+l)c}{T_{ime}}+k\right)+c{a}_{est}\end{array}\hfill & \left(30\right)\hfill \end{array}$

Since we are interested in the noise in subjective value of a constant reward magnitude delayed by varying amounts, if we treat r as a constant (for now), we can write [using Equation (4)] $ST(t)=\frac{r-SV(r,t)}{\frac{r}{T_{ime}}+a_{est}}$. Grouping the terms that are proportional to SV(r, t) separately from the other terms, the above equation becomes

$\begin{array}{ll}\begin{array}{l}\delta SV(r,t)=\left(l-k-l\frac{a_{est}{T}_{ime}+rk}{a_{est}{T}_{ime}+r}\right)SV(r,t)\\ +\left(r\left(\frac{(1+l)c}{T_{ime}}+k+l\frac{a_{est}{T}_{ime}+rk}{a_{est}{T}_{ime}+r}\right)+c{a}_{est}\right)\end{array}\hfill & \left(31\right)\hfill \end{array}$

The above equation also abides by Weber's law. Thus, we have shown that the error in subjective value of a given reward delayed by different amounts is proportional to the subjective value at each given delay. Again, the Weber fraction depends on the reward environment of the animal since it depends on r, a_est, and T_ime.

We can also similarly calculate the subjective value error at a given delay for differing reward magnitudes. To do this, we substitute r as [using Equation (4)] $r=\frac{SV(r,t)+a_{est}ST(t)}{1-\frac{ST(t)}{T_{ime}}}$ in a rewritten version of Equation (25) as shown below.

$\begin{array}{ll}\begin{array}{l}\delta SV(r,t)=lr\left(1-\frac{ST(t)}{T_{ime}}\right)+c\left(\frac{r}{T_{ime}}+a_{est}\right)\\ +k\left(\frac{r}{T_{ime}}+a_{est}\right)ST(t)+\frac{lr(kST(t)+c)}{T_{ime}}\end{array}\hfill & \left(32\right)\hfill \end{array}$

Thus,

$\begin{array}{ll}\begin{array}{l}\delta SV(r,t)=l\left(SV(r,t)+a_{est}ST(t)\right)+r(1+l)\frac{c+kST(t)}{T_{ime}}\\ +a_{est}(c+kST(t))\end{array}\hfill & \left(33\right)\hfill \end{array}$

Or,

$\begin{array}{ll}\begin{array}{l}\delta SV(r,t)=SV(r,t)\left(l+(1+l)\frac{c+kST(t)}{T_{ime}-ST(t)}\right)+a_{est}(lST(t))\\ +ST(t)(1+l)\frac{c+kST(t)}{T_{ime}-ST(t)}+c+kST(t))\end{array}\hfill & \left(34\right)\hfill \end{array}$

where $ST(t)=\frac{t}{1+\frac{t}{T_{ime}}}$.

This too abides by Weber's law. Thus, we have also shown that the error in subjective value at a given delay for different reward magnitudes is proportional to the subjective value.

Discussion

Previously, we presented a general theory of intertemporal decision-making and time perception (TIMERR) that explains many well-established observations in these fields (Namboodiri et al., 2014b). Our theory states that the decisions of animals are a consequence of maximizing reward rates in a limited temporal window including a past integration interval and the delay to a current reward. Interestingly, we showed that the representation of time is also related to the past integration interval in our framework, and that impulsive (low tolerance to delays of rewards) individuals have an impaired perception of time. We then demonstrated that the error in perception of time is approximately scalar, with the deviation from exact Weber's law depending on the past integration interval.

In this paper, we extend the results of our prior work to consider the role of error in time perception on the perception of reward magnitudes and the subjective values of delayed rewards. We showed that the error in perception of the infinitesimally small delay to an immediate reward affects the perception of reward magnitude in accordance with Weber's law. Since the sensory measurement of the reward must be carried out over time, we derived Weber's law in the sensation of reward magnitude by assuming an accumulator model (for this sensory integration) with a Poisson feedback with balanced excitation/inhibition. This could be the underlying reason behind the observation of Weber's law in the perception of reward magnitude by animals. Subsequently, we showed that in TIMERR, the combination of errors in both time and reward magnitude measurement on the subjective value change of a delayed reward also accords with Weber's law. Crucially, the Weber fractions are predicted to depend on the reward history of the animal, thus providing a strong, falsifiable prediction of our theory, along with the predicted correlation between errors in time perception and reward magnitude estimation.

Superficially, it might be assumed that since the perception of reward magnitude abides by Weber's law, so should the subjective value of a delayed reward. In fact, such an assertion has previously been made (Cui, 2011) without the recognition that this requires a specific relation between subjective value, reward magnitude, delay to reward, and the perception of the delay. From our analytical derivation presented above, it is evident that Weber's law in subjective value change is a consequence of the special forms of discounting function (subjective value of a delayed reward divided by the subjective value of that reward when presented immediately) and subjective time representation that result from our theory. In fact, if one were to make the standard assumptions of (1) Weber's law in reward magnitude measurement, (2) a hyperbolic discounting function (Ainslie, 1974; Frederick et al., 2002; Kalenscher and Pennartz, 2008; Cui, 2011), and (3) linear subjective representation of time that abides by Weber's law (Gibbon, 1977; Gibbon et al., 1984), the resultant error in subjective value of a delayed reward is far from proportional to the subjective value, as we show in Appendix A3 in in Supplementary Material.

Recent experiments have shown that the representation of reward magnitude or value is not just dependent on the reward under consideration, but also on other available options (Huber et al., 1982; Bateson et al., 2003; Louie et al., 2013). A recent neuroeconomic model (Louie et al., 2013) employing a divisive normalization scheme wherein each individual reward is compared against the other available options can produce such context dependence. In light of these findings, one might question our assumption of an absolute code for reward magnitude, i.e., our assumption that reward magnitude is represented based only on the magnitude of the reward of interest. It is thus important to point out that our theory predicts context dependent choices even under the assumption that the reward magnitude representation is independent of the other available options. This is because the subjective value of a reward (since every reward is effectively a delayed reward) is affected by the animal's estimate of its past reward rate [Equation (1)]. Thus, the presence of distracters affects the subjective value of a reward due to an effect on the past reward rate in experiments involving sequential choices. Additionally, the current options might affect one's estimate of experienced reward rate (Namboodiri et al., 2014a). Further, as shown in Equations (9, 28), the larger the value of the past reward rate, the larger the error (Weber fraction) in representation of a reward. Thus, our theory predicts that the larger the value of the distracter (thereby making the past reward rate larger), the higher the errors in deciding between two rewards, in accordance with the experimental observations shown in Louie et al. (2013). The key difference between our account and the divisive normalization account (Louie et al., 2013) is that in our account, the context dependence is due to the estimation of past reward rate, whereas in divisive normalization, the context dependence is based only on the currently available options.

There have been prior models of how Weber's law in reward magnitude and time perception affects decisions of animals in the context of external variability along these two dimensions (see Kacelnik and Bateson, 1996; Kacelnik and Brito e Abreu, 1998). These models have been successful at explaining why animals tend to prefer variability in time, but not in reward magnitude, in comparison with fixed options of the same mean (see Kacelnik and Bateson, 1996; Kacelnik and Brito e Abreu, 1998). However, they do not propose an origin of Weber's law for reward magnitude or time, nor do they calculate the net error due to both sources of noise. Unique to our theory, we predict a systematic relationship between the reward history of animals and their perception of these quantities.

Methods

The confirmatory simulations performed for Figures 4, 5 integrated Equations (11, 22) respectively using the Euler-Maruyama method. Thus, the discrete time version of the equation used for Figure 4 was

$r_{t+\Delta t}=r_t+a\Delta t+\sqrt{{\sigma }^2{r}_t+b^{2}a}\sqrt{\Delta t}N(0,1)$

and that for Figure 5 was

$r_{t+\Delta t}=r_t+a\Delta t+\sigma b\sqrt{a{r}_t}\sqrt{\Delta t}N(0,1)$

where N(0,1) is the standard normal distribution. The step size for integration, Δt, was set to 0.001 units. The parameters were changed as shown in the figure legend. In each case, the same random seed was used to initialize the simulations.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was funded by NIMH (R01 MH084911 and R01 MH093665) to Marshall G. Hussain Shuler.

Supplementary Material

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fnint.2014.00079/abstract

References

Ainslie, G. W. (1974). Impulse control in pigeons. J. Exp. Anal. Behav. 21, 485–489. doi: 10.1901/jeab.1974.21-485

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Akre, K. L., and Johnsen, S. (2014). Psychophysics and the evolution of behavior. Trends Ecol. Evol. 29, 291–300. doi: 10.1016/j.tree.2014.03.007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Balci, F., Freestone, D., Simen, P., Desouza, L., Cohen, J. D., and Holmes, P. (2011). Optimal temporal risk assessment. Front. Integr. Neurosci. 5:56. doi: 10.3389/fnint.2011.00056

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Bateson, M., Healy, S. D., and Hurly, T. A. (2003). Context-dependent foraging decisions in rufous hummingbirds. Proc. Biol. Sci. 270, 1271–1276. doi: 10.1098/rspb.2003.2365

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Bateson, M., Kacelnik, A., Road, S. P., and Ox, O. (1995). Accuracy of memory for amount in the foraging starling, Sturnus vulgaris. Anim. Behav. 50, 431–443. doi: 10.1006/anbe.1995.0257

CrossRef Full Text | Google Scholar

Bee, M. A., Vélez, A., and Forester, J. D. (2012). Sound level discrimination by gray treefrogs in the presence and absence of chorus-shaped noise. J. Acoust. Soc. Am. 131, 4188–4195. doi: 10.1121/1.3699271

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Beran, M. J. (2007). Rhesus monkeys (Macaca mulatta) enumerate large and small sequentially presented sets of items using analog numerical representations. J. Exp. Psychol. Anim. Behav. Process. 33, 42–54. doi: 10.1037/0097-7403.33.1.42

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Blanchard, T. C., Pearson, J. M., and Hayden, B. Y. (2013). Postreward delays and systematic biases in measures of animal temporal discounting. Proc. Natl. Acad. Sci. U.S.A. 110, 15491–15496. doi: 10.1073/pnas.1310446110

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Brunton, B. W., Botvinick, M. M., and Brody, C. D. (2013). Rats and humans can optimally accumulate evidence for decision-making. Science 340, 95–98. doi: 10.1126/science.1233912

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Bueti, D., and Walsh, V. (2009). The parietal cortex and the representation of time, space, number and other magnitudes. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 1831–1840. doi: 10.1098/rstb.2009.0028

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Buhusi, C. V., and Meck, W. H. (2005). What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 6, 755–765. doi: 10.1038/nrn1764

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Cantlon, J. F., and Brannon, E. M. (2006). Shared system for ordering small and large numbers in monkeys and humans. Psychol. Sci. 17, 401–406. doi: 10.1111/j.1467-9280.2006.01719.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Cordes, S., Gelman, R., Gallistel, C. R., and Whalen, J. (2001). Variability signatures distinguish verbal from nonverbal counting for both large and small numbers. Psychon. Bull. Rev. 8, 698–707. doi: 10.3758/BF03196206

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Cui, X. (2011). Hyperbolic discounting emerges from the scalar property of interval timing. Front. Integr. Neurosci. 5:24. doi: 10.3389/fnint.2011.00024

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Deco, G., and Rolls, E. T. (2006). Decision-making and Weber's law: a neurophysiological model. Eur. J. Neurosci. 24, 901–916. doi: 10.1111/j.1460-9568.2006.04940.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Dehaene, S. (2003). The neural basis of the Weber-Fechner law: a logarithmic mental number line. Trends Cogn. Sci. 7, 145–147. doi: 10.1016/S1364-6613(03)00055-X

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Dehaene, S., and Brannon, E. (eds.). (2011). Space, Time and Number in the Brain, 1st Edn. San Diego, CA: Academic Press; Elsevier Inc.

Donner, K., Hemilä, S., and Koskelainen, A. (1998). Light adaptation of cone photoresponses studied at the photoreceptor and ganglion cell levels in the frog retina. Vis. Res. 38, 19–36. doi: 10.1016/S0042-6989(97)00144-2

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Droit-Volet, S. (2013). Emotion and magnitude perception: number and length bisection. Front. Neurorobot. 7:24. doi: 10.3389/fnbot.2013.00024

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Forrest, T. G. (1994). From sender to receiver: propagation and environmental effects on acoustic signals. Am. Zool. 34, 644–654. doi: 10.1093/icb/34.6.644

CrossRef Full Text | Google Scholar

Frederick, S., Loewenstein, G., Donoghue, T. O., and Donoghue, T. E. D. O. (2002). Time discounting and time preference: a critical review. J. Econ. Lit. 40, 351–401. doi: 10.1257/jel.40.2.351

CrossRef Full Text | Google Scholar

Gallistel, C. R. (2011). “Mental magnitudes,” in Space, Time and Number in the Brain, eds S. Dehaene and E. Brannon (San Diego, CA: Academic Press; Elsevier Inc.), 3–12. doi: 10.1016/B978-0-12-385948-8.00001-3

CrossRef Full Text | Google Scholar

Gibbon, J. (1977). Scalar expectancy theory and Weber's law in animal timing. Psychol. Rev. 84, 279–325. doi: 10.1037/0033-295X.84.3.279

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gibbon, J., Church, R. M., and Meck, W. H. (1984). “Scalar timing in memory,” in Timing and Time Perception, eds J. Gibbon and L. Allan (New York, NY: New York Academy of Sciences), 52–77.

Pubmed Abstract | Pubmed Full Text | Google Scholar

Huber, J., Payne, J. W., and Puto, C. (1982). Adding asymmetrically dominated alternatives: violations of regularity and the similarity hypothesis. J. Consum. Res. 9, 90–98. doi: 10.1086/208899

CrossRef Full Text | Google Scholar

Kacelnik, A., and Bateson, M. (1996). Risky theories—the effects of variance on foraging decisions. Integr. Comp. Biol. 36, 402–434. doi: 10.1093/icb/36.4.402

CrossRef Full Text | Google Scholar

Kacelnik, A., and Brito e Abreu, F. (1998). Risky choice and Weber's law. J. Theor. Biol. 194, 289–298. doi: 10.1006/jtbi.1998.0763

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Kalenscher, T., and Pennartz, C. M. A. (2008). Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making. Prog. Neurobiol. 84, 284–315. doi: 10.1016/j.pneurobio.2007.11.004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Killeen, P. R., Cate, H., and Tran, T. (1993). Scaling pigeons' choice of feeds: bigger is better. J. Exp. Anal. Behav. 60, 203–217. doi: 10.1901/jeab.1993.60-203

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Louie, K., Khaw, M. W., and Glimcher, P. W. (2013). Normalization is a general neural mechanism for context-dependent decision making. Proc. Natl. Acad. Sci. U.S.A. 110, 6139–6144. doi: 10.1073/pnas.1217854110

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Matell, M. S., and Meck, W. H. (2000). Neuropsychological mechanisms of interval timing behavior. Bioessays 22, 94–103. doi: 10.1002/(SICI)1521-1878(200001)22:1%3C94::AID-BIES14%3E3.0.CO;2-E

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Matthews, H. R., Fain, G. L., Murphy, R. L., and Lamb, T. D. (1990). Light adaptation in cone photoreceptors of the salamander: a role for cytoplasmic calcium. J. Physiol. 420, 447–469.

Pubmed Abstract | Pubmed Full Text | Google Scholar

Namboodiri, V. M. K., Mihalas, S., and Hussain Shuler, M. G. (2014a). Rationalizing decision-making: understanding the cost and perception of time. Timing Time Percept. Rev. 1.

Namboodiri, V. M. K., Mihalas, S., Marton, T. M., and Hussain Shuler, M. G. (2014b). A general theory of intertemporal decision-making and the perception of time. Front. Behav. Neurosci. 8:61. doi: 10.3389/fnbeh.2014.00061

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Nieder, A., and Miller, E. K. (2003). Coding of cognitive magnitude: compressed scaling of numerical information in the primate prefrontal cortex. Neuron 37, 149–157. doi: 10.1016/S0896-6273(02)01144-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Rieke, F., Warland, D., van Steveninck, R., and Bialek, W. (1999). Spikes: Exploring the Neural Code. Cambridge, MA: The MIT Press.

Rovamo, J., Mustonen, J., and Näsänen, R. (1995). Neural modulation transfer function of the human visual system at various eccentricities. Vis. Res. 35, 767–774. doi: 10.1016/0042-6989(94)00171-H

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Shouval, H., Agarwal, A., and Gavornik, J. (2013). Scaling of perceptual errors can predict the shape of neural tuning curves. Phys. Rev. Lett. 110:168102. doi: 10.1103/PhysRevLett.110.168102

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Simen, P., Balci, F., de Souza, L., Cohen, J. D., and Holmes, P. (2011). A model of interval timing by neural integration. J. Neurosci. 31, 9238–9253. doi: 10.1523/JNEUROSCI.3121-10.2011

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Stephens, D. W., and Krebs, J. R. (1986). Foraging Theory. Princeton, NJ: Princeton University Press.

Google Scholar

Treisman, M. (1966). A statistical decision model lor sensory discrimination which predicts Weber' s law and other sensory laws: some results of a computer. Percept. Psychophys. 1, 203–230. doi: 10.3758/BF03215786

CrossRef Full Text | Google Scholar

Walsh, V. (2003). A theory of magnitude: common cortical metrics of time, space and quantity. Trends Cogn. Sci. 7, 483–488. doi: 10.1016/j.tics.2003.09.002

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Weber, E. H. (1978). The Sense of Touch, 1st Edn. London: Academic Press.

Whalen, J., Gallistel, C. R., and Gelman, R. (1999). Nonverbal counting in humans: the psychophysics of number representation. Psychol. Sci. 10, 130–137. doi: 10.1111/1467-9280.00120

CrossRef Full Text | Google Scholar