Marius Usher
Konstantinos Tsetsos
Erica C. Yu
David A. Lagnado- 1Department of Psychology and Sagol School of Neuroscience, Tel-Aviv University, Ramat-Aviv, Israel
- 2Department of Experimental Psychology, University of Oxford, Oxford, UK
- 3Department of Psychology, University of Maryland, Maryland, MD, USA
- 4Department of Cognitive, Perceptual and Brain Science, University College London, London, UK
Decision-making is a dynamic process that begins with the accumulation of evidence and ends with the adjustment of belief. Each step is itself subject to a number of dynamic processes, such as planning, information search and evaluation. Furthermore, choice behavior reveals a number of challenging patterns, such as order effects and contextual preference reversal. Research in this field has converged toward a standard computational framework for the process of evidence integration and belief updating, based on sequential sampling models, which under some conditions are equivalent to normative Bayesian theory (Gold and Shadlen, 2007). A variety of models have been developed within the sequential sampling framework that can account for accuracy, response-time distributional data, and the speed-accuracy trade-off (Busemeyer and Townsend, 1993; Usher and Mcclelland, 2001; Brown and Heathcote, 2008; Ratcliff and McKoon, 2008). Yet there are differences between these models with regard to the mechanism of decision-termination, the optimality of the decision and the temporal weighting of the evidence. There is also a need to extend this framework to preference type of decisions (where the criteria are up to the judge) and to enrich it so as to include control processes (such as exploration/exploitation), information search, and adaptation to the environment, thereby allowing it to capture richer decision problems; for example, when alternatives are not pre-defined, or when the decision-maker is not just accumulating evidence but also adapting beliefs about the data-generating process.
This Research Topic presents new work that investigates the dynamical and mathematical properties of evidence integration and its neural mechanisms and extends this framework to more complex decisions, such as those that occur during risky choice, preference formation, and belief updating. We hope these articles will encourage researchers to explore the computational and normative aspects of the decision process and the observed deviations. We briefly review here the contributions in this collection, starting from simple perceptual decisions in which the information flow is externally controlled to more complex decisions, which allow the observer to control the information flow and other learning strategies, and following on with preference formation.
Fast Perceptual Decisions
The first group of seven articles examines issues that arise in fast perceptual decisions that only allow the subject to control the weighting of the incoming evidence and the termination rule. Nevertheless, the integration time-scale, the temporal weights, and evidence termination can vary and this strongly affects the decision performance (how close people are to optimality) and the fit with the data. Some of these papers also examine the neural mechanisms that implement the decisions. In a mathematically-oriented paper Heathcote and Love (2012) examine a variant of a race model (the linear ballistic accumulator; LBA), which, under certain assumptions about the underlying distributions of starting point and drift-rate variability of evidence accumulation, allows for closed analytical formulas for the full response-time distribution in a lexical decision task and obtains a goodness of fit almost as good as that of the standard LBA model. In another formal paper, van Ravenzwaaij et al. (2012) examine, within the standard drift-diffusion model, the optimality of evidence accumulation strategies in decision situations with unequal frequency of stimuli types. They converge on the result that a bias in the decision starting point is optimal, in both fixed and variable difficulty conditions, though it appears that observers do not fully follow this strategy (but see Moran and Usher, in preparation). In another paper that examines sequential effects and decision biases in binary choice tasks, Goldfarb et al. (2012) present a simple extension of the standard decision model, which assumes changes in starting point depend on stimulus repetitions and alternations, combined with a response criteria increase following errors. This model accounts for a rich data of sequential dependence in response time and accuracy. In a paper contributed by Tsetsos et al. (2012), the aim was to contrast the standard drift-diffusion algorithm, which assumes that the evidence is given temporally-uniform decision weights, and the leaky competing accumulator model (LCA), which predicts a variety of temporal weighting patterns, including (for some model parameters) a specific interaction between stimulus duration and temporal weighting. While the LCA-predicted interaction was confirmed in some of the observers (who performed multiple sessions with the moving dots displays), future work will be needed to further characterize how temporal weighting of evidence depends on task characteristics and individual differences. The issue of temporal weighting and its dependence on characteristics of evidence accumulation and type of decision-boundary is further discussed in a review paper by Zhang (2012), who also examines how these characteristics affect decision optimality. Lastly, two papers discuss the neural mechanisms of perceptual decisions. Simen (2012) examines a two-layer neural model that includes accumulators and bistable cell-assemblies that can implement the decision-boundary—which is assumed without much discussion in the standard approach—and discusses the difficulties of mapping those processing units to the neural recordings observed in brain data. van Vugt et al. (2012) use a model-driven approach to reveal the EEG correlates of evidence accumulation for a motion discrimination task. The authors use a novel computational technique to show that the time-course of the EEG activity demonstrated a non-linear profile—a finding that may arbitrate the dispute between linear (e.g., Brown and Heathcote, 2008) and non-linear (e.g., Usher and Mcclelland, 2001) models of evidence integration. Moreover, this paper indicates the possibility of identifying individual differences in evidence integration (e.g., speed-accuracy trade-off) from the EEG signal, offering a useful tool for characterizing the computational properties of the decision mechanism.
Adaptive Decision Making
The second group of six articles examines decisions that extend over a longer time-frame and which allow the subject to control the evidence accumulation process, and to form and update beliefs about the state of the environment. The study by Knox et al. (2011) of exploration and exploitation suggests that human decision makers learn from interaction with their environment in a reflective manner (without requiring direct observation of changes in the environment) but yet do not plan optimally because they do not consider the long-term information value of actions. The contribution by Osman and Speekenbrink (2012) extends this inquiry by studying how knowledge about the values of actions can be affected by tasks of prediction (outcome estimation) and control (interventions to achieve an outcome). They demonstrate a distinction between prediction and control whereby controllers were able to transfer their knowledge to tests of prediction but not vice versa. In this way, the concept of control is similar to that of planning for a goal rather than for adapting to an environment as in Knox et al. (2011) but, in both of these papers, decision makers cycle from evidence accumulation, to action, to feedback, and back again (cf. model-based learning; Sutton and Barto, 1998). Yu and Lagnado (2012) use this framework in a slot machine paradigm to show that, while participants over time came to understand the observed environment (slot machine payouts) accurately, their understanding of the underlying structure of the environment was flawed. Beliefs about structure and causality were more strongly influenced by initial beliefs than by experience. Also studying decisions from experience, Dutt and Gonzalez (2012) explore the role of inertia, or the tendency to repeat one's final decision, irrespective of its outcome. They show both the advantages and disadvantages of incorporating inertia into an instance-based learning model of repeated binary choice. In contrast to this focus on inertia, Lange et al. (2012) demonstrate how decision makers adapt to the environment, using a new model that combines the HyGene model (Thomas et al., 2008) with the context-activation model (Davelaar et al., 2005). Across two experiments that manipulate serial order, consistency of newly-acquired evidence with previously-generated hypotheses, and elicitation timing, the authors show that not all data have an equal impact on hypothesis generation processes: newly-acquired data can cause inconsistent hypotheses to be purged from working memory. The authors propose that whether this results in a recency or primacy effect is likely to depend on the richness of the information and its rate of presentation.
Preference-Based Decisions
The next group of three articles addresses preference formation, in situations (risky choice and multi-attribute decisions) that do not set up an objective/normative criterion, but rather leaves this to the subject's control. Fiedler and Glockner (2012) monitor how people choose between lotteries using eye-tracking to distinguish between competing models of risky choice. The results disconfirmed Take-the-Best, or lexicographic heuristics, in favor of compensatory models that assume observers integrate outcomes with attentional weights determined by outcome probability. In particular, people gather more information within (rather than between) lotteries and they tend to gather more information (toward the end of the decision) from the chosen alternative, indicating top–down feedback from alternative to processing representations. Also using eye-tracking, Krajbich et al. (2012) propose a formalization of the influence of visual fixations on the dynamics of preference formation. The authors build on the attentional diffusion model (aDDM), which modulates the rate of evidence-accumulation depending on the position of visual fixation, to explain the responses and reaction times of human subjects during purchasing (accept/reject) decisions. The study demonstrates how small attentional fluctuations during the deliberation period can influence the decision outcome. This approach is closely related to theoretical models of multi-attribute choice [e.g., decision field theory, Roe et al. (2001); and value-based LCA, Usher and Mcclelland (2004)], in which attentional switching to different choice aspects drives preference formation. This class of models is extended in Wollschlager and Diederich (2012), which presents a novel model of contextual preference reversal (attraction, similarity, and compromise effects) for multi-alternative, multi-attribute choice: the 2N-ary Choice Tree model. The model offers closed-form expressions for choice probabilities and response time distributions and, contrary to previous theories, explains reversal effects by assuming that attentional weights depend on the alternatives in the choice-set [cf. a recent study, which appeared after this Research Topic and provides an explicit mechanism for how the alternatives affect weights to the choice attributes: Bhatia (2013)].
Novel or Integrative Approaches
Finally, two papers aim to provide novel or integrative frameworks for understanding dynamical decision making. Trueblood and Busemeyer (2012) present a decision model based on principles of quantum theory—a radical shift from the standard framework—which provides a novel account of order effects in belief updating and inference. This paper provides an introduction to the elegant principles of quantum probability. This theory is of great potential, although stronger data might be needed to persuade the skeptical readers (e.g., showing cyclic changes in order effects). In the last paper Fox et al. (2013) present an overarching framework for the entire decision making cycle, from the framing of a decision to establishing preferences and making commitments. They extend the standard model to more ecological and dynamic situations, in which the alternatives are not predefined and the agent faces a variety of constraints and conflicts. The theory situates dynamical decision making with respect to other high-level cognitive capabilities such as problem solving, planning and collaborative decision-making.
Conclusions and Future Work
We believe that this collection has revealed a number of important aspects of the nature of decision processes. More importantly, we hope that it will stimulate readers to keep probing these processes. Various key questions are still unresolved. How close are people to optimality when making decisions? Why does this vary so much between the cases of evidence and preference? Is the Bayesian framework a general one for all types of decisions (can one extend it to more complex cases that allow the subject control over the information flow and the decision criteria?). What are the neural mechanisms, and the nature of individual differences? Future research into these topics, should surely keep us stimulated for the near future.
Acknowledgments
We want to thank Eddy J. Davelaar for very helpful Editorial suggestions on this paper. Marius Usher is supported by the Israeli Science Foundation (Grant 743/12).
References
Bhatia, S. (2013). Associations and the accumulation of preference. Psychol. Rev. 120, 522–543. doi: 10.1037/a0032457
Brown, S. D., and Heathcote, A. (2008). The simplest complete model of choice response time: linear ballistic accumulation. Cogn. Psychol. 57, 153–178. doi: 10.1016/j.cogpsych.2007.12.002
Busemeyer, J. R., and Townsend, J. T. (1993). Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. Psychol. Rev. 100, 432–459. doi: 10.1037/0033-295X.100.3.432
Davelaar, E. J., Goshen-Gottstein, Y., Ashkenazi, A., Haarmann, H. J., and Usher, M. (2005). The demise of short-term memory revisited: empirical and computational investigations of recency effects. Psychol. Rev. 112, 3–42. doi: 10.1037/0033-295X.112.1.3
Dutt, V., and Gonzalez, C. (2012). The role of inertia in modeling decisions from experience with instance-based learning. Front. Psychol. 3:177. doi: 10.3389/fpsyg.2012.00177
Fiedler, S., and Glockner, A. (2012). The dynamics of decision making in risky choice: an eye-tracking analysis. Front. Psychol. 3:335. doi: 10.3389/fpsyg.2012.00335
Fox, J., Cooper, R. P., and Glasspool, D. W. (2013). A canonical theory of dynamic decision-making. Front. Psychol. 4:150. doi: 10.3389/fpsyg.2013.00150
Gold, J. I., and Shadlen, M. N. (2007). The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574. doi: 10.1146/annurev.neuro.29.051605.113038
Goldfarb, S., Wong-Lin, K., Schwemmer, M., Leonard, N. E., and Holmes, P. (2012). Can post-error dynamics explain sequential reaction time patterns? Front. Psychol. 3:213. doi: 10.3389/fpsyg.2012.00213
Heathcote, A., and Love, J. (2012). Linear deterministic accumulator models of simple choice. Front. Psychol. 3:292. doi: 10.3389/fpsyg.2012.00292
Knox, W. B., Otto, A. R., Stone, P., and Love, B. C. (2011). The nature of belief-directed exploratory choice in human decision-making. Front. Psychol. 2:398. doi: 10.3389/fpsyg.2011.00398
Krajbich, I., Lu, D., Camerer, C., and Rangel, A. (2012). The attentional drift-diffusion model extends to simple purchasing decisions. Front. Psychol. 3:193. doi: 10.3389/fpsyg.2012.00193
Lange, N. D., Thomas, R. P., and Davelaar, E. J. (2012). Temporal dynamics of hypothesis generation: the influences of data serial order, data consistency, and elicitation timing. Front. Psychol. 3:215. doi: 10.3389/fpsyg.2012.00215
Osman, M., and Speekenbrink, M. (2012). Prediction and control in a dynamic environment. Front. Psychol. 3:68. doi: 10.3389/fpsyg.2012.00068
Ratcliff, R., and McKoon, G. (2008). The diffusion decision model: theory and data for two-choice decision tasks. Neural Comput. 20, 873–922. doi: 10.1162/neco.2008.12-06-420
Roe, R. M., Busemeyer, J. R., and Townsend, J. T. (2001). Multialternative decision field theory: a dynamic connectionist model of decision making. Psychol. Rev. 108, 370–392. doi: 10.1037/0033-295X.108.2.370
Simen, P. (2012). Evidence accumulator or decision threshold - which cortical mechanism are we observing? Front. Psychol. 3:183. doi: 10.3389/fpsyg.2012.00183
Sutton, R. S., and Barto, A. G. (1998). Introduction to Reinforcement Learning. Cambridge: MIT Press.
Thomas, R. P., Dougherty, M. R., Sprenger, A. M., and Harbison, J. I. (2008). Diagnostic hypothesis generation and human judgment. Psychol. Rev. 115, 155–185. doi: 10.1037/0033-295X.115.1.155
Trueblood, J. S., and Busemeyer, J. R. (2012). A quantum probability model of causal reasoning. Front. Psychol. 3:138. doi: 10.3389/fpsyg.2012.00138
Tsetsos, K., Gao, J., Mcclelland, J. L., and Usher, M. (2012). Using time-varying evidence to test models of decision dynamics: bounded diffusion vs. the leaky competing accumulator model. Front. Neurosci. 6:79. doi: 10.3389/fnins.2012.00079
Usher, M., and Mcclelland, J. L. (2001). The time course of perceptual choice: the leaky, competing accumulator model. Psychol. Rev. 108, 550–592. doi: 10.1037/0033-295X.108.3.550
Usher, M., and Mcclelland, J. L. (2004). Loss aversion and inhibition in dynamical models of multialternative choice. Psychol. Rev. 111, 757–769. doi: 10.1037/0033-295X.111.3.757
van Ravenzwaaij, D., Mulder, M. J., Tuerlinckx, F., and Wagenmakers, E. J. (2012). Do the dynamics of prior information depend on task context? An analysis of optimal performance and an empirical test. Front. Psychol. 3:132. doi: 10.3389/fpsyg.2012.00132
van Vugt, M. K., Simen, P., Nystrom, L. E., Holmes, P., and Cohen, J. D. (2012). EEG oscillations reveal neural correlates of evidence accumulation. Front. Neurosci. 6:106. doi: 10.3389/fnins.2012.00106
Wollschlager, L. M., and Diederich, A. (2012). The 2N-ary choice tree model for N-alternative preferential choice. Front. Psychol. 3:189. doi: 10.3389/fpsyg.2012.00189
Yu, E. C., and Lagnado, D. A. (2012). The influence of initial beliefs on judgments of probability. Front. Psychol. 3:381. doi: 10.3389/fpsyg.2012.00381
Keywords: decision making, belief, evidence accumulation, data-generating process, problem solving
Citation: Usher M, Tsetsos K, Yu EC and Lagnado DA (2013) Dynamics of decision-making: from evidence accumulation to preference and belief. Front. Psychol. 4:758. doi: 10.3389/fpsyg.2013.00758
Received: 15 August 2013; Accepted: 27 September 2013;
Published online: 18 October 2013.
Edited by:
Eddy J. Davelaar, Birkbeck College, UKCopyright © 2013 Usher, Tsetsos, Yu and Lagnado. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: marius@post.tau.ac.il