This article is a commentary on:
Detection of 3D curved trajectories: the role of binocular disparity
A commentary on
Detection of 3D curved trajectories: the role of binocular disparity
by Pierce, R. S., Bian, Z., Braunstein, M. L., and Andersen, G. J. (2013). Front. Behav. Neurosci. 7:12. doi: 10.3389/fnbeh.2013.00012
The perceptual inference of the three-dimensional (3D) external world from two-dimensional (2D) retinal input of the left and right eye is a fundamental problem (Berkeley, 1709/1975; von Helmholtz, 1910/1962) that the visual system has to solve through neural computation (Poggio et al., 1985; Pizlo, 2001). This is true for static scenes as well as for dynamic events. The inverse problem for binocular 3D motion perception implies that the visual system estimates object motion in 3D space from spatial-temporal processing of binocular input. How exactly the visual system achieves these representations is not well understood (Harris et al., 2008).
Velocity in 3D space is typically described by motion direction and speed. In computer vision local velocities are conveniently expressed as vectors in a 3D co-ordinate system. Establishing motion vectors is desirable because local estimates in a sufficiently dense vector field can provide the basis for segmenting an object from its background, for identifying an object, and for planning and executing actions in a dynamic environment. However, a “straight” or linear vector representation can be problematic when trying to capture the perception of curved or non-linear motion.
In the study by Pierce et al. (2013) observers discriminated curved (convex and concave) from straight (linear) motion trajectories of an object moving horizontally through depth. Although human discrimination performance may depend on subtle aspects of the viewing geometry, smooth pursuit, stimulus characteristics and accommodation cues of individual observers (Nefs et al., 2010; Heron and Lages, 2012) the results by Pierce et al. clearly suggest that binocular input facilitates the detection of curved trajectories in depth.
Interocular velocity difference (IOVD) or changing disparity over time (CDOT) are both binocular inputs that may have improved discrimination performance in this task. However, motion detection and therefore IOVD is relatively insensitive to velocity changes (Gottsdanker, 1956; Calderone and Kaiser, 1989; Lisberger and Movshon, 1999) whereas disparity processing and therefore CDOT needs to be coupled with spatio-temporal information to discern a 3D motion trajectory. As a consequence deterministic models of motion perception, such as IOVD and CDOT, may be too limited to match the wealth of perceived motions. It also seems unlikely that the visual system has developed early joint detectors that are tuned to all possible combinations of spatial and temporal frequency, orientation, disparity—as well as curvature—to solve the inverse problem of 3D motion (Lages et al., 2007; Lages and Heron, 2010). Instead the visual system may rely on binocular motion constraints of less specialized detectors in concert with binocular disparity input to capture non-linear 3D motion trajectories.
Early motion and disparity processing in the human visual system show tuning characteristics that supplement each other. Motion processing tends to have high temporal but relatively coarse spatial resolution whereas disparity processing has high spatial and relatively limited temporal resolution (Tyler, 1971; Lappin et al., 2009). It has been suggested that both, early motion and disparity detection, contribute to 3D motion perception and that this information is integrated late in the visual hierarchy (Likova and Tyler, 2007; Lages and Heron, 2008, 2010). At an early level of processing local motion detectors may encode motion constraints so that binocular motion processing remains ambiguous in terms of local motion direction but provides flexible spatio-temporal constraints (Lages and Heron, 2010). Disparity processing on the other hand offers fine spatial detail and depth information to disambiguate motion trajectories. Following the principle of least commitment (Marr, 1976) the two processing streams may define a characteristic spatio-temporal window for 3D motion perception (Tyler, 1971; Lages et al., 2003).
In an influential paper Weiss et al. (2002) demonstrated that 2D motion perception, modeled as a probabilistic inference, results in not necessarily veridical representations of physical stimulus motion (see also Ji and Fermuller, 2006). They proposed a Bayesian model of 2D motion perception that is based on likelihood constraints and a slow motion prior to explain a range of 2D motion illusions. Similarly, Lages (2006) suggested a Bayesian 3D motion model where binocular motion constraints can explain perceptual bias of horizontal motion trajectories in depth (Harris and Dean, 2003).
A geometric-statistical model based on motion and disparity constraints provides a flexible computational framework for binocular 3D motion perception that can capture arbitrary 3D trajectories of moving features and objects. For example, motion and disparity information needs to be combined to disambiguate local motion direction of a single line or edge moving in depth (see Figure 1 for an illustration). Uncertain and ambiguous motion input poses a particular problem for deterministic models because early encoding makes it difficult to explain perceptual bias and to adjust local motion vectors when additional information about the motion path becomes available (e.g., shading, angular size, texture, occlusion, endpoints). In a probabilistic approach binocular constraints can be established in terms of velocity and (dynamic) disparity likelihoods where updates of disparity constraints at intermediate positions help to disambiguate 3D motion trajectories. If the stimulus has unique features such as moving endpoints, corners, or texture elements then their trajectories can be “tracked” in depth over time. If, however, there are only uncertain or ambiguous moving features then a weak prior favoring slow motion (or short displacement in 3D space) suggests a linear trajectory as the default solution (see Figure 1; Lages et al., 2013). Such a prior may reflect tuning characteristics of a population of binocular motion cells and can explain perceptual bias. In general, the spatio-temporal features of a moving stimulus will influence how the 3D motion system combines and disambiguates motion and disparity information. Where exactly in the brain motion and disparity constraints are integrated is a matter of ongoing research (V3B/KO, hMT+/V5, occipital-temporal region; Likova and Tyler, 2007; Rokers et al., 2009; Ban et al., 2012). Importantly however, neural activation specific to motion and disparity processing seems to occur late rather than early in the visual hierarchy.
FIGURE 1
Figure 1. Illustration of a geometric-statistical model for perceiving an oriented line or edge moving in 3D. The left (LE) and right eye (RE), separated by interocular distance i are fixated on point F at distance D. If there are only velocity constraints (shaded triangles) a probabilistic 3D motion model estimates a linear trajectory (straight arrow from F to G) using a weak motion prior favoring slow motion (sphere centered on F). Binocular disparity processing provides additional information in form of intersections of constraints (IOCs; oriented lines) or texture elements (open dots) that help to disambiguate a concave motion path of the stimulus line in depth (curved arrow from F to G).
The observations by Pierce et al. (2013) demonstrate that binocular information processing facilitates discrimination of curved trajectories of a moving object. Perception of lines or contours of an object moving on a curved or non-linear path requires integration of binocular motion and disparity information. This is difficult to achieve in a deterministic framework where monocular motion vectors are linear, may not match up, and where disparity input indicates changed depth over time but no motion. In a probabilistic framework however, motion constraints can be combined with binocular disparity constraints and a motion prior to model the not necessarily veridical perception of curved trajectories.
Acknowledgments
Supported by a Leverhulme Trust research grant (F00-179/BG).
References
Ban, H., Preston, T. J., Meeson, A., and Welchman, A. E. (2012). The integration of motion and disparity cues to depth in dorsal visual cortex. Nat. Neurosci. 15, 636–643. doi: 10.1038/nn.3046
Berkeley, G. (1709/1975). Philosophical Works: Including the Works on Vision, ed M. Ayers (London, Dent.)
Calderone, J. B., and Kaiser, M. K. (1989). Visual acceleration detection: effect of sign and motion orientation. Percept. Psychophys. 45, 391–394. doi: 10.3758/BF03210711
Gottsdanker, R. M. (1956). The ability of human operators to detect acceleration of target motion. Psychol. Bull. 53, 477–487. doi: 10.1037/h0045160
Harris, J. M., and Dean, P. J. (2003). Accuracy and precision of binocular 3-D motion perception. J. Exp. Psychol. Hum. Percept. Perform. 29, 869–881. doi: 10.1037/0096-1523.29.5.869
Harris, J. M., Nefs, H. T., and Grafton, C. E. (2008). Binocular vision and motion in depth. Spat. Vis. 21, 531–547. doi: 10.1163/156856808786451462
Heron, S., and Lages, M. (2012). Screening and sampling in binocular vision studies. Vision Res. 62, 228–234. doi: 10.1016/j.visres.2012.04.012
Ji, H., and Fermuller, C. (2006). Noise causes slant underestimation in stereo and motion. Vision Res. 46, 3105–3120. doi: 10.1016/j.visres.2006.04.010
Lages, M. (2006). Bayesian models of binocular 3-D motion perception. J. Vis. 6, 508–522. doi: 10.1167/6.4.14
Lages, M., Dolia, A., and Graf, E. W. (2007). Dichoptic motion perception limited to depth of fixation. Vision Res. 47, 244–252. doi: 10.1016/j.visres.2006.10.001
Lages, M., and Heron, S. (2008). Motion and disparity processing informs Bayesian 3D motion estimation. Proc. Natl. Acad. Sci. U.S.A. 105, E117. doi: 10.1073/pnas.0809829105
Lages, M., and Heron, S. (2010). On the inverse problem of local binocular 3D motion perception. PLoS Comput. Biol. 6:e1000999. doi: 10.1371/journal.pcbi.1000999
Lages, M., Heron, S., and Wang, H. (2013). “Local constraints for the perception of binocular 3D motion,” in Developing and Applying Biologically-Inspired Vision Systems: Interdisciplinary Concepts, eds M. Pomplun and J. Suzuki (New York, NY: IGI Global), 90–120
Lages, M., Mamassian, P., and Graf, E. W. (2003). Spatial and temporal tuning of motion in depth. Vision Res. 43, 2861–2873. doi: 10.1016/j.visres.2003.08.006
Lappin, J. S., Tadin, D., Nyquist, J. B., and Corn, A. L. (2009). Spatial and temporal limits of motion perception across variations in speed, eccentricity, and low vision. J. Vis. 9, 30.1–14.
Likova, L. T., and Tyler, C. W. (2007). Stereomotion processing in the human occipital cortex. Neuroimage 38, 293–305. doi: 10.1016/j.neuroimage.2007.06.039
Lisberger, S. G., and Movshon, J. A. (1999). S. G. Visual motion analysis for pursuit eye movements in area MT of macaque monkeys. J. Neurosci. 19, 2224–2246.
Marr, D. (1976). Early processing of visual information. Philos. Trans. R. Soc. Lond. B Biol. Sci. 275, 483–519.
Nefs, H. T., O'Hare, L., and Harris, J. M. (2010). Two independent mechanisms for motion-in-depth perception: evidence from individual differences. Front. Psychol. 1:155. doi: 10.3389/fpsyg.2010.00155
Pierce, R. S., Bian, Z., Braunstein, M. L., and Andersen, G. J. (2013). Detection of 3D curved trajectories: the role of binocular disparity. Front. Behav. Neurosci. 7:12. doi: 10.3389/fnbeh.2013.00012
Pizlo, Z. (2001). Perception viewed as an inverse problem. Vision Res. 41, 3145–3161. doi: 10.1016/S0042-6989(01)00173-0
Poggio, T., Torre, V., and Koch, C. (1985). Computational vision and regularization theory. Nature 317, 314–319. doi: 10.1038/317314a0
Rokers, B., Cormack, L. K., and Huk, A. C. (2009). Disparity- and velocity-based signals for three-dimensional motion perception in human MT+. Nat. Neurosci. 12, 1050–1055. doi: 10.1038/nn.2343
Tyler, C. (1971). Stereoscopic depth movement: two eyes less sensitive than one. Science 174, 958–961. doi: 10.1126/science.174.4012.958
von Helmholtz, H. (1910/1962). Helmholtz's Treatise on Physiological Optics, ed J. P. Southall (New York, NY: Dover), 312–313.
Citation: Lages M (2013) Straight or curved? From deterministic to probabilistic models of 3D motion perception. Front. Behav. Neurosci. 7:79. doi: 10.3389/fnbeh.2013.00079
Received: 20 May 2013; Accepted: 17 June 2013;
Published online: 05 July 2013.
Edited by:
Hong-Jin Sun, McMaster University, CanadaCopyright © 2013 Lages. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.
*Correspondence: martin.lages@glasgow.ac.uk