AUTHOR=Ankener Christine S. , Sekicki Mirjana , Staudte Maria 

TITLE=The Influence of Visual Uncertainty on Word Surprisal and Processing Effort

JOURNAL=Frontiers in Psychology

VOLUME=9

YEAR=2018

URL=https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2018.02387

DOI=10.3389/fpsyg.2018.02387

ISSN=1664-1078

ABSTRACT=<p>A word’s predictability or surprisal, as determined by cloze probabilities or language models (<xref ref-type="bibr" rid="B8">Frank, 2013</xref>) is related to processing effort, in that less expected words take more effort to process (<xref ref-type="bibr" rid="B10">Hale, 2001</xref>; <xref ref-type="bibr" rid="B14">Lau et al., 2013</xref>). A word’s surprisal, however, may also be influenced by the non-linguistic context, such as visual cues: In the visual world paradigm (VWP), anticipatory eye movements suggest that listeners exploit the scene to predict what will be mentioned next (<xref ref-type="bibr" rid="B1">Altmann and Kamide, 1999</xref>). <italic>How</italic> visual context affects surprisal and processing effort, however, remains unclear. Here, we present a series of four studies providing evidence on how visually-determined probabilistic expectations for a spoken target word, as indicated by anticipatory eye movements, predict graded processing effort for that word, as assessed by a pupillometric measure (the <italic>Index of Cognitive Activity</italic>, ICA). These findings are a clear and robust demonstration that the non-linguistic context can immediately influence both lexical expectations, and surprisal-based processing effort.</p>