AUTHOR=Clarke Alasdair D., Elsner Micha , Rohde Hannah 

TITLE=Where's Wally: the influence of visual salience on referring expression generation

JOURNAL=Frontiers in Psychology

VOLUME=Volume 4 - 2013

YEAR=2013

URL=https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2013.00329

DOI=10.3389/fpsyg.2013.00329

ISSN=1664-1078

ABSTRACT=Referring expression generation (REG) presents the converse problem to visual
search: Given a scene and a specified target, how does one generate a
description which would allow somebody else to quickly and accurately locate
the target?  Previous work in psycholinguistics and natural language processing
that has addressed this question identifies only a limited role for vision in
this task.  That previous work, which relies largely on simple scenes, tends to
treat vision as a pre-process for extracting feature categories that are
relevant to disambiguation.  However, the visual search literature suggests
that some descriptions are better than others at enabling listeners to search
efficiently within complex stimuli. This paper presents the results of a study
testing whether speakers are sensitive to visual features that allow them to
compose such `good' descriptions.  Our results show that visual properties
(salience,  clutter, area, and distance) influence REG for targets embedded in
images from the *Where's Wally?* books, which are an order of magnitude
more complex than traditional stimuli.  Referring expressions for large salient
targets are shorter than those for smaller and less salient targets, and 
targets within highly cluttered scenes are described using more words.
We also find that speakers are more likely to mention non-target landmarks that
are large, salient, and in close proximity to the target.  These findings
identfy a key role for visual salience in language production decisions and 
highlight the importance of scene complexity for REG.