• Seminars & Events
  • Latest
  • Archive
  • RSS Feed


22 July 2010

Cross-Cultural Evaluation of an Algorithm for the Generation of Multimodal Referring Expressions

Location: Meeting Room 10, 2nd Floor, JLB
Time: 12:30pm - 13:45pm
Speaker(s): Ielka van der Sluis - Department of Computer Science, at the Trinity College Dublin

An issue addressed in many Human Computer Interaction systems is that of identifying a certain object in a visual context accessible to both user and system. This can be done, for instance,  by an Embodied Conversational Agent that points to the object in combination with uttering a linguistic referring expression. The generation of referring expressions (GRE) is a central task in Natural Language Generation, and various algorithms which automatically produce referring expressions have been developed. The multimodal GRE algorithm that was taken as a starting point for the work presented in this talk, approaches GRE as a compositional task in which language and gestures are combined in a natural way. For the evaluation of this algorithm a setting in Second Life was built, and the referring expressions in question form part of a dialogue script “acted out” by two virtual agents in a furniture shop. This setup was created to enable us to assess perceptions of a scene by English- and Japanese-speaking subjects with respect to the naturalness of the dialogue and of the behaviour of virtual agents, among other things. The talk will highlight (1) the suitability of the setting, (2) the design of our evaluation method, (3) the process of translating the dialogues from English into Japanese and (4) some results of the cross-cultural evaluation that was carried out at Trinity College Dublin and the National Institute of Informatics at Tokyo.    KEYWORDS: Embodied Conversational Agents, Automatic Behaviour Generation, Generation of Multimodal Referring Expressions, Virtual Worlds, Translation, Cross-cultural differences

Save to your Calendar

Contact: mailto:r.power@open.ac.uk