OFAI

Technical Reports - Query Results

Your query term was 'number = 2014-07'
1 report found
OFAI-TR-2014-07 ( 598kB PDF file)

Exploring Inter- and Intra-speaker Variability in Multi-modal Task Descriptions

Stephanie Schreitter, Brigitte Krenn

In natural human-human task descriptions, the verbal and the non-verbal parts of communication together comprise the information necessary for understanding. When robots are to learn tasks from humans in the future, the detection and integrated interpretation of both of these cues is decisive. In the present paper, we present a qualitative study on essential verbal and non-verbal cues by means of which information is transmitted during explaining and showing a task to a learner. In order to collect a respective data set for further investigation, 16 (human) teachers explained to a human learner how to mount a tube in a box with holdings, and six teachers did this to a robot learner. Detailed multi-modal analysis revealed that in both conditions, information was more reliable when transmitted via verbal and gestural references to the visual scene and via eye gaze than via the actual wording. In particular, intra-speaker variability in wording and perspective taking by the teacher potentially hinders understanding of the learner. The results presented in this paper emphasize the importance of investigating the inherently multi-modal nature of how humans structure and transmit information in order to derive respective computational models for robot learners.

Citation: Schreitter S., Krenn B.: Exploring Inter- and Intra-speaker Variability in Multi-modal Task Descriptions. In Proceedings of the 17th IEEE International Symposium on the Robot and Human Interactive Communication, (Ro-Man 2014), 2014.