The Effects of Prosodic Features on the Interpretation

The Effects of Prosodic Features on the Interpretation of Clarification Ellipses Jens Edlund, David House and Gabriel Skantze Abstract The Problem Setting Experiment • Elliptical one-word clarification requests are • 8 subjects judged the meaning of Levels of understanding In this paper, the effects of prosodic features on the interpretation of elliptical clarification requests in dialogue are studied. An experiment is presented where subjects were asked to listen to short human-computer dialogue fragments in Swedish, where a synthetic voice was making an elliptical clarification after a user turn. The prosodic features of the synthetic voice were systematically varied, and the subjects were asked to judge what was actually intended by the computer. The results show that an early low F 0 peak signals acceptance, that a late high peak is perceived as a request for clarification of what was said, and that a mid high peak is perceived as a request for clarification of the meaning of what was said. • • potentially ambiguous • Little syntax and structure • Prosody more critical • How do prosodic features affect the interpretation of these utterances? one-word elliptical clarification requests in dialogue context • Task: Select paraphrase for elliptical system utterance • Swedish Allwood et al. (1992), Clark (1996) red, blue, yellow early, mid, late low, high normal, long • LUKAS diphone MBROLA synthesis Level Acceptance H accepts what S says Understanding H understands what S says Perception The Higgins spoken dialog system for pedestrian navigation System utterance: F 0 peak position: F 0 peak height: Vowel duration: = 36 stimuli H hears what S says Contact H hears that S speaks Results Ellipsis interpretation Errors and clarification in dialog • Dialog not always error free • Error detection often made using explicit or implicit spoken clarification/verification: User […] on the right I see a red building. System (low conf. ) Did you say ’A red building’? System (high conf. ) A red building… ok, take a left […]? Traditionally: • Constructed as full propositions • Often perceived as tedious • Clarifies entire user utterances Clarification Ellipses User System […] on the right I see a red building. red(? ) Advantages: • Fast • Focuses on problematic fragment • Often used in human-human dialog (max 48) Level Paraphrase Signal Acceptance Ok, red. Clarify Understanding Do you really mean red? Clarify Perception Did you say red? No effects for: • Subject • Color • Duration Prototypes: • Accept: Early low peak • Clarify Understanding: Mid high peak • Clarify Perception: Late high peak Question intonation • Clear distinction between two question intonations: perception and understanding Swedish question intonation • Raised top-line and widened F 0 range on focal accent (Gårding, 1998) • Delayed focal peak (House, 2003) German dialog • Rodriguez & Schlangen (2004) • Rising boundary tones to clarify acoustic problems (perception) • Used less for reference resolution (understanding) • Three distinct prototypes for different interpretations level Future work • Will be implemented and tested in Higgins to evaluate user responses and • • dialogue efficiency Corpus study of human-human dialog Multi-syllable, multi-word, accent II Back-channels (hmm, eh) Multimodal synthesis