Applications of distributed dialogue systems The KTH Connector

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Applications of distributed dialogue systems: The KTH Connector Jens Edlund Anna Hjalmarsson Aalborg, November 10 th, 2005 © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 1

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Why spoken interfaces? Speech is good because it is • Hands free • Eyes free • Intuitive – already known • Robust – e. g. redundancy, grounding • Flexible • Responsive • Efficient • … © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 2

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson What do they do? • Purpose — — — Problem solving Information seeking Transactions Control … • Initiative — — — System User Mixed • Modality — — Multimodal (input and/or output) Unimodal (input and/or output) • … © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 3

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson What else do they do? Spoken interfaces often replace or complement existing automated interfaces, for example: © 2005 CHIL KTH Transact ions, e. g. ATIS Call routing, Web forms (and sales reps) DTMF (and operators) ASIDE 2005, Aalborg, 2005 -11 -10 4

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Alternative interface systems • Speech is an alternative or substitute • Commonly built to be as good as or better than the corresponding system • Symmetry often required – what can be done with the original system should be doable with speech and vice versa © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 5

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Why spoken interfaces? • • © 2005 CHIL KTH Hands free Eyes free Intuitive – already known Robust – e. g. redundancy, grounding Flexible Responsive Efficient … ASIDE 2005, Aalborg, 2005 -11 -10 These aspects are often not exploited much in alternative interface systems 6

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson What metaphor to rely on? • The voice as an input device? —”You may use your voice to order” From a travel booking instruction in Swedish —”It didn’t give me any alternatives” From a post interview with call routing user • The computer as a human? —Problem: Turing test © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 7

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Summary • Speech can be used successfully as an alternative or complementary interface to other interfaces, particularly when hands and/or eyes are occupied, disabled, or otherwise impratcical to use • The advantages of speech promised by analogies to human-human communication may not be fully exploited in such domains © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 8

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson The

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Background: CHIL • CHIL: Computers in the Human Interaction Loop (EU funded, IP 506909) • The dialogue system as an unobtrusive conversational partner in a group of humans © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 10

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson A telephony based secretary Wide range of complexity • From answer phone… © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 11

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Dialogue setup Multimodal, multiparty, system barge-in © 2005 CHIL KTH Multiparty telephony ASIDE 2005, Aalborg, 2005 -11 -10 13

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson System highlights System may dial users Prosody enhanced endpointing System output is included in discourse model (incrementally) © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 14

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Research highlight: Responsiveness • When should we respond? • Turn yielding & turn holding cues: — Prosody — Gaze —… © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 15

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Research highlight: Incrementality • When can we respond? — After a ”long enough” silence? — At some semantic or syntactic completeness? — After any ”word”? — Anytime? • What has actually happened? — System and user barge-in — Keep track of what we say, as well as what the user says © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 16

Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Research highlight: Unobtrusiveness • With what should we respond? — Long prompts can be annoying — Short ones may be insufficient • How? — Efficiently or politely? — Speech, gesture, other? • Do we have to ”take turn”? — Backchannels — Grounding © 2005 CHIL KTH ASIDE 2005, Aalborg, 2005 -11 -10 17