Applying the Wizard-of-Oz Framework to Cooperative Service Discovery

Applying the Wizard-of-Oz Framework to Cooperative Service Discovery and Configuration Anders Green Helge Hüttenrauch Kerstin Severinson Eklundh KTH NADA Interaction and Presentation Laboratory 100 44 Stockholm, SWEDEN E-mail: {green, hehu, kse}@nada. kth. se http: //www. nada. kth. se/iplab/hri COGNIRON FP 6 -002020 13 th IEEE International Workshop on Robots and Human Interactive Communications September 20 -22, 2004, Kurashiki, Japan 1

The Wizard-of-Oz method - peeking into the future: • Evaluate high-level functions of interactive systems trough simulation • Study the “real” use of service robots • Ordinary users – Users are not robotic researcchers • Empirical Data on human-robot communication. U: Follow me (2) ? R: Robot is following (5) U: Robot, this is an orange (14) R: Found object (25) U: Robot go to the book (31) R: Ok (42) Better pic? http: //www. nada. kth. se/iplab/hri 2

Design phases • Scenario description § KE 1: Home-tour • Dialog patterns § SI G Show, Follow, Goto, Find, etc. • Task coverage & Habituation § Task-oriented dialogue § Feedback dialogue § Error handling dialogue • User instruction § Wizard ”instruction” and tool development Pilot trials? DE • User trial & data collection • Analysis and annotation http: //www. nada. kth. se/iplab/hri HA BI T N UA TI O N DA T CO A & A L L EC NA T I O L YS N IS 3

Scenario description: Home Tour Project scenario description: User instruction: • The robot is delivered to a family home. • After system set-up, a family member teaches the Topology geometry of the house, i. e. specific locations, objects and artefacts, to the robot. • The robot can engage in a dialogue in case of missing or ambiguous information. Training a service robot Imagine that you have bought a service robot that is to perform simple duties in your home. Maybe you have broken your leg while when you went skiing or maybe you are waiting to get your hip joints replaced. Anyway, the robot is here, fresh out of the box and now you have to train it to suit your environment. http: //www. nada. kth. se/iplab/hri 4

Habituation process • • Dialogue patterns § Support response generation Interaction constraints § communicated to user § maintained by wizard • Tasks § Introduction. • Hello robot Greeting/Attention § Demonstration • Show objects and locations § Validation • • Find/Search objects Goto location § Closing the interaction • Good byezfc U: Follow me COOOL! (Robot starts person following) R: Following Wrong: robot starts moving after the robot’s utterance! User monitors 1. 0 – 3 m U: Stop (Robot stops person following) R: Stop following. Error handling http: //www. nada. kth. se/iplab/hri 5

Scenario model User instruction Svårt att läsa? Show a place by using the follow me command. Once the robot is at the right position you can define the location by naming it to the robot. Behavior hypothesis The users will go with the robot to the location. Once this location is reached, the user stops the robot and attempts to name the place. Robot behavior The user will not stop at the location before attempting to name it. The user will try to time the labeling command to the moment when the robot has or is about to reach the goal point or area. The specification of a location is confirmed by the robot using speech. The follow behavior stops after the user started to specify objects or locations. http: //www. nada. kth. se/iplab/hri 6

Tool development • Dialog production • Joystick navigation • Ultra-sound and lasersensor feedback on user’s distance • Robot’s camera steered by joystick POV http: //www. nada. kth. se/iplab/hri 7

TV-set Fruit-basket VCR Coffee-mug Trash can Sofa Too much – ögon virrar rund… Remote Focus på VAD? Kanske bra om man tar det och stegar sig control igenom(? ) R Battery station Colleagues

Här kasta du bort vikt http: //www. nada. kth. se/iplab/hri 9

Data collection Onboard cam

http: //www. nada. kth. se/iplab/hri 11

Conclusion • Wizard-of-Oz provide useful insights about future system • Visualizing an interactive scenario to users and designers allowing for evaluation • Collect data for various purposes • Opportunity to study human-robot communication with methods from other disciplines (e. g. linguistics, psychology) Future work • Interleaved evaluation with “real” components § Follow behavior • Annotation formats § Creating and sharing “corpora” of examples http: //www. nada. kth. se/iplab/hri 12

“Missing” • All of ‘analysis’: L Time consuming J subjective measurements, too • Ethical discussion “to trick people” • Discussion: “Why does it work? ” Can we really achieve machine-like system performance/behavior and measure it? should one iterate design during Woz-studies works only ONCE with users (i. e. no iterative studies possible) http: //www. nada. kth. se/iplab/hri 13