Title A Joint Intention Based Dialog Architecture Rajah

Title A Joint Intention Based Dialog Architecture Rajah. A Subramanian University of Rochester 1

Need for a Generic Dialog Engine n Current spoken dialog systems q q q n Domain based systems. Need to explicitly code all possible rules and conditions for prompting, informing etc. Very difficult to port from one domain to another, breaks easily and cannot be used for complex task-based scenarios. We propose an architecture that can reason in a general manner and can replace many of the rules and condition that are explicit in current dialog systems. q q Handle over-answering, multi-agent interaction, and certain dynamic changes in the environment. Build Systems that are robust and domain independent. 2

Dialog Process We are concerned here in the Dialog Management Strategy. Speech Recognition Language Understanding Dialog Management. Back-end Strategy (various) Speech Synthesis Language Generation 3

Outline of the talk n Overview q q Joint Intentions Theory Semantics of Communicative Acts Belief Reasoner STAPLE interpreter n Room Reservation Demo n Summary, Future & Related work 4

Joint Intention Theory (JIT) - I n n n Belief, Desire, Intention (BDI) Model – Bratman/Israel/Pollack, Georgeff/Lansky … Joint Intention Theory (JIT) (Cohen, Levesque). Uses modal logic (beliefs, goals), dynamic logic of actions (sequence, concurrent actions, etc. ), temporal logic (eventually, always). INTENTION – Choice with commitment. PGOAL x – Individual persistent goal of an agent. Once adopted, the agent is committed to it until achieved, impossible or irrelevant. 5

Joint Intention Theory (JIT) - II n PWAG x y – Joint goal between x and y. q q n n PGOAL to achieve the goal relative to PWAG. if already achieved, impossible or irrelevant, then PGOAL to mutually make it known to concerned agents. Teamwork: JPG x y – Two way PWAG’s/ X has a PWAG towards Y and Y towards X. Prescribes how to execute actions jointly while being in a shared mental state q q q Requires starting mutual belief (MB) Requires MB about task completion, impossibility, or irrelevance It predicts communication in the event of private beliefs 6

Semantics of Communicative Acts n Formal compositional semantics of commonly used speech acts using JIT constructs q REQUEST and INFORM are the basic CA’s. n The goal of a request (x y a ): y eventually does the action a and also have a PWAG with respect to the requester x to do a. n q q The goal of an inform (x y p) is to establish MB between the informee (x) and the informed (y) that x believes the informed predicate (p). The other CA’s such as ASK, AGREE, INFORMREF etc. are built from REQUEST and INFORM. For example, ASK is an REQUEST to INFORMREF. KNOWREF – To know the referent of the description of a required value. 7

Back Communicative Actions (cont. . ) n n Identified speech acts that can be used to create and discharge teams Showed how to achieve mutual belief Explicated defeasible assumptions about communication STAPLE (Sanjeev Kumar) – Implements the constructs JI theory and interprets PGOAL’s, PWAG’s, Communicative Acts, etc. Can you please help me move the table? REQUEST Sure + AGREE = Joint Commitment 8

STAPLE Architecture (Kumar 2006) 9

STAPLE Optimization n Original version – Java only q n Not efficient for unification, that constitutes more than 80% of execution time. To optimize – combined Java and Prolog, completed as part of RPE project. q q q Java for threads, stacks, triggers etc. Prolog for knowledge base, unification etc. More than 150 times faster for single-agent and 70 times faster for multi-agents. 10

Planning Relevant Questions – Snapshot of a Goal Stack in STAPLE INTEND (agent, request (agent, user, action (informref (user, agent, i (D, date (Room, D)))* PGOAL (agent, done (action (ask-ref ( agent, user, i(D, date (Room, D))* PGOAL (agent, (KNOW-REF agent, i (D, date(Room, D))) * PGOAL(agent, Room (reserved(Room, Date, Time, Size)) PWAG(agent, user (done(reserved(Room, Date, Time, …)))) *The PGOAL is actually a bit more complex – the system wants to know the referent of the date that the user wants the room reservation; also the sub-goals are relative to the original goal. 11

STAPLE: Multi-agent Dialog Example Peedy (SYSTEM) Robby (USER) Merlin (DATABASE AGENT) n n Room reservation domain Three Microsoft agent characters play the roles of User, System, & Database agent 12

Multi-Agent Dialog Example - Setup Robby (USER) n 1. 2. Initial state: - Believes that it wants a room at a particular date & time Actions: - Cannot perform any actions other than communicative acts Peedy (SYSTEM) n 1. Initial state: - Believes that the user knows the date & time for reserving the room - Believes that the database agent can query for rooms and hold a room if necessary. 2. Actions: - Can perform action reserve room that happens to be a named action expression (i. e. , a plan) Merlin (DATABASE AGENT) n 1. 2. Initial state: - No domain related beliefs. Actions: - Can perform actions to query the database and to hold a room. 13

Handling Over-Answering Agent: Please say the date you want the room User: I would like it on March 20 th at Three PM Bel(user, / ([ March 20 = i(Date, date (Room, Date))), 3 PM = i(Time, time (Room, Time)))])). Bel(user, March 20 = i(Date, date (Room, Date)))) PGOAL Discharged PGOAL (KNOW-REF i (D, date(Room, D))) Bel(user, 3 PM = i(Time, time (Room, Time)))). PGOAL Discharged PGOAL (KNOW-REF i (T, time(Room, T))) PGOAL (KNOW-REF i (S, size(Room, S))) 14

Multi-Agent Interaction System Agent Original JPG to reserve a room USER … (Action Expression Decomposition) PGOAL(system-agent, done(action (getrooms, X, E), …. Bel(system-agent, cando(getrooms, database-agent)) Database Agent 15

Multi-Agent Interaction Original JPG to reserve a room System Agent USER INTEND(system, action(request(system, [system, database], action(getrooms, …… …. . INTEND(system, action(establish_jpg([system, database], done(action(getrooms(……relative to PWAG reserveroom PGOAL(system-agent, done(action (getrooms, X, E), …. REQUEST Database Agent 16

Back Multi-Agent Interaction Original JPG to reserve a room System Agent USER JPG to do AGREE getrooms REQUEST Database Agent 17

Handling Dynamic Changes Original JPG to System USER reserve a room Agent Conversation between user and system to confirm the room JPG to do holdroom becomes impossible Database Agent INFORM – not able to hold the room INTEND(database, done(action(inform( database, system, always(not(done (action(holdroom(…. . PGOAL(database, mb([system, database], INTEND(database, action(holdroom, database) Room is TAKEN!!! always(not(done(action(holdroom(…. . Relative to original PWAG to reserve a room 18

Handling Dynamic Changes System Agent JPG to do holdroom becomes impossibe Original JPG to reserve a room USER Check if there are other rooms that are available or inform user that no rooms are available. Database Agent 19

Back Demo 20

The Point n n We get automatic team and task-related communication behavior q If agent believes that somebody else knows an answer then initiates dialogue to find out that answer. q If an agent believes that somebody else can do the action that it wanted done then it establishes a team for that purpose q Dialogue about mutual belief establishment follows automatically. q No mutual observation leads to explicit communication, otherwise explicit communication is absent. The above behavior can be chained to elicit even more interesting behavior. q Example: A sub-dialogue may get started in the middle of joint action execution to find the truth value of a proposition if it is the precondition for doing a part of the jointly committed action. 21

Future Work n In Progress q q n Implicit Confirmations Explicit Clarifications Still Mulling over… q q Partial and weak beliefs Belief update and revision 22

Related Work n Dialogue & Communication q q n Collagen: MERL – Rich, Sidner, Lesh Artimis & Arcol: France Telecom - Sadek, Panaget, Bretier, … TRIPS: University of Rochester – Allen, Ferguson, … FIPA, KQML Teamwork q q STEAM & Teamcore: USC – Tambe et al. Grate: Liverpool, UK - Jennings) 23

Appendix A: STAPLE Program – Multi Agent Interaction and Dynamic Changes n n bel(system, can_do(getrooms(Room, List), database)). bel(system, exists(D, bel(user, equals(D, i(Date, date(Room, Date)))))). plan(reserveroom, 4) : [args: [Room, [Date, Time], E 1, E 2], achieve_precondition: true, effects: [(room. Reserved(Room, [Date, Time]), 1. 0)], body: { action(roominfo([date(Room, Date), time(Room, Time)]), system, E 1)}]. plan(roominfo, 1) : [args: [[date(Room, Date), time(Room, Time)]], effects: [(rooms. Available(Room, [Date, Time]), 1. 0)], body: { action(getrooms(Room, [Date, Time]), X, Ev 1)}]. 24

Appendix B: Belief Reasoner (BEL self p) : - p (BEL α p) : - (BEL α □p) (BEL α p) : - (BEL α p)) : - (BEL α p) : + (BEL α p) (BEL α p) : - (BEL α p) (BEL α □□p): -(BEL α □p) (BEL α p): - (BEL α p) : - (BEL α q=>p) / (BEL α q) (BEL α p/q) : - (BEL α p) / (BEL α q) (BEL α □ (p/q)) : - (BEL α □ p) (BEL α □ q) 25

Appendix C: Mutual Belief Reasoner (BEL self p) : - (MB self y p) (BEL self (BEL y p)) : - (MB self y p) (BMB self y p) : - (MB self y p) (BMB y self p) : - (MB self y p) (BEL self (MB self y (BEL self p))) : - (MB self y p) (BEL self (MB self y (BEL y p))) : - (MB self y p) (BEL self (MB self y p/q)) : - (BEL self (MB self y p)) / (BEL self (MB self y q)) (BEL self (MB self y □ (p/q))) : - (BEL self (MB self y □ p)) (BEL self (MB self y □ q)) 26