Metacognitive Capabilities for Advice Taking Jihie Kim Yolanda Gil Jim Blythe Intelligent Systems Division USC/Information Sciences Institute www. isi. edu/ikcap/ USC Information Sciences Institute Jihie Kim 1
Challenges Systems must understand how user’s advice/guidance/request fits before accepting it. Self-assessment of competence and confidence USC Information Sciences Institute Jihie Kim 2
![Key Idea Apply metacognitive skills in human learning [White and Frederickson 98, Harris 1990]](https://present5.com/presentation/fca7271399aa64083a85bd0d08e794e9/image-3.jpg "Key Idea Apply metacognitive skills in human learning [White and Frederickson 98, Harris 1990]")
Key Idea Apply metacognitive skills in human learning [White and Frederickson 98, Harris 1990] Awareness of its own knowledge and the ability to understand, control and manipulate its own cognitive process USC Information Sciences Institute Jihie Kim 3
Approach Explicit representation of meta-level goals to understand, control, and manipulate system’s cognitive process of KA and develop strategies for them. Develop annotations to the dialogue history based on the goals and strategies (episodic memory) Apply our techniques in proactive KA dialogue [Kim & Gil AIEd 2002, Gil & Kim Cog. Sci-2001; Kim& Gil ITS 2001] USC Information Sciences Institute Jihie Kim 4
Meta-Level Goals Establish confidence on KB items Ensure that KB items are useful and run efficiently Self-evaluate successful/unsuccessful procedures that are often used vs rarely used ones procedures that are efficient/inefficient evaluate other KB items in a similar way Be aware of the goals set (i. e. understand the purpose) Achieve goals or Ensure progress towards the goals Achieve proficiency goals (intentions) that are often set errors/gaps found progress towards a goal (history) answer changes (history) redundant calls to a procedure USC Information Sciences Institute Jihie Kim 5
Meta-Level Goals (cont) Ensure that advices were useful in making the problem solving successful Ask for more advices when there are conflicts, changes in answers, redundancies, etc. Ensure that advices are consistent with existing knowledge advices that were useful for making PS successful advices that didn’t affect the results frequent rejects or dislikes contradicting advices redundant advices Establish competence: assess own problem solving capability kinds of problems that can be solved kinds of problems that can’t be solved yet (based on attempts) USC Information Sciences Institute Jihie Kim 6
Developing Strategies for Meta-Level Goals on KB items Establish confidence on KB items Ensure that KB items are useful and run efficiently self evaluate successful/unsuccessful procedures check the involved PS episodes, whether any of them that use the procedure succeeded Warning : find_laptop still doesn’t return any results procedures that are often used vs rarely used ones check how used, how many times Warning: the procedure of ‘find_midrange_systems’ hasn’t been used for a while. procedures that are efficient/inefficient check how used, how long it took to run Warning: calls to find_all_laptops took more than 5 min. evaluate other KB items in a similar way USC Information Sciences Institute Jihie Kim 7
Developing Strategies for Meta-Level Goals (cont) Goals on Problem Solving Goals Be aware of the goals set (i. e. understand the purpose) Achieve goals or Ensure progress towards the goals Achieve proficiency monitor goals that are often set check the goals set in the history goals set but not visited for a while check the goals & PS episodes Warning: are you still interested in send_email to Susie? errors/gaps found Warning (if error message not sent from other modules): there are still some errors … progress towards a goal (history) number of errors/ number of steps taken in PS episode Feedback Q: now we can perform find-laptops, but still don’t know how to do get_autorization answer changes (history) number of errors/ number of steps taken in PS episode Feedback: now we have IBM think pad as well as SONY VAIO as the candidates redundant calls to a procedure Warning: in solving ‘buy laptop’ the same calls to find-laptop have been made multiple 8 USC Information Sciences Institute Jihie Kim times
Developing Strategies for Meta-Level Goals (cont) Goals on advices taken Ensure that advices were useful in making the problem solving successful Ask for more advices when there are conflicts, changes in answers, redundancies, etc. Ensure that advices are consistent with existing knowledge advices that were useful for making PS successful check changes in the PS episodes Feedback: since you raised the max price, now we can find candidate laptops advices that didn’t affect the results check changes in the PS episodes Warning: you advice to reduce the min speed didn’t change the answer frequent rejects or dislikes contradicting advices check inconsistent results redundant advices USC Information Sciences Institute Jihie Kim 9
Developing Strategies for Meta-Level Goals (cont) Goals on Overall problem solving capability Establish competence: assess own problem solving capability kinds of problems that can be solved monitor PSs completed and how well they are done kinds of problems that can’t be solved yet (based on attempts) check the answers Warning: we can’t solve find_autorization yet USC Information Sciences Institute Jihie Kim 10
Summary: Metacognitive Capabilities for Advice Taking User Advice KB changes KB item usage Follow-up Episodic Memory dialogue Meta reasoning with Annotations - Self assessment of competence 3/28/03 Jihie buys laptop - Strategies for resolving conflicts - Initial task goals - solution proposed - Relevance (usage) and gaps - advice: new cost limit - 2 nd solution -. . . KB annotations 9/1/03 Jim buys laptop -. . . 10/28/03 Jihie finds laptop USC Information Sciences Institute Problem Solving KB-1 KB-2 . . . KB Jihie Kim 11
Скачать презентацию Metacognitive Capabilities for Advice Taking Jihie Kim Yolanda
fca7271399aa64083a85bd0d08e794e9.ppt