11 Learning and problem solving agents Prof. Gheorghe Tecuci Learning Agents Laboratory Computer Science Department George Mason University 2003, G. Tecuci, Learning Agents Laboratory 1
Overview Learning and problem solving agents: Disciple An agent for center of gravity analysis Modeling of problem solving through task reduction Knowledge base: object ontology + rules Rule-based problem solving Control of the problem solving process Control of modeling, learning and problem solving Multistrategy rule learning 2003, G. Tecuci, Learning Agents Laboratory Multistrategy rule refinement 2
What are intelligent agents An intelligent agent is a system that: • perceives its environment (which may be the physical world, a user via a graphical user interface, a collection of other agents, the Internet, or other complex environment); • reasons to interpret perceptions, draw inferences, solve problems, and determine actions; and • acts upon that environment to realize a set of goals or tasks for which it was designed. input/ sensors user/ environment output/ effectors Intelligent Agent 3
The architecture of an intelligent agent Implements a general problem solving method that uses the knowledge from the knowledge base to interpret the input and provide an appropriate output. Intelligent Agent Input/ Sensors User/ Environment Problem Solving Engine Learning Engine Output/ Effectors Knowledge Base Ontology Implements learning methods for extending and refining the knowledge in the knowledge base. Rules/Cases/… Data structures that represent the objects from the application domain, general laws governing them, actions that can be performed with them, etc. 4
How are agents built and why it is hard Intelligent Agent Domain Expert Knowledge Engineer Inference Engine Dialog Programming Knowledge Base Results The knowledge engineer attempts to understand how the subject matter expert reasons and solves problems and then encodes the acquired expertise into the agent's knowledge base. This modeling and representation of expert’s knowledge is long, painful and inefficient (known as the “knowledge acquisition bottleneck”). 5
Another approach to agent building Intelligent Agent Domain Expert Knowledge Engineer Inference Engine Dialog Programming Knowledge Base Results Instructable Agent Domain Expert Inference Engine Dialog Learning Engine Knowledge Base Agent training directly by the subject matter expert. 7
Disciple approach to agent building LALAB has developed a theory, a methodology, and a family of tools for the rapid building of knowledge bases and agents by subject matter experts, with limited assistance from knowledge engineers, to overcome the knowledge acquisition bottleneck. Disciple approach: Develop learning and problem solving agents that can be taught by subject matter experts to become knowledge-based assistants. The expert teaches the agent to perform various tasks in a way that resembles how the expert would teach a person. Mixed-initiative problem solving Teaching and learning Multistrategy learning The agent learns from the expert, building, verifying and improving its knowledge base. Rapid agent development and 8 maintenance
Disciple’s vision on the future of software development Mainframe Computers Personal Computers Learning Agents Software systems developed and used by persons that are not computer experts Software systems developed by computer experts and used by persons that are not computer experts Software systems developed and used by computer experts 10
Vision on the use of Disciple in Education Disciple Agent KB teaches … The expert/teacher teaches Disciple through examples and explanations, in a way that is similar to how the expert would teach a student. teaches Disciple Agent KB teaches Disciple tutors the student in a way that is similar to how the expert/teacher has taught it. 12
DARPA’s HPKB program: evaluation Disciple-WA (1997 -1998): Estimates the best plan of working around damage to a transportation infrastructure, such as a damaged bridge or road. Disciple-WA demonstrated that a knowledge engineer can use Disciple to rapidly build and update a knowledge base capturing knowledge from military engineering manuals and a set of sample solutions provided by a subject matter expert. 72% increase of KB size in 17 days Development of Disciple’s KB during evaluation. Evolution of KB coverage and performance from the pre-repair phase to the post-repair phase. Disciple-WA features: • High knowledge acquisition rate; • High problem solving performance (including unanticipated solutions). • Demonstrated at EFX’ 98 as part of an 14 integrated application led by Alphatech.
DARPA’s HPKB program: evaluation Disciple-COA (1998 -1999): Identifies strengths and weaknesses in a Course of Action, based on the principles of war and the tenets of army operations. Disciple-COA demonstrated the generality of its learning methods that used an object ontology created by another group (TFS/Cycorp). It also demonstrated that a knowledge engineer and a subject matter expert can jointly teach Disciple. 100% 46% increase of KB size in 8 days Evolution of KB coverage and performance from the pre-repair phase to the post-repair phase for the final 3 evaluation items. Disciple-COA features: Development of Disciple’s KB during evaluation. • High knowledge acquisition rate; • Better performance than the other evaluated systems; • Better performance than the evaluating 16 experts (many unanticipated solutions).
Overview Learning and problem solving agents: Disciple An agent for center of gravity analysis Modeling of problem solving through task reduction Knowledge base: object ontology + rules Rule-based problem solving Control of the problem solving process Control of modeling, learning and problem solving Multistrategy rule learning 2003, G. Tecuci, Learning Agents Laboratory Multistrategy rule refinement 18
Center of gravity challenge problem Develop an intelligent agent that is able to identify and test strategic Center of Gravity candidates for a war scenario. The center of gravity of an entity (state, alliance, coalition, or group) is the foundation of capability, the hub of all power and movement, upon which everything depends, the point against which all the energies should be directed. Carl Von Clausewitz, “On War, ” 1832. If a combatant eliminates or influences the enemy’s strategic center of gravity, then the enemy will lose control of its power and resources and will eventually fall to defeat. If the combatant fails to adequately protect his own strategic center of gravity, he invites disaster. Giles and Galvin, 1996 19
Approach to center of gravity analysis Centers of Gravity: Primary sources of moral or physical strength, power or resistance. Critical Capabilities: Primary abilities which merit a Center of Gravity to be identified as such in the context of a given scenario, situation or mission. Critical Requirements: Essential conditions, resources and means for a Critical capability to be fully operative. Critical Vulnerabilities: Critical Requirements or components thereof which are deficient, or vulnerable to neutralization, interdiction or attack (moral/physical harm) in a manner achieving decisive results – the smaller the resources and effort applied and the smaller the risk and cost, the better. Joe Strange, Centers of Gravity & Critical Vulnerabilities, 1996. 20
First computational approach to COG analysis • Approach to center of gravity analysis based on the concepts of critical capabilities, critical requirements and critical vulnerabilities, which have been recently adopted into the joint military doctrine. • Application to current war scenarios (e. g. War on terror Identification of COG Testing of COG candidates 2003, Iraq 2003) with state and non-state actors (e. g. Al candidates Identify potential primary Test each identified COG Qaeda). sources of moral or physical strength, power and resistance from: Government Military candidate to determine whether it has all the necessary critical capabilities: Which are the critical capabilities? People Are the critical requirements of these capabilities satisfied? Economy If not, eliminate the candidate. Alliances If yes, do these capabilities have any vulnerability? Etc. 21
Critical capabilities needed to be a COG people leader be protected stay informed communicate military receive communication from the highest level leadership be deployable communicate desires to the highest level leadership be indispensable be influential support the goal be a driving force have support be irreplaceable exert power industrial capacity financial capacity support the highest level leadership external support have a positive impact will of multi member force be influential ideology 23
Illustration: Saddam Hussein (Iraq 2003) Critical capability to be protected Corresponding critical requirement Have means to be protected from all threats Means Vulnerabilities Republican Guard Protection Unit loyalty not based on conviction and can be influenced by US-led coalition Iraqi Military loyalty can be influenced by US-led coalition can be destroyed by US-led coalition Complex of Iraqi Bunkers location known to US led coalition design known to US led coalition can be destroyed by US-led coalition System of Saddam Doubles loyalty of Saddam Doubles to Saddam can be influenced by US-led coalition 24
Use of Disciple at the US Army War College 319 jw Case Studies in Center of Gravity Analysis Disciple helps the students to perform a Disciple was taught based on the center of gravity analysis of an expertise of Prof. Comello in center of assigned war scenario. gravity analysis. Teaching Problem Disciple Learning Agent KB solving Global evaluations of Disciple by officers from the Spring 03 course The use of Disciple is an assignment that is well suited to the course's learning objectives Disciple helped me to learn to perform a strategic COG analysis of a scenario Disciple should be used in future versions of this course 25
Student – Disciple collaboration Is guided by Disciple to describe the relevant aspects of a strategic environment. Develops a formal representation of the scenario. Identifies and tests strategic COG candidates. Student Studies the logic behind COG identification and testing. Disciple Generates a COG analysis report. Critiques Disciple’s analysis and finalizes the analysis report. 27
The student is guided by Disciple to describe the relevant aspects of a strategic scenario. Disciple represents the scenario as instances in its object ontology. Disciple 28
Execution of the elicitation scripts Script type: Elicit the feature Has_as_opposing_force for an instance <scenario-name> Controls: Question: Name the opposing forces in <scenario-name> Answer variable: <opposing-force> Control type: multiple-line, height 4 Ontology actions: <opposing-force> instance-of Opposing_force <scenario-name> Has_as_opposing_force <opposing-force> Script calls: Elicit properties of the instance <opposing-force> Sicily_1943 in new window <object> subconcept-of Force subconcept-of Scenario Opposing_force instance-of Has_as_opposing_force Allied_Forces_1943 instance-of European_Axis_1943 … 30
Sample object ontology <object> subconcept_of Scenario subconcept_of Force_goal subconcept_of Strategic_COG_relevant_factor … Resource_or_ infrastructure_ element … subconcept_of Multi_state_force subconcept_of Group Product subconcept_of instance_of Opposing_force subconcept_of Single_state_force subconcept_of Single_group_force Sicily_1943 subconcept_of Economic_ Multi_state_alliance instance_of Strategically_ factor essential_ Multi_group_force subconcept_of Britain_1943 instance_of has as goods_or_ opposing materiel Equal_partners_ US_1943 has_as_industrial_factor component_ force multi_state_ subconcept_of state alliance instance_of subconcept_of US_7 th_Army_ instance_of Industrial_ (Force_343) component_ War_materiel_ factor instance_of state and_transports Allied_Forces_1943 th_Army_ instance_of Br_8 has_as_primary_ (Force_545) has_as_subgroup force_element subconcept_of instance_of brief_description has_as_subgroup instance_of Industrial_ War_materiel_ has_as_subgroup Western_Naval_TF capacity Allied_forces_operation_Husky and_transports has_as_subgroup _of_US_1943 Eastern_Naval_TF type_of_operations instance_of has_as_subgroup instance_of is_a_major_ th_Air_Force generator_of US_9 has_as_subgroup “WWII Allied invasion industrial_ instance_of capacity_ of Sicily in 1943” “combined and joint operations” Northwest_Africa_Air_Force US_1943 War_scenario … … … 32
Disciple identifies and tests COG candidates The students study the logic behind COG identification and testing 33
Disciple generates a COG analysis report for the student to finalize 35
Demonstration Disciple as a strategic leader assistant Disciple 37
Overview Learning and problem solving agents: Disciple An agent for center of gravity analysis Modeling of problem solving through task reduction Knowledge base: object ontology + rules Rule-based problem solving Control of the problem solving process Control of modeling, learning and problem solving Multistrategy rule learning 2003, G. Tecuci, Learning Agents Laboratory Multistrategy rule refinement 38
Problem solving approach: Task reduction Disciple uses the task-reduction paradigm A complex problem solving task is performed by: T 1 S 1 • successively reducing it to simpler tasks; • finding the solutions of the simplest tasks; • successively composing these solutions until the solution to the initial task is obtained. T 11 S 11 … T 1 n S 1 n T 111 S 111 … T 11 m S 11 m 39
Question-answering guided task reduction Let T 1 be the problem solving task to be performed. Finding a solution is an iterative process where, at each step, we consider some relevant information that leads us to reduce the current task to several simpler tasks. The question Q associated with the current task identifies the type of information to be considered. The answer A identifies that piece of information and leads us to the reduction of the current task. T 1 Q 1 S 1 A 11 S 11 … T 11 a S 11 a T 11 b. S 11 b … Q 11 b S 11 b A 1 n S 1 n T 1 n … A 11 b 1 S 11 b 1… A 11 bm S 11 bm T 11 b 1 T 11 bm 40
Modeling COG analysis through task reduction (and solution composition) The will_of_Allied_Forces_1943 is a COG candidate with respect to the cohesion of Allied_Forces_1943 What kind of scenario is Sicily_1943? The will of Allied Forces 1943 is a strategic COG candidate that cannot be eliminated because it has all the necessary critical capabilities . . . Sicily_1943 is a war scenario The will_of_Allied_Forces_1943 is a COG candidate with respect to the cohesion of Allied_Forces_1943 Identify and test a strategic COG candidate for Sicily_1943 which is a war scenario The will of Allied Forces 1943 is a strategic COG candidate that cannot be eliminated because it has all the necessary critical capabilities Solution composition Task reduction Identify and test a strategic COG candidate for Sicily_1943 . . . 41
Identify and test a strategic COG candidate for Sicily_1943 which is a war scenario Which is an opposing force in the Sicily_1943 scenario? Allied_Forces_1943 The will_of_Allied_Forces_1943 is a COG candidate with respect to the cohesion of Allied_Forces_1943 The will of Allied Forces 1943 is a strategic COG candidate that cannot be eliminated because it has all the necessary critical capabilities . . . Identify and test a strategic COG candidate for Allied_Forces_1943 Is Allied_Forces_1943 a singlemember force or a multi-member force? The will_of_Allied_Forces_1943 is a COG candidate with respect to the cohesion of Allied_Forces_1943 is a multimember force The will of Allied Forces 1943 is a strategic COG candidate that cannot be eliminated because it has all the necessary critical capabilities Identify and test a strategic COG candidate for Allied_Forces_1943 which is a multi-member force . . . 42
The will_of_Allied_Forces_1943 is a COG candidate with respect to the cohesion of Allied_Forces_1943 Identify and test a strategic COG candidate for Allied_Forces_1943 which is a multi-member force The will of Allied Forces 1943 is a strategic COG candidate that cannot be eliminated because it has all the necessary critical capabilities What type of strategic COG candidate should I consider for this multi-member force? . . . I consider a candidate corresponding to the multi -member nature of the force Identify and test a strategic COG candidate corresponding to the multi-member nature of Allied_Forces_1943 I consider a candidate corresponding to a member of the multimember force Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 The will_of_Allied_Forces_1943 is a COG candidate with respect to the cohesion of Allied_Forces_1943 The will of Allied Forces 1943 is a strategic COG candidate that cannot be eliminated because it has all the necessary critical capabilities . . . 43
Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Which is a member of Allied_Forces_1943? The will_of_the_people_of_US_1943 is a strategic COG candidate with respect to the people_of_US_1943 The will of people of US 1943 is a strategic COG candidate that cannot be eliminated because it has all the necessary critical capabilities . . . US_1943 Identify and test a strategic COG candidate for US_1943 . . . What kind of force is US 1943? US_1943 is a singlemember force Identify and test a strategic COG candidate for US_1943 which is a single-member force The will_of_the_people_of_US_1943 is a strategic COG candidate with respect to the people_of_US_1943 The will of people of US 1943 is a strategic COG candidate that cannot be eliminated because it has all the necessary critical capabilities . . . 44
Identify and test a strategic COG candidate for US 1943 which is a single-member force . . . What type of strategic COG candidate should I consider for this single-member force? I consider a strategic COG candidate with respect to the people of US 1943 Identify and test a strategic COG candidate with respect to the people of US 1943 . . . I consider a strategic COG candidate with respect to the government of US 1943 Identify and test a strategic COG candidate with respect to the government of US 1943 . . . I consider a strategic COG candidate with respect to the armed forces of US 1943 Identify and test a strategic COG candidate with respect to the armed forces of US 1943 . . . I consider a candidate corresponding to the economy US 1943 Identify and test a strategic COG candidate corresponding to the economy of US 1943 . . . I consider a candidate corresponding to other sources of moral or physical strength, power and resistance of US 1943 Identify and test a strategic COG candidate with respect to other sources of moral or physical strength, power and resistance of US 1943 . . . 45
Identify and test a strategic COG candidate with respect to the government of US 1943 President Roosevelt is a strategic COG candidate with respect to the government_of_US_1943 Who or what is a main controlling element of the government_of_US_1943? President Roosevelt is a strategic COG candidate that can be eliminated because it does not have all the necessary critical capabilities President Roosevelt that has a critical role in setting objectives and making decisions Identify President Roosevelt as a strategic COG candidate with respect to the government_of_US_1943 President Roosevelt is a strategic COG candidate with respect to the government_of_US_1943 Test whether President Roosevelt is a viable strategic COG candidate President Roosevelt is a strategic COG candidate that can be eliminated because it does not have all the necessary critical capabilities 46
President Roosevelt is a strategic COG candidate that can be eliminated Test whether President Roosevelt is a viable strategic COG candidate Which are the critical capabilities that President Roosevelt should have to be a COG candidate? Does President Roosevelt have all the necessary critical capabilities? The necessary critical capabilities are: be protected, stay informed, communicate, be influential, be a driving force, have support and be irreplaceable Test whether President Roosevelt has the critical capability to be protected. President Roosevelt is protected by US Service 1943 which has no significant vulnerability Test whether President Roosevelt has the critical capability to stay informed. President Roosevelt receives essential intelligence from intelligence agencies which have no significant vulnerability Test whether President Roosevelt has the critical capability to communicate through executive orders, through military orders, and through the Mass Media of US 1943. These communication means have no significant vulnerabilities Test whether President Roosevelt has the critical capability to be influential because he is the head of the government of US 1943, the commander in chief of the military of US 1943, and is a trusted leader who can use the Mass Media of US 1943. These influence means have no significant vulnerabilities. Test whether President Roosevelt has the critical capability to be a driving force. The main reason for President Roosevelt to pursue the goal of unconditional surrender of European Axis is “preventing separate peace by the members of the Allied Forces”. Also, “the western democratic values” provides President Roosevelt with determination to persevere in this goal. There is no significant vulnerability in the reason and determination. Test whether President Roosevelt has the critical capability to have support No. President Roosevelt has the critical capability to have support because he is the head of a democratic government with a history of good decisions, a trusted commander in chief of the military, and the people are willing to make sacrifices for unconditional surrender of European Axis. The means to secure continuous support have no significant vulnerability. Test whether President Roosevelt has the critical capability to be irreplaceable President Roosevelt does not have the critical capability to be irreplaceable. US 1943 would maintain the goal of unconditional surrender of European Axis irrespective of its leader because “the goal was established and the country was committed to it”. There is no significant vulnerability resulted from the replacement of President Roosevelt 47
Test whether President Roosevelt has the critical capability to be influential Which are the critical requirements for President Roosevelt to be influential? President Roosevelt needs means to influence the government, means to influence the military and means to influence the people President Roosevelt has the critical capability to be influential because he is the head of the government of US 1943, the commander in chief of the military of US 1943, and is a trusted leader who can use the Mass Media of US 1943. These influence means have no significant vulnerabilities. Does President Roosevelt satisfy the critical requirements to be influential? Yes. Test whether President Roosevelt has means to influence the government President Roosevelt can influence the government of US 1943 because he is the head of the government of US 1943. The influence means have no significant vulnerability. Test whether President Roosevelt has means to influence the military President Roosevelt can influence the military of US 1943 because he is the commander in chief of the military of US 1943. The influence means have no significant vulnerability. Test whether President Roosevelt has means to influence the people The influence of President Roosevelt over the people of US 1943, as a trusted leader using the Mass Media of US 1943, has no significant vulnerability 48
Test whether President Roosevelt has means to influence the government President Roosevelt can influence the government of US 1943 because he is the head of the government of US 1943. The influence means have no significant vulnerability. What is a means for President Roosevelt to influence the government of US 1943? President Roosevelt is the head of the government of US 1943 Test whether the influence of President Roosevelt over the government of US 1943, as the head of the government of US 1943, has any significant vulnerability The influence of President Roosevelt over the government of US 1943, as the head of the government of US 1943, has no significant vulnerability Does the influence of President Roosevelt over the government of US 1943 have any significant vulnerability? No 49
Test whether President Roosevelt has means to influence the military The influence of President Roosevelt over the military of US 1943, as the commander in chief of the military of US 1943, has no significant vulnerability What is a means for President Roosevelt to influence the military of US 1943? President Roosevelt is the commander in chief of the military of US 1943 Test whether the influence of President Roosevelt over the military of US 1943, as the commander in chief of the military of US 1943, has any significant vulnerability The influence of President Roosevelt over the military of US 1943, as the commander in chief of the military of US 1943, has no significant vulnerability Does the influence of President Roosevelt over the military of US 1943 have any significant vulnerability? No 50
Test whether President Roosevelt has means to influence the people The influence of President Roosevelt over the people of US 1943, as a trusted leader using the Mass Media of US 1943, has no significant vulnerability What is a means for President Roosevelt to influence the people of US 1943? President Roosevelt is trusted by the people of US 1943 and can use Mass Media of US 1943 to influence them Test whether the influence of President Roosevelt over the people of US 1943, as a trusted leader using the Mass Media of US 1943, has any significant vulnerability The influence of President Roosevelt over the people of US 1943, as a trusted leader using the Mass Media of US 1943, has no significant vulnerability Does the influence of President Roosevelt over the people of US 1943 have any significant vulnerability? No 51
Overview Learning and problem solving agents: Disciple An agent for center of gravity analysis Modeling of problem solving through task reduction Knowledge base: object ontology + rules Rule-based problem solving Control of the problem solving process Control of modeling, learning and problem solving Multistrategy rule learning 2003, G. Tecuci, Learning Agents Laboratory Multistrategy rule refinement 52
The structure of the knowledge base Knowledge Base = Object ontology + Task reduction rules The object ontology is a hierarchical description of the objects from the domain, specifying their properties and relationships. It includes both descriptions of types of objects (called concepts) and descriptions of specific objects (called instances). The task reduction rules specify generic problem solving steps of reducing complex tasks to simpler tasks. They are described using the objects from the ontology. 2003, G. Tecuci, Learning Agents Laboratory 53
Fragment of the object ontology governing_body ad_hoc_ governing_body established_ governing_body other_type_of_ governing_body state_government feudal_god_ king_government other_state_ government democratic_ government monarchy group_governing_body other_ group_ governing_ body dictator deity_figure representative_ parliamentary_ democracy government_ of_Italy_1943 totalitarian_ government police_ state government_ of_US_1943 government_ of_Britain_1943 military_ dictatorship religious_ dictatorship fascist_ state communist_ dictatorship 2003, G. Tecuci, Learning Agents Laboratory government_ of_USSR_1943 government_ of_Germany_1943 democratic_ council_ or_board autocratic_ leader theocratic_ government religious_ dictatorship chief_and_ tribal_council theocratic_ democracy 54
Fragment of feature ontology has_as_controlling_leader D: agent R: person has_as_religious_leader D: governing_body R: person has_as_god_king D: governing_body R: person has_as_monarch D: governing_body R: person has_as_military_leader D: governing_body R: person has_as_political_leader D: governing_body R: person has_as_head_of_government D: governing_body R: person 2003, G. Tecuci, Learning Agents Laboratory has_as_commander_in_chief D: force R: person has_as_head_of_state D: governing_body R: person 56
Sample task A task is a representation of anything that an agent may be asked to perform. General task: Identify and test a strategic COG candidate corresponding to the ? O 1 INFORMAL STRUCTURE OF THE TASK Identify and test a strategic COG candidate corresponding to the economy of a force The economy is ? O 1 Condition ? O 1 is type_of_economy FORMAL STRUCTURE OF THE TASK Instantiated task: Identify and test a strategic COG candidate corresponding to the economy_of_US_1943 INFORMAL STRUCTURE OF THE TASK 2003, G. Tecuci, Learning Agents Laboratory Identify and test a strategic COG candidate corresponding to the economy of a force The economy is economy_of_US_1943 FORMAL STRUCTURE OF THE TASK 58
Exercise How could the agent generate plausible formalizations? Identify and test a strategic COG candidate for Sicily_1943 What kind of scenario is Sicily_1943? Sicily_1943 is a war scenario Identify and test a strategic COG candidate for Sicily_1943 which is a war scenario 2003, G. Tecuci, Learning Agents Laboratory 59
Sample task reduction rule IF Identify and test a strategic COG candidate corresponding to the ? O 1 which is an industrial_economy IF Identify and test a strategic COG candidate corresponding to the economy of a force which is an industrial economy The industrial economy is ? O 1 Question Who or what is a strategically critical element with respect to the ? O 1 ? Answer ? O 2 because it is an essential generator of war_materiel for ? O 3 from the strategic perspective Condition ? O 1 is industrial_economy THEN Identify ? O 2 as a COG candidate with respect to the ? O 1 ? O 4 is force has_as_economy ? O 1 has_as_industrial_factor ? O 2 Test ? O 2 which is a strategic COG candidate with respect to the ? O 1 INFORMAL STRUCTURE OF THE RULE A rule is an ontology-based representation of an elementary problem solving step. 2003, G. Tecuci, Learning Agents Laboratory ? O 2 is industrial_capacity generates_essential_war_materiel_from_ the_strategic_perspective_of ? O 3 is multi_state_force has_as_member ? O 4 THEN Identify a strategically critical element as a COG candidate with respect to an industrial economy The strategically critical element is ? O 2 The industrial economy is ? O 1 Test a strategically critical element which is a strategic COG candidate with respect to an industrial economy The strategically critical element is ? O 2 The industrial economy is ? O 1 61
Overview Learning and problem solving agents: Disciple An agent for center of gravity analysis Modeling of problem solving through task reduction Knowledge base: Object ontology + Rules Rule-based problem solving Control of the problem solving process Control of modeling, learning and problem solving Multistrategy rule learning 2003, G. Tecuci, Learning Agents Laboratory Multistrategy rule refinement 63
Illustration of rule-based task reduction Identify and test a strategic COG candidate corresponding to the economy of a force which is an industrial economy The industrial economy is economy_of_US_1943 ? O 1 economy_of_US_1943 Rule condition industrial_economy instance-of force economy_of_US_1943 instance-of has_as_economy ? O 4 has_as_industrial_factor has_as_member instance-of ? O 3 Condition ? O 1 is industrial_economy ? O 2 industrial_capacity multi_state_force IF Identify and test a strategic COG candidate corresponding to the economy of a force which is an industrial economy The industrial economy is ? O 1 instance-of ? O 2 generates_essential_ war_materiel_from_ the_ strategic_perspective_of is industrial_capacity generates_essential_war_materiel_from_ the_strategic_perspective_of ? O 3 is multi_state_force has_as_member ? O 4 is force has_as_economy ? O 1 has_as_industrial_factor ? O 2 THEN Identify a strategically critical element as a COG candidate with respect to an industrial economy The strategically critical element is ? O 2 The industrial economy is ? O 1 Test a strategically critical element which is a strategic COG candidate with respect to an industrial economy The strategically critical element is ? O 2 The industrial economy is ? O 1 2003, G. Tecuci, Learning Agents Laboratory 64
Matchings Object ontology Rule condition industrial_economy force subconcept-of instance-of single_member_force economy_of_US_1943 force subconcept-of instance-of single_state_force has_as_economy ? O 4 industrial_capacity has_as_industrial_factor multi_state_force has_as_member instance-of ? O 2 generates_essential_ war_materiel_from_ the_ strategic_perspective_of ? O 3 ? O 2 industrial_capacity_of_US_1943 industrial_economy instance-of economy_of_US_1943 instance-of has_as_economy US_1943 industrial_capacity has_as_industrial_factor instance-of Industrial_capacity_ multi_state_force of_US_1943 subconcept-of multi_state_ alliance subconcept-of generates_essential_ equal_partner_ war_materiel_from_ the_ multi_state_ strategic_perspective_of alliance ? O 3 Allied_forces_1943 ? O 4 US_1943 has_as_member instance-of Allied_forces_1943 2003, G. Tecuci, Learning Agents Laboratory 66
Identify and test a strategic COG candidate corresponding to the economy of a force which is an industrial economy The industrial economy is economy_of_US_1943 Rule condition industrial_economy instance-of force economy_of_US_1943 instance-of has_as_economy ? O 4 industrial_capacity has_as_industrial_factor multi_state_force has_as_member instance-of ? O 2 generates_essential_ war_materiel_from_ the_ strategic_perspective_of ? O 1 economy_of_US_1943 IF Identify and test a strategic COG candidate corresponding to the economy of a force which is an industrial economy The industrial economy is ? O 1 Condition ? O 1 is industrial_economy ? O 2 is industrial_capacity generates_essential_war_materiel_from_ the_strategic_perspective_of ? O 3 is multi_state_force has_as_member ? O 4 is force has_as_economy ? O 1 has_as_industrial_factor ? O 2 THEN Identify a strategically critical element as a COG candidate with respect to an industrial economy The strategically critical element is ? O 2 The industrial economy is ? O 1 ? O 3 Test a strategically critical element which is a strategic COG candidate with respect to an industrial economy The strategically critical element is ? O 2 The industrial economy is ? O 1 Identify a strategically critical element as a COG candidate with respect to an industrial economy ? O 1 economy_of_US_1943 The strategically critical element is industrial_capacity_of_US_1943 The industrial economy is economy_of_US_1943 ? O 2 industrial_capacity_of_US_1943 Test a strategically critical element which is a strategic COG candidate with respect to an industrial economy The strategically critical element is industrial_capacity_of_US_1943 The industrial economy is economy_of_US_1943 2003, G. Tecuci, Learning Agents Laboratory ? O 3 Allied_forces_1943 ? O 4 US_1943 68
Generating the informal reduction Identify and test a strategic COG candidate corresponding to the economy_of_US_1943 which is an industrial_economy ? O 1 economy_of_US_1943 IF Identify and test a strategic COG candidate corresponding to the ? O 1 which is an industrial_economy Who or what is a strategically critical element with respect to the economy_of_US_1943? industrial_capacity_of_US_1943 because it is an essential generator of war materiel for Allied_forces_1943 from the strategic perspective Identify industrial_capacity_of_US_1943 as a COG candidate with respect to the economy_of_US_1943 Test industrial_capacity_of_US_1943 which is a strategic COG candidate with respect to the economy_of_US_1943 2003, G. Tecuci, Learning Agents Laboratory Question Who or what is a strategically critical element with respect to the ? O 1 ? Answer ? O 2 because it is an essential generator of war_materiel for ? O 3 from the strategic perspective THEN Identify ? O 2 as a COG candidate with respect to the ? O 1 Test ? O 2 which is a strategic COG candidate with respect to the ? O 1 economy_of_US_1943 ? O 2 industrial_capacity_of_US_1943 ? O 3 Allied_forces_1943 ? O 4 US_1943 70
Successive rule applications Identify and test a strategic COG candidate corresponding to the economy_of_US_1943 Rule_1 What is the type of economy_of_US_1943 ? industrial_economy Rule_2 Identify and test a strategic COG candidate corresponding to the economy_of_US_1943 which is an industrial_economy Who or what is a strategically critical element with respect to the economy_of_US_1943? industrial_capacity_of_US_1943 because it is an essential generator of war materiel for Allied_forces_1943 from the strategic perspective Identify industrial_capacity_of_US_1943 as a COG candidate with respect to the economy_of_US_1943 2003, G. Tecuci, Learning Agents Laboratory Test industrial_capacity_of_US_1943 which is a strategic COG candidate with respect to the economy_of_US_1943 71
Task reduction rule with “Except when” conditions IF <task> Condition <condition 1> Except when condition <condition 2> In addition to the regular rule condition that needs to be satisfied, a rule may contain one or several except when conditions that should not be satisfied for the rule to be applicable. Except when condition <condition n> THEN <subtask 1> … <subtask m> 2003, G. Tecuci, Learning Agents Laboratory 72
Plausible version space rule IF Identify and test a strategic COG candidate corresponding to the economy of a force which is an industrial economy The industrial economy is ? O 1 Plausible upper bound condition ? O 1 is type_of_economy ? O 2 is economic_factor generates_essential_war_materiel_from_the_strategic_perspective_of ? O 3 is multi_state_force has_as_member ? O 4 is force has_as_economy ? O 1 has_as_industrial_factor ? O 2 Plausible lower bound condition ? O 1 is industrial_economy ? O 2 is industrial_capacity generates_essential_war_materiel_from_the_strategic_perspective_of ? O 3 is multi_state_alliance has_as_member ? O 4 is single_state_force has_as_economy ? O 1 has_as_industrial_factor ? O 2 THEN Identify a strategically critical element as a COG candidate with respect to an industrial economy The strategically critical element is ? O 2 The industrial economy is ? O 1 Test a strategically critical element which is a strategic COG candidate with respect to an industrial economy The strategically critical element is ? O 2 The industrial economy is ? O 1 73 2003, G. Tecuci, Learning Agents Laboratory
Overview Learning and problem solving agents: Disciple An agent for center of gravity analysis Modeling of problem solving through task reduction Knowledge base: Object ontology + Rules Rule-based problem solving Control of the problem solving process Control of modeling, learning and problem solving Multistrategy rule learning 2003, G. Tecuci, Learning Agents Laboratory Multistrategy rule refinement 75
The search space for problem solving Let us consider the problem solving task 'Pa‘ and let R 1, R 2, and R 3 be the applicable rules which indicate the reduction of 'Pa' to ‘C(Pb, Pc)', to 'Pd', and to ‘C(Pe, Pf, Pg)', respectively. Therefore, to solve the problem 'Pa', one may either: - solve the problems 'Pb' and 'Pc', or - solve the problem 'Pd', or - solve the problems 'Pe', 'Pf' and 'Pg'. One may represent all these alternatives in the form of an AND/OR tree. 2003, G. Tecuci, Learning Agents Laboratory 76
The search space for problem solving (cont. ) The node 'Pa' is called an OR node since for solving the problem 'Pa' it is enough to solve ‘C(Pb, Pc)' OR to solve 'Pd' OR to solve ‘C(Pe, Pf, Pg)'. The node ‘C(Pb, Pc)' is called an AND node since for solving it one must solve both 'Pb' AND 'Pc'. The AND/OR tree may be further developed by considering all the rules applicable to its leaves (Pb, Pc, Pd, Pe, Pf, Pg), building the entire search space for the problem 'Pa'. This space contains all the solutions to 'Pa'. 2003, G. Tecuci, Learning Agents Laboratory 77
Solution tree To find a solution one needs only to build enough of the tree to demonstrate that 'Pa' is solved. Such a tree is called a solution tree. A node is solved in one of the following cases: - it is a terminal node (a primitive task with known solution); - it is an AND node whose successors are solved; - it is an OR node which has at least one solved successor. 2003, G. Tecuci, Learning Agents Laboratory 78
Solution tree (cont. ) Once the problem solver detects that a node is solved it sends this information to the ancestors of the node. When the node 'Pa' becomes solved, one has found a solution to 'Pa'. solved 2003, G. Tecuci, Learning Agents Laboratory solved 79
Solution tree (cont. ) A node is unsolvable in one of the following cases: - it is a nonterminal node that has no successors (i. e. a nonprimitive problem to which no rule applies); - it is an AND node which has at least one unsolvable successor; - it is an OR node which has all the successors unsolvable. 2003, G. Tecuci, Learning Agents Laboratory 80
Solution tree (cont. ) Once the problem solver detects that a node is unsolvable it sends this information to the ancestors of the node. If the node 'Pa' becomes unsolvable, then no solution to 'Pa' exists. unsolvable solved 2003, G. Tecuci, Learning Agents Laboratory unsolvable 81
General search strategies The presented method assumes an exhaustive search of the solution space. Usually, however, the real world problems are characterized by huge search spaces and one has to use heuristic methods in order to limit the search. What types of search control decisions can you identify? Attention focusing: What problem, among the leaves of the problem solving tree, to reduce next? Meta-rule: What rule, among the applicable ones, to use for reducing the current problem? 2003, G. Tecuci, Learning Agents Laboratory 82
Overview Learning and problem solving agents: Disciple An agent for center of gravity analysis Modeling of problem solving through task reduction Knowledge base: Object ontology + Rules Rule-based problem solving Control of the problem solving process Control of modeling, learning and problem solving Multistrategy rule learning 2003, G. Tecuci, Learning Agents Laboratory Multistrategy rule refinement 83
Use of Disciple at the US Army War College 589 jw Military Applications of Artificial Intelligence course Students teach test the Disciple their trained COG analysis Disciple expertise, using agent based sample scenarios on a new (Iraq 2003, War on scenario terror 2003, Arab(North Israeli 1973) Korea 2003) Global evaluations of Disciple by officers during three experiments I think that a subject matter expert can use Disciple to build an agent, with limited assistance from a knowledge engineer Spring 2001 COG identification 2003, G. Tecuci, Learning Agents Laboratory Spring 2002 COG identification and testing Spring 2003 COG testing based on critical capabilities 84
Control of modeling, learning and problem solving Input Task Mixed. Initiative Problem Solving Ontology + Rules Generated Reduction New Reduction Accept Reduction Reject Reduction Solution Modeling Formalization Task Refinement Rule Refinement Learning 2003, G. Tecuci, Learning Agents Laboratory 86
I need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 1 Which is a member of Allied_Forces_1943? Provides an example 2 Learns Rule_15 US_1943 Therefore I need to Identify and test a strategic COG candidate for US_1943 … I need to 3 Identify and test a strategic COG candidate corresponding to a member of the European_Axis_1943 Applies Rule_15 4 Which is a member of European_Axis_1943? ? Accepts the example 2003, G. Tecuci, Learning Agents Laboratory Germany_1943 5 Refines Rule_15 Therefore I need to Identify and test a strategic COG candidate for Germany_1943 88
Overview Learning and problem solving agents: Disciple An agent for center of gravity analysis Modeling of problem solving through task reduction Knowledge base: Object ontology + Rules Rule-based problem solving Control of the problem solving process Control of modeling, learning and problem solving Multistrategy rule learning 2003, G. Tecuci, Learning Agents Laboratory Multistrategy rule refinement 90
The rule learning problem: definition GIVEN: • an example of a problem solving episode; • a knowledge base that includes an object ontology and a set of problem solving rules; • an expert that understands why the given example is correct and may answer agent’s questions. DETERMINE: • a plausible version space rule that is an analogy-based generalization of the specific problem solving episode. 2003, G. Tecuci, Learning Agents Laboratory 91
Input example Which is a member of Allied_Forces_1943? US_1943 I need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Therefore I need to Identify and test a strategic COG candidate for US_1943 This is an example of a problem solving step from which the agent will learn a general problem solving rule. 2003, G. Tecuci, Learning Agents Laboratory 92
Learned PVS rule IF Identify and test a strategic COG candidate corresponding to a member of the ? O 1 IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ? O 1 Question Which is a member of ? O 1 ? Answer ? O 2 explanation ? O 1 has_as_member ? O 2 THEN Identify and test a strategic COG candidate for ? O 2 INFORMAL STRUCTURE OF THE RULE 2003, G. Tecuci, Learning Agents Laboratory Plausible Upper Bound Condition ? O 1 is multi_member_force has_as_member ? O 2 is force Plausible Lower Bound Condition ? O 1 is equal_partners_multi_state_alliance has_as_member ? O 2 is single_state_force THEN Identify and test a strategic COG candidate for a force The force is ? O 2 93
Basic steps of the rule learning method 1. Formalize and learn the tasks 2. Find a formal explanation of why the example is correct. This explanation is the best possible approximation of the question and the answer, in the object ontology. 3. Generalize the example and the explanation into a plausible version space rule. 2003, G. Tecuci, Learning Agents Laboratory 95
1. Formalize the tasks I need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 I need to Identify and test a strategic COG candidate corresponding to a member of a force The force is Allied_Forces_1943 Therefore I need to Identify and test a strategic COG candidate for US_1943 Therefore I need to Identify and test a strategic COG candidate for a force The force is US_1943 2003, G. Tecuci, Learning Agents Laboratory 96
Task learning Identify and test a strategic COG candidate for US_1943 Identify and test a strategic COG candidate for a force The force is US_1943 Identify and test a strategic COG candidate for ? O 1 object subconcept_of INFORMAL STRUCTURE OF THE TASK force subconcept_of Identify and test a strategic COG candidate for a force The force is ? O 1 subconcept_of multi_member_force opposing_force subconcept_of multi_state_force single_member_force instance_of subconcept_of multi_state_alliance Plausible upper bound condition ? O 1 is force Plausible lower bound condition ? O 1 is single_state_force FORMAL STRUCTURE OF THE TASK 2003, G. Tecuci, Learning Agents Laboratory subconcept_of equal_partners_ multi_state_ alliance instance_of Single_state_force instance_of has_as_member US_1943 Allied_Forces_1943 98
2. Find an explanation of why the example is correct Which is a member of Allied_Forces_1943? US_1943 I need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Therefore I need to Identify and test a strategic COG candidate for US_1943 The explanation is the best possible approximation of the question and the answer, in the object ontology. Allied_Forces_1943 2003, G. Tecuci, Learning Agents Laboratory has_as_member US_1943 100
3. Generate the PVS rule Allied_Forces_1943 has_as_member US_1943 IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ? O 1 Rewrite as Most generalization Condition ? O 1 is Allied_Forces_1943 has_as_member ? O 2 is US_1943 Most specific generalization explanation ? O 1 has_as_member ? O 2 Plausible Upper Bound Condition ? O 1 is multi_member_force has_as_member ? O 2 is force Plausible Lower Bound Condition ? O 1 is equal_partners_multi_state_alliance has_as_member ? O 2 is single_state_force THEN Identify and test a strategic COG candidate for a force The force is ? O 2 2003, G. Tecuci, Learning Agents Laboratory 102
Analogical reasoning Analogy criterion multi_member_force instance_of ? O 1 instance_of has_as_ member less general than explanation has_as_ Allied_Forces_1943 member US_1943 less general than Identify and test a strategic COG candidate for a force The force is US_1943 2003, G. Tecuci, Learning Agents Laboratory similar explanation similar has_as_ European_Axis_1943 member Germany_1943 explains? explains initial example I need to Identify and test a strategic COG candidate corresponding to a member of a force The force is Allied_Forces_1943 Therefore I need to ? O 2 similar example I need to Identify and test a strategic COG candidate corresponding to a member of a force The force is European_Axis_1943 Therefore I need to Identify and test a strategic COG candidate for a force The force is Germany_1943 104
Generalization by analogy multi_member_force has_as_ Allied_Forces_1943 member US_1943 instance_of ? O 1 explains initial example generalization force instance_of has_as_ member ? O 2 explains I need to Identify and test a strategic COG candidate corresponding to a member of a force The force is Allied_Forces_1943 Therefore I need to Identify and test a strategic COG candidate corresponding to a member of a force The force is ? O 1 Therefore I need to Identify and test a strategic COG candidate for a force The force is US_1943 Identify and test a strategic COG candidate for a force The force is ? O 2 Knowledge-base constraints on the generalization: Any value of ? O 1 should be an instance of: DOMAIN(has_as_member) RANGE(The force is) = multi_member_force = multi_member_force Any value of ? O 2 should be an instance of: RANGE(has_as_member) RANGE(The force is) = force 106 2003, G. Tecuci, Learning Agents Laboratory
Overview Learning and problem solving agents: Disciple An agent for center of gravity analysis Modeling of problem solving through task reduction Knowledge base: Object ontology + Rules Rule-based problem solving Control of the problem solving process Control of modeling, learning and problem solving Multistrategy rule learning 2003, G. Tecuci, Learning Agents Laboratory Multistrategy rule refinement 108
4. The rule refinement problem (definition) GIVEN: • a plausible version space rule; • a positive or a negative example of the rule (i. e. a correct or an incorrect problem solving episode); • a knowledge base that includes an object ontology and a set of problem solving rules; • an expert that understands why the example is positive or negative, and can answer agent’s questions. DETERMINE: • an improved rule that covers the example if it is positive, or does not cover the example if it is negative; • an extended object ontology (if needed for rule refinement). 2003, G. Tecuci, Learning Agents Laboratory 109
Version space rule learning and refinement Let E 1 be the first task reduction from which the rule is learned. The agent learns a rule with a very specific lower bound condition (LB) and a very general upper bound condition (UB). UB LB+E 1 Let E 2 be a new task reduction generated by the agent and accepted as correct by the expert. Then the agent generalizes LB as little as possible to cover it. UB Let E 3 be a new task reduction generated by the agent which is rejected by the expert. Then the agent specialize UB as little as possible to uncover it and to remain more general than LB. UB After several iterations of this process LB may become identical with UB and a rule with an exact condition is learned. UB=LB _ + + 2003, G. Tecuci, Learning Agents Laboratory LB+ +E 2 LB+ + _ E 3 … _ _ 110
Rule refinement with a positive example Positive example that satisfies the upper bound I need to Identify and test a strategic COG candidate corresponding to a member of the European_Axis_1943 explanation ? O 1 has_as_member ? O 2 Therefore I need to Identify and test a strategic COG candidate for Germany_1943 Plausible Upper Bound Condition ? O 1 is multi_member_force has_as_member ? O 2 is force less general than IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ? O 1 Plausible Lower Bound Condition ? O 1 is equal_partners_multi_state_alliance has_as_member ? O 2 is single_state_force THEN Identify and test a strategic COG candidate for a force The force is ? O 2 2003, G. Tecuci, Learning Agents Laboratory Condition satisfied by positive example ? O 1 is European_Axis_1943 has_as_member ? O 3 ? O 2 is Germany_1943 explanation European_Axis_1943 has_as_member Germany_1943 111
Minimal generalization of the plausible lower bound Plausible Upper Bound Condition ? O 1 is multi_member_force has_as_member ? O 2 is force less general than (or at most as general as) New Plausible Lower Bound Condition ? O 1 is multi_state_alliance has_as_member ? O 2 is ? O 2 single_state_force minimal generalization Plausible Lower Bound Condition (from rule) Condition satisfied by the positive example ? O 1 is equal_partners_multi_state_alliance has_as_member ? O 2 ? O 1 is European_Axis_1943 has_as_member ? O 2 is single_state_force ? O 2 is Germany_1943 2003, G. Tecuci, Learning Agents Laboratory 113
Forces composition_of_forces force single_member_force single_state_force US_1943 multi_member_force single_group_force Germany_1943 multi_state_force multi_state_alliance is the minimal generalization of equals_partners_multi_state_alliance that covers European_Axis_1943 dominant_partner_ multi_state_alliance European_Axis_1943 … multi_state_coalition dominant_partner_ multi_state_coalition equal_partners_ multi_state_alliance Allied_Forces_1943 2003, G. Tecuci, Learning Agents Laboratory multi_group_force equal_partners_ multi_state_coalition 115
Refined rule IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ? O 1 explanation ? O 1 has_as_member ? O 2 Plausible Upper Bound Condition ? O 1 is multi_member_force has_as_member ? O 2 is force Plausible Lower Bound Condition ? O 1 is equal_partners_multi_state_alliance has_as_member ? O 2 is single_state_force THEN Identify and test a strategic COG candidate for a force The force is ? O 2 2003, G. Tecuci, Learning Agents Laboratory generalization IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ? O 1 Plausible Upper Bound Condition ? O 1 is multi_member_force has_as_member ? O 2 is force Plausible Lower Bound Condition ? O 1 is multi_state_alliance has_as_member ? O 2 is single_state_force THEN Identify and test a strategic COG candidate for a force The force is ? O 2 116
Demonstration Teaching Disciple to test leaders who are COG candidates Disciple 2003, G. Tecuci, Learning Agents Laboratory 117
Recommended reading Tecuci G. , Building Intelligent Agents: A Theory, Methodology, Tool and Case Studies, Academic Press, 1998. Tecuci G. , Boicu M. , Bowman M. , and Marcu M. , with a commentary by Burke M. : An Innovative Application from the DARPA Knowledge Bases Programs: Rapid Development of a High Performance Knowledge Base for Course of Action Critiquing, in AI Magazine, 22, 2, 2001, pp. 43 -61. AAAI Press, Menlo Park, California, 2001. http: //lalab. gmu. edu/publications/default. htm Describes the course of action domain. Tecuci G. , Boicu M. , Marcu D. , Stanescu B. , Boicu C. and Comello J. , Training and Using Disciple Agents: A Case Study in the Military Center of Gravity Analysis Domain, in AI Magazine, AAAI Press, Menlo Park, California, 2002. http: //lalab. gmu. edu/publications/default. htm 2003, G. Tecuci, Learning Agents Laboratory 118
Скачать презентацию 11 Learning and problem solving agents Prof Gheorghe
d0f305c73bee6072f5df4fd8c188f0ee.ppt