1 Introduction Prof Gheorghe Tecuci Learning Agents Laboratory

1. Introduction Prof. Gheorghe Tecuci Learning Agents Laboratory Computer Science Department George Mason University 1

Overview What is Artificial Intelligence What is Machine Learning History of Machine Learning Basic bibliography and reading 2

What is Artificial Intelligence is the Science and Engineering that is concerned with theory and practice of developing systems that exhibit the characteristics we associate with intelligence in human behavior: perception, natural language processing, reasoning, planning and problem solving, learning and adaptation, etc. 3

Central goals of Artificial Intelligence Understand the principles that make intelligence possible (in humans, animals, and artificial agents) Developing intelligent machines or agents (no matter whether they operate as humans or not) Formalizing knowledge and mechanizing reasoning in all areas of human endeavor Making the working with computers as easy as working with people Developing human-machine systems that exploit the complementariness of human and automated reasoning 4

What is an intelligent agent An intelligent agent is a system that: • perceives its environment (which may be the physical world, a user via a graphical user interface, a collection of other agents, the Internet, or other complex environment); • reasons to interpret perceptions, draw inferences, solve problems, and determine actions; and • acts upon that environment to realize a set of goals or tasks for which it was designed. input/ sensors user/ environment output/ effectors Intelligent Agent 5

Characteristic features of intelligent agents Knowledge representation and reasoning Transparency and explanations Ability to communicate Use of huge amounts of knowledge Exploration of huge search spaces Use of heuristics Reasoning with incomplete or conflicting data Ability to learn and adapt 6

Overview What is Artificial Intelligence What is Machine Learning History of Machine Learning Basic bibliography and reading 7

What is Machine Learning is the domain of Artificial Intelligence which is concerned with building adaptive computer systems that are able to improve their competence and/or efficiency through learning from input data or from their own problem solving experience. 8

The architecture of a learning agent Implements a general problem solving method that uses the knowledge from the knowledge base to interpret the input and provide an appropriate output. Learning Agent Input/ Sensors User/ Environment Problem Solving Engine Learning Engine Output/ Effectors Knowledge Base Ontology Rules/Cases/Methods Implements learning methods for extending and refining the knowledge base to improve agent’s competence and/or efficiency in problem solving. Data structures that represent the objects from the application domain, general laws governing them, actions that can be performed with them, etc. 9

What is Learning? Learning denotes changes in the system that are adaptive in the sense that they enable the system to do the same task or tasks drawn from the same population more effectively the next time (Simon, 1983). Learning is making useful changes in our minds (Minsky, 1985). Learning is constructing or modifying representations of what is being experienced (Michalski, 1986). A computer program learns if it improves its performance at some task through experience (Mitchell, 1997). 10

So what is Learning? Learning is a very general term denoting the way in which people and computers: (1) acquire and organize knowledge (by building, modifying and organizing internal representations of some external reality); (2) discover new knowledge and theories (by creating hypotheses that explain some data or phenomena); (3) acquire skills (by gradually improving their motor or cognitive skills through repeated practice, sometimes involving little or no conscious thought). Learning results in changes in the agent (or mind) that improve its competence and/or efficiency. 11

Two complementary dimensions for learning Competence A system is improving its competence if it learns to solve a broader class of problems, and to make fewer mistakes in problem solving. Efficiency A system is improving its efficiency, if it learns to solve the problems from its area of competence faster or by using fewer resources. 12

Main directions of research in Machine Learning Discovery of general principles, methods, and algorithms of learning Automation of the construction of knowledge-based systems Modeling human learning mechanisms 13

Learning strategies A Learning Strategy is a basic form of learning characterized by the employment of a certain type of inference (like deduction, induction or analogy) and a certain type of computational or representational mechanism (like rules, trees, neural networks, etc. ). • Rote learning • Instance-based learning • Learning from instruction • Reinforcement learning • Learning from examples • Neural networks • Explanation-based learning • Genetic algorithms and evolutionary computation • Conceptual clustering • Quantitative discovery • Abductive learning • Learning by analogy • Reinforcement learning • Bayesian learning • Multistrategy learning 14

Overview What is Artificial Intelligence What is Machine Learning History of Machine Learning Basic bibliography and reading 15

History of Machine Learning Early enthusiasm (1955 - 1965) • Learning without knowledge; • Neural modeling (self-organizing systems and decision space techniques); • Evolutionary learning; • Rote learning (Samuel Checker’s player). 16

History of Machine Learning (cont. ) Dark ages (1962 - 1976) • To acquire knowledge one needs knowledge; • Realization of the difficulty of the learning process and of the limitations of the explored methods (e. g. the perceptron cannot learn the XOR function); • Symbolic concept learning (Winston’s influential thesis, 1972). 17

History of Machine Learning (cont. ) Renaissance (1976 - 1988) • Exploration of different strategies (EBL, CBR, GA, NN, Abduction, Analogy, etc. ); • Knowledge-intensive learning; • Successful applications; • Machine Learning conferences/workshops worldwide. 18

History of Machine Learning (cont. ) Maturity (1988 - present) • Experimental comparisons; • Revival of non-symbolic methods; • Computational learning theory; • Multistrategy learning; • Integration of machine learning and knowledge acquisition; • Emphasis on practical applications. 19

Successful applications of Machine Learning • Learning to recognize spoken words (all of the most successful systems use machine learning); • Learning to drive an autonomous vehicle on public highway; • Learning to classify new astronomical structures (by learning regularities in a very large data base of image data); • Learning to play games; • Automation of knowledge acquisition from domain experts; • Learning agents. 20

Basic bibliography Mitchell T. M. , Machine Learning, Mc. Graw Hill, 1997. Shavlik J. W. and Dietterich T. (Eds. ), Readings in Machine Learning, Morgan Kaufmann, 1990. Buchanan B. , Wilkins D. (Eds. ), Readings in Knowledge Acquisition and Learning: Automating the Construction and the Improvement of Programs, Morgan Kaufmann, 1992. Langley P. , Elements of Machine Learning, Morgan Kaufmann, 1996. Michalski R. S. , Carbonell J. G. , Mitchell T. M. (Eds), Machine Learning: An Artificial Intelligence Approach, Morgan Kaufmann, 1983 (Vol. 1), 1986 (Vol. 2). Kodratoff Y. and Michalski R. S. (Eds. ) Machine Learning: An Artificial Intelligence Approach (Vol. 3), Morgan Kaufmann Publishers, Inc. , 1990. Michalski R. S. and Tecuci G. (Eds. ), Machine Learning: A Multistrategy Approach (Vol. 4), Morgan Kaufmann Publishers, San Mateo, CA, 1994. Tecuci G. and Kodratoff Y. (Eds. ), Machine Learning and Knowledge Acquisition: Integrated Approaches, Academic Press, 1995. Tecuci G. , Building Intelligent Agents: An Apprenticeship Multistrategy Learning Theory, Methodology, Tool and Case Studies, Academic Press, 1998. 21

Recommended reading Mitchell T. M. , Machine Learning, Chapter 1: Introduction, pp. 1 -19, Mc. Graw Hill, 1997. 22