Introduction Introduction What is AI

Introduction

Introduction • What is AI? • The foundations of AI • A brief history of AI • The state of the art 2

What is AI? - definition • Intelligence: “ability to learn, understand think” (Oxford dictionary) • AI is the study of how to make computers make things which at the moment people do better. • Examples: Speech recognition, Smell, Face, Object, Intuition, Inferencing, Learning new skills, Decision making, Abstract thinking 3

AI Definitions What is AI ? • A broad field which means different things to different people • Concerned with getting computers to do tasks that require human intelligence There are many tasks which we might reasonably think require intelligence which computers do without even thinking However There are many tasks that people do without thinking which are extremely difficult to automate Recognizing a Face Complex Arithmetic

What is AI? 2. Thinking humanly 3. Thinking rationally 1. Acting humanly 4. Acting rationally • The top row is concerned with thought processes and reasoning • Second row address behavior 5

Do human behavior is rational? • A system is rational, if it does the right thing, given what it knows • We can distinguish human and rational behavior and say, human are not rational, ie. , Irrational (emotionally unstable) • However, we are not perfect • Ex: Not everyone gets same grade (A) on the exam

AI Definitions What is AI ? Definitions organized into four categories Think like human The exciting new effort to make computers think … machines with minds, in the full and literal sense. Think Rationally The study of the computations that make it possible to perceive, reason, and act. [Winston, 1992] Act humanly The study of how to make computers do things at which, at the moment, people are better. Act rationally The branch of computer science that is concerned with the automation of intelligent behavior. [Luger and [Haugeland 85]. [Rich & Knight, 1991] Stubblefield, 1993]

1. Systems that act like humans • The overall behaviour of the system should be human like. • It could be achieved by observation. 8

1. Acting Humanly: The Turing Test • Alan Turing (1912 -1954) • “Computing Machinery and Intelligence” (1950) Imitation Game Human Interrogator AI System 9

Turing Test ? • You enter a room which has a computer terminal. • You have a fixed period of time to type what you want into the terminal, and study the replies. • At the other end of the line is either a human being or a computer system. • If it is a computer system, and at the end of the period you cannot reliably determine whether it is a system or a human, then the system is deemed to be intelligent.

Turing Test computer person tester 11

Recognizing AI • The Turing test is one criterion, but it’s controversial – It’s an imitation game, in which a human interrogator is isolated in a room, with teletype connections to an unseen human and an unseen computer – The interrogator asks the same questions of the human and the machine and tries to determine which is which – If the interrogator can’t tell them apart, the computer is intelligent • Positive features of the Turing test – It’s objective and unbiased – It’s independent of how the machine operates – There are no agreed upon alternative tests

Recognizing AI • Negative features of the Turing test – It overlooks aspects of intelligence such as perception and mobility – It overlooks human intelligence – It can be gimmicky and detract from serious AI research efforts • What is the amount of knowledge that a machine would need to pass Turing test ?

What sort of Functionality Is Needed? • To act humanly? – – – Natural language processing Knowledge Representation Automated Reasoning Machine Learning Computer Vision Robotics 14

Act Like Human Computer would need: 1. Natural Language Processing Communication. 2. Knowledge Representation store info before and during interrogation. 3. Automated Reasoning answer questions and draw new conclusions. 4. Machine learning adapt to new circumstances. 5. Computer vision to perceive objects 6. Robotics to manipulate objects and move about

Turing test • Turing test avoided physical interaction between the interrogator and the computer • Turing test involves first four aspects (1 -4) • Total Turing test includes a video signal so that the interrogator can test the perceptual abilities as well as to pass physical objects • Total Turing test involves last two aspects (5 – 6)

2. Think Like Human The Cognitive Modeling approach • To develop a program that think like human, the way the human think should be known. • Knowing the precise theory of mind ( how human think? ) expressing theory as a computer program. • GPS (General Problem Solver) [ by Newell & Simon, 1961] • Concerned with comparing the trace of its reasoning steps to traces of human subjects solving the same problem rather than correctly solve problems

2. Thinking Humanly: Cognitive Modelling • Not content to have a program correctly solving a problem. • More concerned with comparing its reasoning steps to traces of human solving the same problem. Cognitive science: • Computer models from AI + Experimental techniques from psychology Construction of human mind working theories • Requires testable theories of the workings of the human mind • Requires experimental investigation of actual humans or animals • Note: We will not pursue human mind theory here as we have only a computer for experimentation 18

2. Systems that think like humans • Most of the time it is a black box where we are not clear about our thought process. • One has to know functioning of brain and its mechanism for possessing information. • It is an area of cognitive science. – The stimuli are converted into mental representation. – Cognitive processes manipulate representation to build new representations that are used to generate actions. • Neural network is a computing model for processing information similar to brain. 19

3. Thinking Rationally: Laws of Thought • Aristotle was one of the first to attempt to codify “right thinking”, i. e. , irrefutable reasoning processes. 20

3. Think Rationally The Law of Thought Approach Aristotle and his syllogism ( right thinking) : always gave correct conclusions given correct premises Ex: • • • Socrates is a Man. %Fact All men are Mortal. % Rule if X is a Man then X is Mortal. Therefore Socrates is Mortal. % Inference These laws of thoughts initiated the field of LOGIC. Formal logic provides a precise notation and rules for representing and reasoning with all kinds of things in the world.

3. Think Rationally • Two main obstacles 1. Not easy to translate an informal knowledge into a formal logic. Ex: when the knowledge is less than 100% certain 2. There is a difference between solving a problem “in principle” and solving it “in practice” Ex: It is usually the case that problems with few hundred facts – Can exhaust the computational power of any computer. – So, it is required to have some guidance as to which reasoning steps to try first Note: Both obstacles apply while building any computational reasoning systems. – Thus the need for heuristics.

4. Acting Rationally The Rational Agent Approach • Agent • agent is something that acts • • • Computer agents Operate autonomously perceive their environments Persist over a prolonged time period Adapt to change, create and pursue goals • Rational agent • Acts so as to achieve the best outcome or when there is uncertainty, the best expected outcome

4. Acting Rationally • • Laws of thought approach vs. Act rationally Laws of thought approach Emphasis on correct inference Making correct inferences is part of being rational agent • Act rationally = reason logically to the conclusion that a given action will achieve one’s goals and then act on that conclusion

4. Acting Rationally • Correct inference is not always == rationality • Ex: even if there is no correct thing to do, something must be done • Hence, it means that there also ways of acting rationally that cannot be said to involve inference • Ex: reflex actions ( acting rationally without involving inference) • Fast action is required than careful deliberation

4. Acting Rationally Two main advantages of rational agent approaches 1. More general than “the laws of thought” approach ( correct inference is one of several possible mechanism to achieve rationality) 2. More amenable to scientific development than approaches based on human behavior/thought. Rationality vs. human behavior • The rationality is mathematically well defined, general (many agent design have been generated to achieve it) • Human behavior adapted for one specific environment, defined by sum total of all the things that humans do

4. Acting Rationally • We focus on general principles of rational agents and on components for constructing them • perfect rationality • Always doing the right thing is not feasible in complicated • • environments because of high computational demands But a good starting point for analysis It simplifies the problem • Limited rationality • Acting appropriately when there is not enough time to do all • the computations Not in the syllabus 27

The foundations of AI

The foundations of AI Philosophy Knowledge Rep. , Logic, Foundation of AI (is AI possible? ) Maths Search, Analysis of search algos, logic Economics Expert Systems, Decision Theory, Principles of Rational Behavior Psychology Behavioristic insights into AI programs Neuroscience (Brain Science) Learning, Neural Nets Control theory and Cybernetics Information Theory & AI, Entropy, Robotics Computer Sc. & Engg. Systems for AI

The Foundations of AI 1. Philosophy (423 BC - present): • • Can formal rules be used to draw valid conclusions? How does the mind arise from a physical brain? Where does knowledge come from? How does knowledge lead to action? - Logic, methods of reasoning. - Mind as a physical system. - Foundations of learning, language, and rationality. 30

The Foundations of AI 2. Mathematics (c. 800 - present): • What are the formal rules to draw valid conclusions? • What can be computed? • How do we reason with uncertain information? • Three areas developed– Logic, Computation and Probability 31

The Foundations of AI 2. Mathematics (c. 800 - present): • A) Formal representation and proof • Development of Formal logic – 1) Propositional or Boolean logic – 2) Development of First-Order logic by extending the Boolean logic to include objects and relations • B) Algorithms, computation, decidability, tractability • First algorithm was developed - Euclid’s algorithm for computing GCD • C) Probability – Baye’s rule is the underlying approach for uncertain reasoning in AI systems. 32

The Foundations of AI 3. Economics : • How should we make decisions so as to maximize payoff? • How should we do this when others may not go along? • How should we do this when the pay off may be far in the future ? 33

The Foundations of AI 3. Economics : • • Decision theory Game theory Operations Research Markov Decision Processes 34

The Foundations of AI 4. Neuroscience: • • The study of nervous system, particularly the brain The exact way in which the brain enables thought is unknown However, it does enable thought has the evidence “A strong blow to the head can lead to mental incapacitation” • Neurons – Brain consists of nerve cells or neurons • There is no theory on how an individual memory is stored • A collection of simple cells can lead to thought, action, consciousness that “brains cause minds” 35

The Foundations of AI 4. Neuroscience: • The parts of a nerve cell or neuron. • Each neuron consists of a cell body or soma, that contains a cell nucleus. • Branching out from the cell body are a number of fibers called dendrites and a single long fiber called the axon. • A neuron makes connections with 10 to 100, 000 other neurons at junctions called synapses. • Signals are propagated from neuron to neuron by a complicated electrochemical reaction. • The signals control brain activity in the short term and also enable long-term changes in the connectivity of neuron. • These mechanisms are thought to form the basis for learning 36 in the brain.

The Foundations of AI 4. Neuroscience: The parts of a nerve cell or neuron. 37

The Foundations of AI 4. Neuroscience: A comparison of resources available at IBM BLUE GENE Supercomputer , a PC of 2008 and the human brain • Note: The brain’s numbers are fixed whereas the supercomputer’s numbers increase by a factor of 10 every 5 year or so • The PC lags behind on all metrics except cycle time 38

The Foundations of AI 4. Neuroscience: • Brains and digital computers have somewhat different properties. • computers have a cycle time that is a million times faster than a brain. • The brain makes up for that with far more storage and interconnection than even a high-end personal computer • The largest supercomputers have a capacity that is similar to the brain's. • Even with a computer of virtually unlimited capacity, we still would not know how to achieve the brain's level of intelligence 39

The Foundations of AI 4. Neuroscience: • How do the brain works? – Early studies (1824) relied on injured and abnormal people to understand what parts of brain work • More recent studies use accurate sensors to correlate brain activity to human thought – The measurement of intact brain activity began in 1929 with the invention by Hans Berger of the electroencephalograph (EEG). – The recent development of functional magnetic resonance imaging (f. MRI) is giving neuroscientists detailed images of brain activity – Individual neurons can be stimulated electrically, chemically, or even optically allowing neuronal input output relationships to be mapped. • Despite these advances, we are still a long way from understanding how cognitive processes actually work. 40

The Foundations of AI 5. Psychology (1879 - present): • How do humans and animals think and act? - Adaptation. - Phenomena of perception and motor control. - Experimental techniques. 41

The Foundations of AI 6. Computer Engineering: • How can we build an efficient computer? • • For artificial intelligence to succeed, we need two things: intelligence and an artifact. The computer has been the artifact of choice Each generation of computer hardware has brought an increase in speed and capacity and a decrease in price 42

The Foundations of AI 6. Computer Engineering: • In 2005 power dissipation problems led to multiply the number of CPU cores rather than the clock speed. • Current expectations are that future increases in power will come from massive parallelism – Convergence with the properties of the brain. • The software side of computer science has supplied AI with: – The operating systems, programming languages, and tools needed to write modem programs 43

The Foundations of AI 7. Control theory and cybernetics: • How can artifacts operate under their own control? – Previous Assumption: Only living things could modify their behavior in response to changes in the environment – Machines can modify their behavior in response to the environment (sense/action loop) • Ex: Water-flow regulator, steam engine governor, thermostat • The theory of stable feedback systems (1894) • Build systems that transition state to goal state with minimum energy • In 1950, control theory could linear systems • from only AI largely rose as a response to this shortcoming initial describe 44

The Foundations of AI 8. Linguistics (1957 - present): - How does language relate to thought? • Knowledge representation: the study of how to put knowledge into a form that a computer can reason with • Grammar • Speech demonstrates human intelligence – Analysis of human language reveals thought taking place in ways not understood in other settings • Language and thought are believed to be tightly intertwined 45

The Foundations of AI 8. Linguistics (1957 - present): • Modern linguistics and AI intersect in a hybrid field called computational linguistics or natural language processing • 46

Summary - The Foundations of AI • Philosophy • Mathematics • Economics • Neuroscience • Psychology • Computer engineering • Control theory • Linguistics Logic, methods of reasoning, mind as physical system foundations of learning, language, rationality Formal representation and proof algorithms, computation, (un)decidability, (in)tractability, probability utility, decision theory physical substrate for mental activity phenomena of perception and motor control, experimental techniques building fast computers design systems that maximize an objective function over time knowledge representation, grammar

A Brief History of AI

A Brief History of AI • The gestation of AI (1943 - 1956): - 1943: Mc. Culloch & Pitts: Boolean circuit model of brain. - 1950: Turing’s “Computing Machinery and Intelligence”. - 1956: Mc. Carthy’s name “Artificial Intelligence” adopted. • Why was it necessary for AI to become a separate field ? • Why couldn't all the work done in AI have taken place under the name of control theory or operations research or decision theory, which have objectives similar to those of AI? • Why isn't AI a branch of mathematics? • The first answer is that AI from the start embraced the idea of duplicating human faculties such as creativity, self-improvement, and language use. • None of the other fields were addressing these issues. • The second answer is methodology. AI is the only one of these fields that is clearly a branch of computer science (USING COMPUTER SIMULATIONS) • AI is the only field to attempt to build machines that will function autonomously in complex, changing environments 49

A Brief History of AI • Early enthusiasm, great expectations (1952 - 1969): • Early successful AI programs: – Newell & Simon’s Logic Theorist – Newell & Simon’s General Problem Solver (GPS) to imitate human problem solving • Solved puzzles and considered sub goals and possible actions similar to which humans approached the same problems. • Thus, GPS was probably the first program to embody the "thinking humanly" approach – Newell & Simon’s Physical symbol system hypothesis: • which states that "a physical symbol system has the necessary and sufficient means for general intelligent action. " • Any system (human or machine) exhibiting intelligence must operate by manipulating data structures composed of symbols. 50

A Brief History of AI • Early enthusiasm, great expectations (1952 1969): • Early successful AI programs: – – Samuel’s Checkers Gelernter’s Geometry Theorem Prover Robinson’s complete algorithm for logical reasoning. At 1958 in MIT AI Lab Mc. Carthy defined the highlevel language Lisp • which was the dominant AI programming language for the next 30 years. 51

A Brief History of AI • A dose of reality (1966 - 1973): • The first kind of difficulty arose because most early programs knew nothing of their subject matter • They succeeded by means of simple syntactic manipulations • Early machine translation efforts to speed up the translation of Russian scientific papers in the wake of the Sputnik launch in 1957. • It was thought initially that simple syntactic transformations based on the grammars of Russian and English, and word replacement from an electronic dictionary, would suffice to preserve the exact meanings of sentences. • The fact is that accurate translation requires background knowledge in order to resolve ambiguity and establish the content of the sentence. 52

A Brief History of AI • A dose of reality (1966 - 1973): • The second kind of difficulty was the intractability of many of the problems • Most of the early AI programs solved problems by trying out different combinations of steps until the solution was found. • This strategy worked initially with limited objects and hence very few possible actions and very short solution sequences. • Before theory of computational complexity was developed, it was widely thought that "scaling up" to larger problems was simply a matter of faster hardware and larger memories. 53

A Brief History of AI • Knowledge-based systems (1969 - 1979): • Uses more powerful, domain-specific knowledge • Allows larger reasoning steps and • Can more easily handle typical expertise. • Ex 1 : 1969: DENDRAL by Buchanan et al. – generated all possible structures consistent with the formula of the molecule – The significance of DENDRAL was that it was the first successful knowledge-intensive system: – its expertise derived from large numbers of special-purpose rules. 54

A Brief History of AI • Knowledge-based systems (1969 - 1979): • Ex 2: 1976: MYCIN by Shortliffle • Expert systems in medical diagnosis • With about 450 rules, MYCIN was better than junior doctors. • Two major differences from DENDRAL. – First, unlike the DENDRAL rules, no general theoretical model existed from which the MYCIN rules could be deduced. – They had to be acquired from extensive interviewing of experts, who in turn acquired them from textbooks, other experts, and direct experience of cases. – Second, the rules had to reflect the uncertainty associated with medical knowledge. – MYCIN incorporated a calculus of uncertainty called certainty factors which seemed (at the time) to fit well with how doctors assessed the impact of evidence on the diagnosis. 55

A Brief History of AI • AI becomes an industry (1980 - present): - Expert systems industry booms. - 1981: Japan’s 10 -year Fifth Generation project to build intelligent computers. 56

A Brief History of AI • The return of NNs (1986 - present): - Mid 80’s: Back-propagation learning algorithm reinvented. - 1988: Resurgence of probability. - 57

A Brief History of AI • AI adopts the scientific method (1987 - present): • In terms of methodology, AI has finally come firmly under the scientific method. • To be accepted, hypotheses must be subjected to rigorous empirical experiments – and the results must be analyzed statistically for their importance – It is now possible to replicate experiments by using shared repositories of test data and code. 58

A Brief History of AI • The emergence of intelligent agents (1995 - present) • One of the most important environments for intelligent agents is the Internet. • AI systems have become so common in Web-based applications that the "-bot'' suffix has entered everyday language. • Internet tools, such as search engines, recommender systems, and Web site aggregators use AI technologies 59

A Brief History of AI • The availability of very large data sets( 2001 -present) • In the 60 -year history of computer science, the emphasis has been on the algorithm as the main subject of study. • But recent work in AI suggests that for many problems, it makes more sense to worry about the data • This is true because of the increasing availability of very large data sources • Examples: • trillions of words of English, billions of images from the Web, or billions of base pairs of genomic sequences 60

A Brief History of AI • The availability of very large data sets( 2001 -present) • Example 2: • Hays and Efros (2007) discuss the problem of filling in holes in a photograph. • Suppose you use Photoshop to mask out an ex-friend from a group photo, but now you need to fill in the masked area with something that matches the background. • Hays and Efros defined an algorithm that searches through a collection of photos to find something that will match. • They found the performance of their algorithm was poor when they used a collection of only ten thousand photos • but excellent performance when they grew the collection to two million photos. 61

A Brief History of AI • The availability of very large data sets( 2001 present) • Work like this suggests the "knowledge bottleneck'' in AI • The problem of how to express all the knowledge that a system needs • may be solved in many applications by learning methods rather than hand-coded knowledge engineering 62

Summary - Abridged history of AI • • 1943 1950 1956 1952— 69 • 1950 s • 1965 • 1966— 73 • • • 1969— 79 1980 -1986 -1987 -1995 -- Mc. Culloch & Pitts: Boolean circuit model of brain Turing's "Computing Machinery and Intelligence" Dartmouth meeting: "Artificial Intelligence" adopted Look, Ma, no hands! – long list of Xs and a belief that “ a machine can never do X” Early AI programs, including Samuel's checkers program, Newell & Simon's Logic Theorist, Gelernter's Geometry Engine Robinson's complete algorithm for logical reasoning AI discovers computational complexity Neural network research almost disappears Early development of knowledge-based systems AI becomes an industry Neural networks return to popularity AI becomes a science The emergence of intelligent agents

State of the art • What can AI do today? • A few examples of artificial intelligence systems that exist today • 1. Robotic vehicles: • A driverless robotic car named STANLEY sped through the rough terrain of the Mojave dessert at 22 mph, finishing the 132 -mile course first to win the 2005 DARPA Grand Challenge. • STANLEY is fitted with cameras, radar, and laser range finders to sense the environment and on board software to command the steering braking, and acceleration (Thrun, 2006). • The following year CMU's Boss won the Urban Challenge, safely driving in traffic through the streets of a closed Air Force base, obeying traffic rules and avoiding pedestrians and other vehicles

State of the art • What can AI do today? • A few examples of artificial intelligence systems that exist today • 2. Speech recognition: • A traveler calling United Airlines to book a flight • can have the entire conversation guided by an automated speech recognition and dialog management system.

State of the art • What can AI do today? • A few examples of artificial intelligence systems that exist today • 3. Autonomous planning and scheduling: • A hundred million miles from Earth, NASA's Remote Agent program became the first on-board autonomous planning program • to control the scheduling of operations for a spacecraft

State of the art • What can AI do today? • A few examples of artificial intelligence systems that exist today • 4. Game playing: • IBM’s Deep Blue defeated the reigning world chess champion Garry Kasparov in 1997 • Human champions studied Kasparov's loss • were able to draw a few matches in subsequent years • but the most recent human-computer matches have been won convincingly by the computer.

State of the art • What can AI do today? • 5. Spam fighting: • Each day, learning algorithms classify over a billion messages as spam • saving the recipient from having to waste time deleting • for many users it could comprise 80% or 90% of all messages, if not classified away by algorithms. • Because the spammers are continually updating their tactics • it is difficult for a static programmed approach to keep up and learning algorithms work best

State of the art • What can AI do today? • 6. Logistics planning: • During the 1991 Gulf War, US forces deployed an AI logistics planning and scheduling program • that involved up to 50, 000 vehicles, cargo, and people • Parameters are starting points, destinations, routes, and conflict resolution • 7. Robotics: • The i. Robot Corporation has sold over two million Roomba robotic vacuum cleaners for home use.

State of the art • What can AI do today? • 8. Machine Translation: • A computer program automatically translates from Arabic to English • The program uses a statistical model built from examples of Arabic-to-English translations and from examples of English text • There are two trillion words (Brants et at. , 2007). • None of the computer scientists on the team speak Arabic

State of the art • 9. No hands across America (driving autonomously 98% of the time from Pittsburgh to San Diego) • 10. Proverb solves crossword puzzles better than most humans

State of the art • Science, engineering, and mathematics of AI is therefore required