CES 510 Intelligent System Design B Ravikumar Department

CES 510 Intelligent System Design B. Ravikumar Department of Engg Science 116 I Darwin Hall 664 3335 ravi 93@gmail. com

Textbook · Chris Manning and Hinrich Shutze, Foundations of Statistical Natural Language Processing, MIT Press, 1999. · Various supplementary readings. Other Useful Books: Jurafsky & Martin, SPEECH and LANGUAGE PROCESSING: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition.

Overview of Artificial Intelligence • major applications • • • image processing and vision robotics game playing speech recognition natural language understanding etc.

What is Artificial Intelligence (John Mc. Carthy , Basic Questions) · · What is artificial intelligence? It is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable. · · Yes, but what is intelligence? Intelligence is the computational part of the ability to achieve goals in the world. Varying kinds and degrees of intelligence occur in people, many animals and some machines. · · Isn't there a solid definition of intelligence that doesn't depend on relating it to human intelligence? Not yet. The problem is that we cannot yet characterize in general what kinds of computational procedures we want to call intelligent. We understand some of the mechanisms of intelligence and not others. · More in: http: //www-formal. stanford. edu/jmc/whatisai/node 1. html

What is Artificial Intelligence? · Human-like (“How to simulate humans intellect and behavior on by a machine. ) • Mathematical problems (puzzles, games, theorems) • Common-sense reasoning (if there is parking-space, probably illegal to park) • Expert knowledge: lawyers, medicine, diagnosis • Social behavior · Rational-like: • achieve goals, have performance measure

What is Artificial Intelligence · Thought processes • “The exciting new effort to make computers think. . Machines with minds, in the full and literal sense” (Haugeland, 1985) · Behavior • “The study of how to make computers do things at which, at the moment, people are better. ” (Rich, and Knight, 1991)

The Turing Test (Can Machine think? A. M. Turing, 1950) · Requires • • • Natural language Knowledge representation Automated reasoning Machine learning (vision, robotics) for full test

What is AI? · · Turing test (1950) Requires: • • • · Natural language Knowledge representation automated reasoning machine learning (vision, robotics. ) for full test Thinking humanly: • Introspection, the general problem solver (Newell and Simon 1961) • Cognitive sciences · Thinking rationally: • Logic • Problems: how to represent and reason in a domain · Acting rationally: • Agents: Perceive and act

History of AI · Mc. Culloch and Pitts (1943) • Neural networks that learn · Minsky (1951) • Built a neural net computer · Darmouth conference (1956): • Mc. Carthy, Minsky, Newell, Simon met, • Logic theorist (LT)- proves a theorem in Principia Mathematica-Russel. • The name “Artficial Intelligence” was coined. · 1952 -1969 • • • GPS- Newell and Simon Geometry theorem prover - Gelernter (1959) Samuel Checkers that learns (1952) Mc. Carthy - Lisp (1958), Advice Taker, Robinson’s resolution Microworlds: Integration, block-worlds. 1962 - the perceptron convergence (Rosenblatt)

The Birthplace of “Artificial Intelligence”, 1956 · Darmouth workshop, 1956: historical meeting of the perceived founders of AI met: John Mc. Carthy, Marvin Minsky, Alan Newell, and Herbert Simon. · A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence. J. Mc. Carthy, M. L. Minsky, N. Rochester, and C. E. Shannon. August 31, 1955. "We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College in Hanover, New Hampshire. The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. " And this marks the debut of the term "artificial intelligence. “

History, continued · 1966 -1974 a dose of reality • Problems with computation · 1969 -1979 Knowledge-based system • Expert systems: — Dendral: Inferring molecular structures — Mycin: diagnosing blood infections — Prospector: recomending exploratory drilling (Duda). · · 1986 -present: return to neural networks Machine learning theory · · Genetic algorithms, genetic programming Statistical approaches and data mining

State of the art · · · · Deep Blue defeated the reigning world chess champion Garry Kasparov in 1997 Proved a mathematical conjecture (Robbins conjecture) unsolved for decades No hands across America (driving autonomously 98% of the time from Pittsburgh to San Diego) During the 1991 Gulf War, US forces deployed an AI logistics planning and scheduling program that involved up to 50, 000 vehicles, cargo, and people NASA's on-board autonomous planning program controlled the scheduling of operations for a spacecraft Proverb solves crossword puzzles better than most humans DARPA grand challenge 2003 -2005, Robocup

What’s involved in Intelligence? Intelligent agents · Ability to interact with the real world • • · to perceive, understand, and act e. g. , speech recognition and understanding and synthesis e. g. , image understanding e. g. , ability to take actions, have an effect Knowledge Representation, Reasoning and Planning • modeling the external world, given input • solving new problems, planning and making decisions • ability to deal with unexpected problems, uncertainties · Learning and Adaptation • we are continuously learning and adapting • our internal models are always being “updated” — e. g. a baby learning to categorize and recognize animals

Course overview n n n Intelligent systems are autonomous systems (hardware / software or a combination) that behaves as if it exhibits some form of intelligence. Concept goes back to Alan Turing who thought about machine intelligence and devised Turing test to distinguish a machine from a human through interaction. Some major areas: n n Symbolic information processing – deductive systems Game playing – chess, backgammon etc. natural language understanding – answering queries, translation, text classification etc. Machine learning - adaptive behavior through stimulus n n n Neural networks Statistical modeling Fuzzy logic, genetic programming etc.

Course overview n n In this course we will introduce statistical techniques for inferring structure from text. The aim of the course is to introduce existing techniques in statistical NLP and to stimulate thought into bettering these. Applications of NLP n n Information Retrieval Information Extraction Natural language interface to database Statistical Machine Translation

Tools n n n Probability Theory Information Theory Algorithms Data Structures Probabilistic AI Grammars and automata

The Steps in NLP

The steps in NLP (Cont. ) n n n Morphology: Concerns the way words are built up from smaller meaning bearing units. (come(s), co(mes)) Syntax: concerns how words are put together to form correct sentences and what structural role each word has. Semantics: concerns what words mean and how these meanings combine in sentences to form sentence meanings. Pragmatics: concerns how sentences are used in different situations and how use affects the interpretation of the sentence. Discourse: concerns how the immediately preceding sentences affect the interpretation of the next sentence.

Parsing (Syntactic Analysis) n Assigning a syntactic and logical form to an input sentence n n uses knowledge about word and word meanings (lexicon) uses a set of rules defining legal structures (grammar) (S (NP (NAME Sam)) (VP (V ate) (NP (ART the) (N apple)))) I made her duck.

Word Sense Resolution n Many words have many meanings or senses. We need to resolve which of the senses of an ambiguous word is invoked in a particular use of the word. I made her duck. (made her a bird for lunch or made her move her head quickly downwards? )

Reference Resolution n Domain Knowledge (banking transaction) Discourse Knowledge World Knowledge U: I would like to open a fixed deposit account. S: For what amount? U: Make it for 800 dollars. S: For what duration? U: What is the interest rate for 3 months? S: Six percent. U: Oh good then make it for that duration.

Why NLP is difficult? n n n n n Different ways of Parsing a sentence Word category ambiguity Word sense ambiguity Words can mean more than their sum of parts (The Times of India) Imparting world knowledge is difficult ("the blue pen ate the ice-cream") Fictitious worlds ("people on mars can fly") Defining scope ("people like ice-cream, " does this mean all people like ice cream? ) Language is changing and evolving Complex ways of interaction between the kinds of knowledge exponential complexity at each point in using the knowledge

Inferring Knowledge from text n n Words n word frequencies n collocations n word sense n n-grams (words appear in certain order) Grammar n word categories n syntactic structure Discourse n Sentence meanings Applications n Information Retrieval n Information Extraction n Natural language interface n Statistical Machine Translation

Simple Applications n n n Word counters (wc in UNIX) Spell Checkers, grammar checkers Predictive Text on mobile handsets

More significant Applications n n n n n Intelligent computer systems NLU interfaces to databases Computer aided instruction, automatic graders Information retrieval Intelligent Web searching Data mining Machine translation Speech recognition Natural language generation Question answering

Spoken Dialogue System Us e r Speech Recognition Speech Synthesis Semantic Interpretation Discourse Interpretation Response Generation Dialogue Management

Parts of the Spoken Dialogue System n n n Signal Processing: n Convert the audio wave into a sequence of feature vectors. Speech Recognition: n Decode the sequence of feature vectors into a sequence of words. Semantic Interpretation: n Determine the meaning of the words. Discourse Interpretation: n Understand what the user intends by interpreting utterances in context. Dialogue Management: n Determine system goals in response to user utterances based on user intention. Speech Synthesis: n Generate synthetic speech as a response.

Levels of Sophistication in a Dialogue System n n n Touch-tone replacement: System Prompt: "For checking information, press or say one. " Caller Response: "One. " Directed dialogue: System Prompt: "Would you like checking account information or rate information? " Caller Response: "Checking", or "checking account, " or "rates. " Natural language: System Prompt: "What transaction would you like to perform? " Caller Response: "Transfer Rs. 500 from checking to savings. "