Artificial Intelligence Introduction Chapter Data Driven Methods

Artificial Intelligence Introduction Chapter

Data Driven Methods What is Artificial Intelligence? Common AI techniques Choosing between AI techniques Pre and post processing Powered by De. Sia. More 2

What is AI? • Various definitions: – Building intelligent entities. – Getting computers to do tasks which require human intelligence. • But what is “intelligence”? • Simple things turn out to be the hardest to automate: – Recognising a face. – Navigating a busy street. – Understanding what someone says. • All tasks require reasoning. De. Sia. More on knowledge. Powered by 3

What is AI? • Machines that perceive, understand react to their environment – Goal of Babbage, etc. – Oldest endeavor in computer science • Machines that think – Robots: factory floors, home vacuums – Still quite impractical Powered by De. Sia. More 4

Who does AI? • Many disciplines contribute to goal of creating/modelling intelligent entities: – Computer Science – Psychology (human reasoning) – Philosophy (nature of belief, rationality, etc) – Linguistics (structure and meaning of language) – Human Biology (how brain works) • Subject draws on ideas from each discipline. Powered by De. Sia. More 5

Typical AI Problems • Intelligent entities (or “agents”) need to be able to do both “mundane” and “expert” tasks: • Mundane tasks - consider going shopping: – Planning a route, and sequence of shops to visit! – Recognising (through vision) buses, people. – Communicating (through natural language). – Navigating round obstacles on the street, and manipulating objects for purchase. • Expert tasks are things like: – medical diagnosis. – equipment repair. Powered by 6 • Often “mundane” tasks are. De. Sia. More hardest. the

AI vs. humans • AI applications built on Aristotlean logic – Induction, semantic queries, system of logic – Human reasoning involves more than just induction • Computers never as good as humans – In reasoning and making sense of data – In obtaining a holistic view of a system • Computers much better than humans – In processing reams of data – Performing complex calculations Powered by De. Sia. More 7

Successful AI applications • Targeted tasks more amenable to automated methods – Build special-purpose AI systems • Determine appropriate dosage for a drug • Classify cells as benign or cancerous – Called “expert systems” • Methodology based on expert reasoning • Quick and objective ways to obtain answers Powered by De. Sia. More 8

Data Driven Methods What is Artificial Intelligence? Common AI techniques Choosing between AI techniques Pre and post processing Powered by De. Sia. More 9

AI Technique • Intelligence requires Knowledge • Knowledge posesses less desirable properties such as: – Voluminous – Hard to characterize accurately – Constantly changing – Differs from data that can be used • AI technique is a method that exploits knowledge that should be represented in such a way that: – Knowledge captures generalization – It can be understood by people who must provide it – It can be easily modified to correct errors. – It can be used in variety of by De. Sia. More situations Powered 10

The State of the Art • Computer beats human in a chess game. • Computer-human conversation using speech recognition. • • • Expert system controls a spacecraft. Robot can walk on stairs and hold a cup of water. Language translation for webpages. Home appliances use fuzzy logic. . . . Powered by De. Sia. More 11

Fuzzy logic • Fuzzy logic addresses key problem in expert systems – How to represent domain knowledge – Humans use imprecisely calibrated terms – How to build decision trees on imprecise thresholds Powered by De. Sia. More 12

Advantages of fuzzy logic • Considerable skill for little investment – Fuzzy logic systems piggy bank on human analysis • Humans encode rules after intelligent analysis of lots of data • Verbal rules generated by humans are robust – Simple to create • Not much need for data or ground truth • Logic tends to be easy to program • Fuzzy rules are human understandable Powered by De. Sia. More 13

Where not to use fuzzy logic • Do not use fuzzy logic if: – – Humans do not understand the system Different experts disagree Knowledge can not be expressed with verbal rules Gut instinct is involved • Not just objective analysis • A fuzzy logic system is limited – Piece-wise linear approximation to a system – Non-linear systems can not be approximated • Many environment applications are non-linear Powered by De. Sia. More 14

Neural Networks • Neural networks can approximate non-linear systems – Evidence-based • Weights chosen through optimization procedure on known dataset (“training”) – Works even if experts can’t verbalize their reasoning, or if there is ground truth Powered by De. Sia. More 15

Advantages of neural networks • Can approximate any smooth function – The three-layer neural network • Can yield true probabilities – If output node is a sigmoid node • Not hard to train – Training process is well understood • Fast in operations – Training is slow, but once trained, the network can calculate the output for a set of inputs quite fast • Easy to implement – Just a sum of exponential functions Powered by De. Sia. More 16

Disadvantages of neural networks • A black box – The final set of weights yields no insights – Magnitude of weights doesn’t mean much • Measure of skill needs to be differentiable – RMS error, etc. – Can not use Probability of Detection, for example • Training set has to be complete – Unpredictable output on data unlike training – Need lots of data – Need expert willing to do lot of truthing Powered by De. Sia. More 17

Recap: • Fuzzy logic – Humans provide the rules – Not optimal • Neural network – Humans can not understand system – Optimal • Middle ground? – Genetic Algorithms – Decision Trees Powered by De. Sia. More 18

Genetic algorithms • In genetic algorithms – One fixes the model (rule base, equations, class of functions, etc. ) – Optimize the parameters to model on training data set – Use optimal set of parameters for unknown cases Powered by De. Sia. More 19

An example genetic algorithm Sources: http: //tx. technion. ac. il/~edassau/web/genetic_algorithms. htm http: //cswww. essex. ac. uk/research/NEC/ Powered by De. Sia. More 20

Advantages of genetic algorithms • Near-optimal parameters for given model – Human-understandable rules – Best parameters for them • Cost function need not be differentiable – The process of training uses natural selection, not gradient descent • Requires less data than a neural network – Search space is more limited Powered by De. Sia. More 21

Disadvantages of genetic algorithms • Highly dependent on class of functions – If poor model is chosen, poor results • Optimization may not help at all • Known model does not always lead to better understanding – Magnitude of weights, etc. may not be meaningful if inputs are correlated – Problem may have multiple parametric solutions Powered by De. Sia. More 22

Data Driven Methods What is Artificial Intelligence? Common AI techniques Choosing between AI techniques Pre and post processing Powered by De. Sia. More 23

Typical data-driven application Which features? Input Data How do we find f() Features AI application in run-time f(features) Result Powered by De. Sia. More 24

What is the role of the data? • Validation – Test known model – Technique: • Difference between model output and ground truth helps to validate the model • Calibration – Find parameters to model with desired structure – Technique: • Tuned fuzzy logic method • Genetic algorithms • Induction – Find model and parameters from just data – Technique: • Neural network methods, bagged/boosted decision trees, support vector machines, etc. Powered by De. Sia. More 25

What is the problem to solve? • Do you have a bunch of data and want to: – Estimate an unknown parameter from it? • True rainfall based on radar observations? • Amount of liquid content from in-situ measurements of temperature, pressure, etc? • Regression – Classify what the data correspond to? • • A water surge? A temperature inversion? A boundary? Classification • Regression and classification aren’t that different – Classification: estimate probability of an event • A function from 0 -1 Powered by De. Sia. More 26

Which AI technique? • Do you have expert knowledge? – Humans have a “model” in their head? Should the final f() be understandable? – Create fuzzy logic rules from experts’ reasoning • Aggregate the individual fuzzy logic rules • Can tune the fuzzy rules based on data – Using regression, decision trees or neural networks for RMS error criterion – Genetic algorithms for error criteria like ROC, economic cost, etc. • Many times the original rules are just fine • Do you already know the model? – A power-law relationship? Gaussian? Quadratic? Rules? – Just need to find parameters to this model? • If linear, just use linear regression • If non-linear: use genetic algorithms • Use continuous GAs • Both of these can be used for regression (therefore, also classification) Powered by De. Sia. More 27

Which AI technique (contd. ) • Do you know nothing about the data? – Not the suspected equation/model (GA)? – Not the suspected rules (fuzzy logic)? – Use a AI technique that supplies its equations/rules • “black box”. • For classification, use: – Bagged decision trees or Support Vector Machines • If output is probabilistic, remember to apply Platt scaling • Summary statistics on bagged DTs can help answer “why” – Neural Networks • For regression, use: – Neural networks Powered by De. Sia. More 28

Where do your data come from? • Observed data – Compute features – Choose AI technique • The 4 choices in the previous two slides • Simulated data: – – Example: trying to replicate a very complex model Throw randomly-generated data at model Compute features Choose AI technique: • GA for parametric approximations • NN when you don’t know how to approximate Powered by De. Sia. More 29

Where do you get your inputs? • What type of data do you have? – Individual observations? • Sample them (choose at random) and use directly – Sparse observations in a time series? • Generate time-based features (1 D moving windows) • Signal processing features from time series – Data from remotely sensed 2 D grids? • Generate image-based features using convolution filters • Do you need: – Pixel-based regression/classification? » Use convolution features directly – Object-based regression/classification? » Identify regions using region growing » Use region-aggregate features Powered by De. Sia. More 30

Typical data-driven application Observed data Signal/image processing; sampling Features normalize/create chromosome/ determine confidences f() Fz. Logic/Gen. Alg/NN/Dec. Tree Platt method/region-average/threshold A data-driven application in run-time Result Powered by De. Sia. More 31

Data Driven Methods What is Artificial Intelligence? Common AI techniques Choosing between AI techniques Pre and post processing Powered by De. Sia. More 32

Preprocessing • Often can not use pixel data directly – Too much data, too highly correlated – May need to segment pixels into objects and use features computed on the objects • Different data sets may not be collocated – Need to interpolate to line them up – Mapping, objective analysis • Noise in data may need to be reduced – Smoothing – Present statistic of data, rather than data itself • Features need to be extracted from data – Human experts often good source of ideas on signatures to extract from data Powered by De. Sia. More 33

Postprocessing • The output of an expert system may be grid point by grid point – May need to provide output on objects • Storms, forests, etc. – Can average outputs over objects’ pixels • May need probabilistic output – Scale output of maximum marginal techniques – Use a sigmoid function • Called Platt scaling Powered by De. Sia. More 34

Summary • What is Artificial Intelligence? – Data-driven methods to perform specific targeted tasks • Common AI techniques – Fuzzy logic, neural networks, genetic algorithms, decision trees • Choosing between AI techniques – Understand the role of your data – Do experts understand the system? (have a model) – Do experts expect to understand the system? (readability) • Pre and post processing – Image processing techniques on spatial grids Powered by De. Sia. More 35

AI Introduction END Powered by De. Sia. More 36