Lecture Slides for INTRODUCTION TO Machine Learning 2

Lecture Slides for INTRODUCTION TO Machine Learning 2 nd Edition ETHEM ALPAYDIN © The MIT Press, 2010 alpaydin@boun. edu. tr http: //www. cmpe. boun. edu. tr/~ethem/i 2 ml 2 e

CHAPTER 1: Introduction

Why “Learn” ? Machine learning is programming computers to optimize a performance criterion using example data or past experience. There is no need to “learn” to calculate payroll Learning is used when: Human expertise does not exist (navigating on Mars), Humans are unable to explain their expertise (speech recognition) Solution changes in time (routing on a computer network) Solution needs to be adapted to particular cases (user biometrics(生物 統計學)) Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 3

What We Talk About When We Talk About “Learning” Learning general models from a data of particular examples Data is cheap and abundant; knowledge is expensive and scarce (不足). Example in retail (零售): Customer transactions(交易) to consumer behavior: People who bought “X” also bought “Y” Build a model that is a good and useful approximation to the data. Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 4

What is Machine Learning? Optimize a performance criterion using example data or past experience. Role of Statistics: Inference from a sample Role of Computer science: Efficient algorithms to Solve the optimization problem Representing and evaluating the model for inference Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 5

Applications Association(聯想) Auto-association Hetero-association Supervised Learning(監督式學習) Classification (Recognition) Regression Unsupervised Learning(非監督式學習) Clustering (Grouping) Reinforcement Learning … Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 6

Learning Associations Basket analysis To find associations between products bought by customers Learning a conditional probability P (Y | X ) probability that somebody who buys X also buys Y where X and Y are products/services. Example P ( chips | beer ) = 0. 7 70 percent of customers who buy beer also buy chips. Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 7

Classification Example: Credit scoring Differentiating between lowrisk and high-risk customers from their income and savings Discriminant: IF income > θ 1 AND savings > θ 2 THEN low-risk ELSE high-risk Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 8

Classification: Applications Pattern recognition Character recognition: Different handwriting styles. Face recognition: Pose, lighting, occlusion, make-up, hair style Speech recognition: Temporal dependency. Use of a dictionary or the syntax of the language. Sensor fusion: Combine multiple modalities; eg, visual (lip image) and acoustic(聽覺的) for speech Medical diagnosis: From symptoms(症狀) to illnesses . . . Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 9

Face Recognition Training examples of a person Test images AT&T Laboratories, Cambridge UK http: //www. uk. research. att. com/facedatabase. html Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 10

Regression Example: Price of a used car y = g (x | θ ) y : price x : car attributes e. g. milage (英里數) g ( ): model θ: parameters e. g. age(年份) Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) y = wx+w 0 11

Regression: Applications Navigating a car Angle of the steering wheel (CMU Nav. Lab) Kinematics (運動學) of a robot arm (x, y) α 2 α 1= g 1(x, y) α 2= g 2(x, y) α 1 Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 12

Supervised Learning: Uses Prediction of future cases: Use the rule to predict the output for future inputs Knowledge extraction: The rule is easy to understand Compression: The rule is simpler than the data it explains Outlier detection: Exceptions that are not covered by the rule Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 13

Unsupervised Learning “what normally happens” The aim is to find the regularities in the input. Density estimation: we want to see what generally happens and what does not. Clustering: Grouping similar instances Example applications Customer segmentation in CRM (customer relationship management) Image compression: Color quantization Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 14

Reinforcement Learning a policy: A sequence of outputs The output of the system is a sequence of actions. An action is good if it is part of a good policy. No supervised output but delayed reward Examples: Game playing Robot in a maze Partial observability. . . Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 15

Exercise What is learning? What is supervised learning? Unsupervised learning? Reinforcement learning? Explain them and give an example of each type of learning. Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 16

Resources: Journals Journal of Machine Learning Research (www. jmlr. org) Machine Learning Neural Computation Neural Networks IEEE Transactions on Pattern Analysis and Machine Intelligence Annals of Statistics Journal of the American Statistical Association. . . Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 17

Resources: Conferences International Conference on Machine Learning (ICML) European Conference on Machine Learning (ECML) Neural Information Processing Systems (NIPS) Uncertainty in Artificial Intelligence (UAI) Computational Learning Theory (COLT) International Joint Conference on Artificial Intelligence (IJCAI) International Conference on Neural Networks (Europe). . . Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V 1. 1) 18