Introduction of Bayesian Network 4 2005 CSE

Introduction of Bayesian Network 4 / 2005 CSE 634 Data Mining Prof. Anita Wasilewska 105269827 Hiroo Kusaba Software Engineering Laboratory 1

References n [1] D. Heckerman: “A Tutorial on Learning with Bayesian Networks”, In “Learning in Graphical Models”, ed. M. I. Jordan, The MIT Press, 1998. n [2] http: //www. cs. huji. ac. il/~nir/Nips 01 -Tutorial/ n [3]Jiawei Han: ”Data Mining Concepts and Techniques”, ISBN 1 -53860 -489 -8 n [4] Whittaker, J. : Graphical Models in Applied Multivariate Statistics, John Wiley and Sons (1990) Software Engineering Laboratory 2

Contents n Brief introduction n Review u u A little review of probability Bayes theorem n Bayesian Classification n Steps of using Bayesian Network Software Engineering Laboratory 3

n Random variables X, Y, Xi, Θ Capitals n Condition (or value) of a variable x, y, xi, θ small n Set of a variable X, Y, Xi, Θ in Capital bold n Set of a condition (or value) x, y, xi, θ small bold n P(x/a) : Probability that an event x occurs (or happens) under the condition of a Software Engineering Laboratory 4

What is Bayesian Network ? n Network which express the dependencies among the random variables n Each node has posterior probability which depends on the previous random variable n The whole network also express the joint probability distribution from all of the random variables n Pa is parent(s) of a node i Software Engineering Laboratory 5

How is it used ? n Bayesian Learning u Estimating dependencies between the random variables from the actual data n Bayesian Inference u When some of the random variables are defined it calculate the other probabilities w Patiants condition as a random variable, from the condition it predicts the desease Software Engineering Laboratory 6

What is so good about it? n Conditional independencies and graphical expression capture structure of many real-world distributions. [1] n Learned model can be used for many tasks n Supports all the features of probabilistic learning u u Model selection criteria Dealing with missing data and hidden variables Software Engineering Laboratory 7

Example of Bayesian Network u u X 0 1 Structure of a network Conditional Probability w X, Y, Z are random variables which takes either 0 or 1 w p(X), p(Y|X), p(Z|Y) P(X) 0. 5 X 0 0 1 1 Y 0 1 P(Y|X) 0. 1 0. 9 0. 2 0. 8 X Y 0 0 1 1 Software Engineering Laboratory Y Z 0 1 Z P(Z|Y) 0. 3 0. 7 0. 4 0. 6 8

Example of Bayesian Network 2 n What is the Joint probability of P(X, Y, Z)? u X 0 0 P(X, Y, Z) = P(X)*P(Y|X)*P(Z|Y) Y 0 0 1 1 Z 0 1 P(X, Y, Z) 0. 015 0. 035 0. 180 0. 270 X 1 1 Software Engineering Laboratory Y 0 0 1 1 Z 0 1 P(X, Y, Z) 0. 030 0. 070 0. 160 0. 240 9

A little Review of probability 1 n Probability : How likely is it that an event will happen? n Sample Space S u u Element of S: elementary event An event A is a subset of S n P(A) ≧ 0 n P(S) = 1 Software Engineering Laboratory 10

A little review of probability 2 n Discrete probability distribution u P(A) = Σｓ∈A　P（ｓ） n Conditional probability distribution u P(A|B) = P(A, B) / P(B) n If the events are independent u P(A, B) = P(A)*P(B) A B n Bayes Theorem Software Engineering Laboratory 11

Bayes Theorem Software Engineering Laboratory 12

Example of Bayes Theorem n You are about to be tested for a rare desease. How worried should you be if the test result is positive ? n Accuracy of the Test is P(T) = 85% n Chance of Infection P(I) = 0. 01% n What is P(I / not T) n http: //www. gametheory. net/Mike/applets/Bayes/B ayes. html Software Engineering Laboratory 13

Bayesian Classification n Suppose that there are m classes, Given an unknown data sample, x the Bayesian classifier assigns an unknown sample x to the class c if and only if Software Engineering Laboratory 14

n We have to maximize n In order to reduce computation class conditional independence is made Software Engineering Laboratory 15

Example of Bayesian Classification in the text book[3] n Customer under 30 and income is “medium” and student and credit rating is “fair”, which category does the customer belongs? Buy or not. Software Engineering Laboratory 16

Bayesian Network which express the dependencies among the random variables n The whole network also express the joint probability distribution from all of the random variables n Pa is parent(s) of a node i X Y Z Pai are a subset Software Engineering Laboratory 17

Steps to apply Bayesian Network n Step 1 Create a Bayesian Belief Network u u u Include all the variables that are important in your system Use causal knowledge to guide the connections made in the graph Use your prior knowledge to specify the conditional distributions n Step 2 Calculate the p(xi|pai) for your goal Software Engineering Laboratory 18

Example from [1] n Example to make a BN from the prior knowledge n BN to find a credit card fraud u Define random variables w w w Fraud(F): Probability that owner is a fraud Gas(G): Bought a gas in 24 hours Jewelry(J): Bought a jewelry in 24 hours Age(A): Age of owner of the card Sex(S): Gender of the owner of the card Software Engineering Laboratory 19

n Give orders to random variables n Define dependencies, but you have to be careful. F A G J F J G S A S Software Engineering Laboratory 20

Next topic n Training with Bayesian Network u u Bayes Inference If the training data is complete If the training data is missing Network Evaluation Software Engineering Laboratory 21

Thank you for listening. Software Engineering Laboratory 22