PROBABILITY AND STATISTICS FOR ENGINEERING Hossein Sameti Sharif University of Technology Spring 2008
Text Book A. Papoulis and S. Pillai, Probability, Random Variables and Stochastic Processes, 4 th Edition, Mc. Graw Hill, 2000 3/16/2018 Sharif University of Technology 2
TABLE OF CONTENTS PROBABILITY THEORY Lecture – 1 Lecture – 2 Lecture – 3 Lecture – 4 Lecture – 5 Lecture – 6 Lecture – 7 Lecture – 8 Lecture – 9 Lecture – 10 Lecture – 11 Lecture – 12 Lecture – 13 Basics Independence and Bernoulli Trials Random Variables Binomial Random Variable Applications, Conditional Probability Density Function and Stirling’s Formula. Function of a Random Variable Mean, Variance, Moments and Characteristic Functions Two Random Variables One Function of Two Random Variables Two Functions of Two Random Variables Joint Moments and Joint Characteristic Functions Conditional Density Functions and Conditional Expected Values Principles of Parameter Estimation The Weak Law and the Strong Law of Large numbers STOCHASTIC PROCESSES Lecture – 14 Lecture – 15 Lecture – 16 Lecture – 17 Lecture – 18 Lecture – 19 Lecture – 20 Stochastic Processes - Introduction Poisson Processes Mean square Estimation Long Term Trends and Hurst Phenomena Power Spectrum Series Representation of Stochastic processes Extinction Probability for Queues and Martingales Source AT: www. mhhe. com/papoulis 3
Teaching Assistants: Hoda Akbari Course Evaluation: Homeworks: 10 -20% Midterm: 25 -30% Final Exam: 50 -60% 3/16/2018 Sharif University of Technology 4
Probability And Statistics Source: http: //ocw. mit. edu 3/16/2018 Sharif University of Technology 5
PROBABILITY THEORY 1. Basics
Random Phenomena, Experiments § Study of random phenomena § Different outcomes § Outcomes that have certain underlying patterns about them § Experiment - repeatable conditions § Certain elementary events Ei occur in different but completely uncertain ways. § probability of the event Ei : P(Ei )>=0 3/16/2018 Sharif University of Technology 7
Probability Definitions § Laplace’s Classical Definition - without actual experimentation - provided all these outcomes are equally likely. Example • a box with n white and m red balls elementary outcomes: {white , red} Probability of “selecting a white ball”: • 3/16/2018 Sharif University of Technology 8
Probability Definitions § Relative Frequency Definition - The probability of an event A is defined as - n. A is the number of occurrences of A - n is the total number of trials 3/16/2018 Sharif University of Technology 9
Probability Definitions Example 1. The probability that a given number is divisible by a prime p: 3/16/2018 Sharif University of Technology 10
Counting - Remark § General Product Rule if an operation consists of k steps each of which can be performed in ni ways (i = 1, 2, …, k), then the entire operation can be performed in ni ways. Example - Number of PINs - Number of elements in a Cartesian product - Number of PINs without repetition - Number of Input/Output tables for a circuit with n input signals - Number of iterations in nested loops 3/16/2018 Sharif University of Technology 11
Permutations and Combinations - Remark § If order matters choose k from n: - Permutations : § If order doesn't matters choose k from n: - Combinations : Example A fair coin is tossed 7 times. What is the probability of obtaining 3 heads? What is the probability of obtaining at most 3 heads? 3/16/2018 Sharif University of Technology 12
Example: The Birthday Problem § Suppose you have a class of 23 students. Would you think it likely or unlikely that at least two students will have the same birthday? § It turns out that the probability of at least two of 23 people having the same birthday is about 0. 5 (50%). 3/16/2018 Sharif University of Technology 13
Axioms of Probability- Basics § The axiomatic approach to probability, due to Kolmogorov, developed through a set of axioms § The totality of all events known a priori, constitutes a set Ω, the set of all experimental outcomes. 3/16/2018 Sharif University of Technology 14
Axioms of Probability- Basics § A and B are subsets of Ω. A B 3/16/2018 A B Sharif University of Technology A 15
Mutually Exclusiveness and Partitions § A and B are said to be mutually exclusive if § A partition of is a collection of mutually exclusive(ME) subsets of such that their union is . A 3/16/2018 B Sharif University of Technology 16
De-Morgan’s Laws A B 3/16/2018 A B A Sharif University of Technology B A B 17
Events § Often it is meaningful to talk about at least some of the subsets of as events § we must have mechanism to compute their probabilities. Example Tossing two coins simultaneously: A: The event of “Head has occurred at least once”. 3/16/2018 Sharif University of Technology 18
Events and Set Operators § “Does an outcome belong to A or B” § “Does an outcome belong to A and B” § “Does an outcome fall outside A”? § These sets also qualify as events. § We shall formalize this using the notion of a Field. 3/16/2018 Sharif University of Technology 19
Fields § A collection of subsets of a nonempty set forms a field F if § Using (i) - (iii), it is easy to show that the following also belong to F. 3/16/2018 Sharif University of Technology 20
Fields If then § We shall reserve the term event only to members of F. § Assuming that the probability P(Ei ) of elementary outcomes Ei of Ω are apriori defined. § The three axioms of probability defined below can be used to assign probabilities to more ‘complicated’ events. 3/16/2018 Sharif University of Technology 21
Axioms of Probability § For any event A, we assign a number P(A), called the probability of the event A. § Conclusions: 3/16/2018 Sharif University of Technology 22
Probability of Union of to Non-ME Sets A 3/16/2018 Sharif University of Technology 23
Union of Events § Is Union of denumerably infinite collection of pairwise disjoint events Ai an event? § If so, what is P(A ) ? § We cannot use third probability axiom to compute P(A), since it only deals with two (or a finite number) of M. E. events. 3/16/2018 Sharif University of Technology 24
An Example for Intuitive Understanding § in an experiment, where the same coin is tossed indefinitely define: A = “head eventually appears”. § Our intuitive experience surely tells us that A is an event. If We have: § Extension of previous notions must be done based on our intuition as new axioms. 3/16/2018 Sharif University of Technology 25
σ-Field (Definition): § A field F is a σ-field if in addition to the three mentioned conditions, we have the following: - For every sequence of pairwise disjoint events belonging to F, their union also belongs to F 3/16/2018 Sharif University of Technology 26
Extending the Axioms of Probability § If Ai s are pairwise mutually exclusive § from experience we know that if we keep tossing a coin, eventually, a head must show up: § But: § Using the fourth probability axiom we have: 3/16/2018 Sharif University of Technology 27
Reasonablity § In previously mentioned coin tossing experiment: § So the fourth axiom seems reasonable. 3/16/2018 Sharif University of Technology 28
Summary: Probability Models § The triplet ( , F, P) - is a nonempty set of elementary events -F is a -field of subsets of . - P is a probability measure on the sets in F subject the four axioms § The probability of more complicated events must follow this framework by deduction. 3/16/2018 Sharif University of Technology 29
Conditional Probability § In N independent trials, suppose NA, NB, NAB denote the number of times events A, B and AB occur respectively. § According to the frequency interpretation of probability, for large N, § Among the NA occurrences of A, only NAB of them are also found among the NB occurrences of B. § Thus the following is a measure of “the event A given that B has already occurred”: 3/16/2018 Sharif University of Technology 30
Satisfying Probability Axioms § We represent this measure by P(A|B) and define: § As we will show, the above definition is a valid one as it satisfies all probability axioms discussed earlier. 3/16/2018 Sharif University of Technology 31
Satisfying Probability Axioms 3/16/2018 Sharif University of Technology 32
Properties of Conditional Probability Example In a dice tossing experiment, - A : outcome is even - B: outcome is 2. The statement that B has occurred makes the odds for A greater 3/16/2018 Sharif University of Technology 33
Law of Total Probability § We can use the conditional probability to express the probability of a complicated event in terms of “simpler” related events. § Suppose that § So, 3/16/2018 Sharif University of Technology 34
Conditional Probability and Independence § A and B are said to be independent events, if § This definition is a probabilistic statement, not a set theoretic notion such as mutually exclusiveness. § If A and B are independent, § Thus knowing that the event B has occurred does not shed any more light into the event A. 3/16/2018 Sharif University of Technology 35
Independence - Example From a box containing 6 white and 4 black balls, we remove two balls at random without replacement. What is the probability that the first one is white and the second one is black? 3/16/2018 Sharif University of Technology 36
Example - continued § Are W 1 and B 2 independent? Removing the first ball has two possible outcomes: These outcomes form a partition because: So, Thus the two events are not independent. 3/16/2018 Sharif University of Technology 37
General Definition of Independence § Independence between 2 or more events: § Events A 1, A 2, . . . , An are mutually independent if, for all possible subcollections of k ≤ n events: Example In experiment of rolling a die, A = {2, 4, 6} B = {1, 2, 3, 4} C = {1, 2, 4}. Are events A and B independent? What about A and C? 3/16/2018 Sharif University of Technology Source: http: //ocw. mit. edu 38
Bayes’ Theorem We have: Thus, Also, 3/16/2018 Sharif University of Technology 39
Bayesian Updating: Application Of Bayes’ Theorem § Suppose that A and B are dependent events and A has apriori probability of P(A ). § How does Knowing that B has occurred affect the probability of A? § The new probability can be computed based on Bayes’ Theorm. § Bayes’ Theorm shows how to incorporate the knowledege about B’s occuring to calculate the new probability of A. 3/16/2018 Sharif University of Technology 40
Bayesian Updating - Example § Suppose there is a new music device in the market that plays a new digital format called MP∞. Since it’s new, it’s not 100% reliable. § You know that - 20% of the new devices don’t work at all, - 30% last only for 1 year, - and the rest last for 5 years. § If you buy one and it works fine, what is the probability that it will last for 5 years? Source: http: //ocw. mit. edu 3/16/2018 Sharif University of Technology 41
Generalization of Bayes’ Theorem A more general version of Bayes’ theorem involves partition of Ω : In which, Represents a collection of mutually exclusive events with assiciated apriori probabilities: With the new information “B has occurred”, the information about Ai can be updated by the n conditional probabilities: 3/16/2018 Sharif University of Technology 42
Bayes’ Theorem - Example 1. Two boxes, B 1 and B 2 contain 100 and 200 light bulbs respectively. The first box has 15 defective bulbs and the second 5. Suppose a box is selected at random and one bulb is picked out. a) What is the probability that it is defective? 3/16/2018 Sharif University of Technology 43
Example - Continued Suppose we test the bulb and it is found to be defective. What is the probability that it came from box 1? § Note that initially, § But because of greater ratio of defective bulbs in B 1 , this probability is increased after the bulb determined to be defective. . 3/16/2018 Sharif University of Technology 44
Скачать презентацию PROBABILITY AND STATISTICS FOR ENGINEERING Hossein Sameti Sharif
f407f58e03d4813de4ed89f1a134ebb8.ppt