3. Probability Theory - stat.wisc.edu

1 Ismor Fischer, 5/29/2012 .. 3. Probability Theory Basic Ideas, Definitions, and Properties POPULATION = Unlimited supply of five types of fruit, in equal proportions. O1 = Macintosh apple O2 = Golden Delicious apple O3 = Granny Smith apple O4 = Cavendish (supermarket) banana O5 = Plantain banana Experiment 1: Randomly select one fruit from this population, and record its type. Sample Space: The set S of all possible elementary outcomes of an experiment. S = {O1, O2, O3, O4, O5} #(S) = 5 Event: Any subset of a sample space S. ( Elementary outcomes = simple events.) A = Select an apple. = {O1, O2, O3} #(A) = 3 B = Select a banana. = {O4, O5} #(B) = 2 Event P(Event) A 3/5 = B 2/5 = 5/5 = P(A) = The Probability of randomly selecting an apple is As # trials P(B) = The Probability of randomly selecting a banana is 1/1 1/2 2/3 3/4 3/5 4/6 1/3 1/4 2/5 2/6.

2 A B 0 1 , .. A B B A A A 1 2 5 3 4 .. 6 # trials of experiment #(Event)#(trials) Ismor Fischer, 5/29/2012 General formulation may be facilitated with the use of a Venn diagram: Event A = {O1, O2, .., Om} S #(A) = m k Definition: The Probability of event A, denoted P(A), is the long-run relative frequency with which A is expected to occur, as the experiment is repeated indefinitely. Fundamental Properties of Probability For any event A = {O1, O2, .., Om} in a sample space S, 1. 0 P(A) 1 2. P(A) = 1231() ()()()()mimiPOPOPOPOPO== + + ++ Special Cases: P( ) = 0 P(S) = 1()ikiPO= = 1 certainty 3. If all the elementary outcomes of S are equally likely, , P(O1) = P(O2) = .. = P(Ok) = 1k, #( )()#( )AmPAk==S. Example: P(A) = 3/5 = , P(B) = 2/5 = Sample Space: S = {O1, O2, .., Ok} #(S) = k O1 O2 O4 Om.

3 Ok O3 .. A Om+1 Om+2 Om+3 Experiment Ismor Fischer, 5/29/2012 New Events from Old Events Experiment 2: Select a card at random from a standard deck (and replace). Sample Space: S = {A , .., K } #(S) = 52 Events: A = Select a 2. = {2 , 2 , 2 , 2 } #(A) = 4 B = Select a . = {A , 2 , .., K } #(B) = 13 Probabilities: Since all elementary outcomes are equally likely, it follows that P(A) = #( )#( )AS = 452 and P(B) = #( )#( )BS = 1352 . (1) Ac = not A = {All outcomes that are in S, but not in A.} Example: Ac = Select either A, 3, 4, .., or K. P(Ac) = 1 452 = 4852 . Example: Experiment = Toss a coin once. Events: A = {Heads} Ac = {Tails} Probabilities: Fair P(A) = P(Ac) = 1 = Biased P(A) = P(Ac) = 1 = P(Ac) = 1 P(A) complement A 2 3 4 5 6 7 8 9 10 J Q K A 2 3 4 5 6 7 8 9 10 J Q K A 2 3 4 5 6 7 8 9 10 J Q K A 2 3 4 5 6 7 8 9 10 J Q K A B Ismor Fischer, 5/29/2012 (2) A B = A and B = {All outcomes in S that A and B share in common.}

4 } = {All outcomes that result when events A and B occur simultaneously.} Example: A B = Select a 2 and a = {2 } P(A B) = 152 . Definition: Two events A and B are said to be disjoint, or mutually exclusive, if they cannot occur simultaneously, , A B = , hence P(A B) = 0. Example: A = Select a 2 and C = Select a 3 are disjoint events. Exercise: Are 44 44{2 , 3 , 4 , 5 ,..}A= and 66 66{2 , 3 , 4 , 5 ,..}B= disjoint? If not, find A B. (3) A B = A or B = {All outcomes in S that are either in A or B, inclusive.} Example: A B = Select either a 2 or a has Probability P(A B) = 452 + 1352 152 = 1652. Example: A C = Select either a 2 or a 3 has Probability P(A C) = 452 + 452 0 = 852. S A B P(A B) = P(A) + P(B) P(A B) = 0, if A and B are disjoint. intersection union Ismor Fischer, 5/29/2012 S A C B Note: Formula (3) extends to n 3 disjoint events in a straightforward manner: (4) P(A1 A2.)

5 An) = P(A1) + P(A2) + .. + P(An). Question: How is this formula modified if the n events are not necessarily disjoint? Example: Take n = 3 Then P(A B C) = P(A) + P(B) + P(C) P(A B) P(A C) P(B C) + P(A B C). Exercise: For S = {January,.., December}, verify this formula for the three events A = Has 31 days, B = Name ends in r, and C = Name begins with a vowel. Exercise: A single tooth is to be randomly selected for a certain dental procedure. Draw a Venn diagram to illustrate the relationships between the three following events: A = upper jaw, B = left side, and C = molar, and indicate all corresponding probabilities. Calculate the Probability that all of these three events, A and B and C, occur. Calculate the Probability that none of these three events occur.

6 Calculate the Probability that exactly one of these three events occurs. Calculate the Probability that exactly two of these three events occur. (Think carefully.) Assume equal likelihood in all cases. The three set operations union, intersection, and complement can be unified Exercise: Using a Venn diagram, convince yourself that these statements are true in general. Then verify them for a specific example, , A = Pick a picture card and B = Pick a black card. incisors incisors premolars premolars canine canine canine canine molars DeMorgan s Laws (A B) c = Ac Bc (A B) c = Ac Bc Ismor Fischer, 5/29/2012 Slight Suppose that out of the last n = 40 races, a certain racing horse won x = 25, and lost the remaining n x = 15. Based on these statistics, we can calculate the following Probability estimates for future races: P(Win) 255408xn= = = = p P(Lose) 1153408xn = = = = 1 p = q Odds of winning = (Win)5 / 85(Lose)3 / 83PP== 5 to 3 Definition: For any event A, let P(A) = p, th us P(Ac) = q = 1 p.

7 The odds of event A = pq = 1pp , , the Probability that A does occur, divided by the Probability that it does not occur. (In the preceding example, A = Win with Probability p = 5/8.) Note that if odds = 1, then A and Ac are equally likely to occur. If odds > 1 (likewise, < 1), then the Probability that A occurs is greater (likewise, less) than the Probability that it does not occur. Example: Suppose the Probability of contracting a certain disease in a particular group of high risk individuals is P(D+) = , so that the Probability of being disease-free is P(D ) = Then the odds of contracting the disease in this group is equal to = 3 (or 3 to 1 ).*31/ 49 Likewise, if in a reference group of low risk individuals, the prevalence of the same disease is only P(D+) = , so that P(D ) = , then their odds = = 1/49 ( ). As its name suggests, the corresponding odds ratio between the two groups is defined as the ratio of their respective odds, , = 147.

8 That is, the odds of the high-risk group contracting the disease are 147 times larger than the odds of the low-risk reference group. (Odds ratios have nice properties, and are used extensively in epidemiological studies.) * That is, within this group, the Probability of disease is three times larger than the Probability of no disease. Out of every 8 races, the horse wins 5 and loses 3, on average.