Basics of Probability - University of Arizona

1 Basics of ProbabilityAugust 27 and September 1, 20091 IntroductionA phenomena is calledrandomif the exact outcome is uncertain. The mathematical study of randomnessis called thetheory of Probability model has two essential pieces of its description. S, the sample space, the set of possible outcomes. Aneventis a collection {s1,s2, ,sn}and a subset of the sample spaceA S. P, the Probability assigns a number to each , a Probability is a function. We are familiar with functions in which both the domain and range aresubsets of the real numbers.

2 The domain of a Probability function is the collection of all possible range is still a number. We will see soon which numbers we will accept as possible probabilities operations ofunion,intersectionandcomplementallow us to define new events. Identities inset theory tell that certain operations result in the same event. For example, if we take eventsA,B, andC,then we have the B=B A, A B=B (A B) C=A (B C),(A B) C=A (B C). (B C) = (A B) (A C), A (B C) = (A B) (A C). s Laws(A B)c=Ac Bc,(A B)c=Ac third element in a Probability model is a a collection of subsets ofSsatisfying thefollowing conditions:1.

3 IfA F, thenAc If{Aj;j 1} F, then j=1Aj Set Theory - Probability Theory DictionaryEvent LanguageSet LanguageSet NotationVenn Diagramsample spaceuniversal setSeventsubsetA,B,C, outcomeelementsimpossible eventempty set notAAcomplementAcAorBAunionBA BAandBAintersectBA BdifferenceAbut notBA\B=A Bcsymmetric differenceeitherAorBA Bbut not both= (A\B) (B\A)AandBareAandBareA B= mutually exclusivedisjointifAthenBAis a subset ofBA BWheneverSis finite or countable, then we takeFto be all subsets ofS. WhenSis uncountable, then,in general, we cannot make this choice forFand maintain other more desirable properties.

4 The best knownexample is to takeS= [0,1] and let the Probability of an interval be equal to the length of the we cannot define a probabilityPon all of the subsets of [0,1] so thatP([a,b]) =b Examples of sample spaces and events1. Toss a coinS={H,T}#(S) = 2 Toss headsA={H}#(A) = 12. Toss a coin three {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}#(S) =Toss at least two heads in a {HHH, HHT, THH}#(A) =Toss at least two {HHH, HHT, HTH, THH}#(B) =3. Toss a coin 100 times.#(S) =A={67 heads}#(A) =4. Roll two dice.#(S) =A={sum is 7}#(A) =B={maximum value is 4}#(B) =C={sum is not 7}#(C) =D={sum is 7 or maximum value is 4}#(D) =5.

5 Roll three dice.#(S) =A={sum is 9}#(A) =B={sum is 10}#(B) =C={sum is 9 or 10}#(B) =6. Pick a card from a deck.#(S) =A={pick a }#(A) =B={level is 4}#(B) =C={level is not 4}#(C) =7. Pick two cards form the deck.(a) Replacing the first before choosing the second.#(S) =(b) Choosing the first, then the second without replacing.#(S) =(c) Choosing two cards simultaneously.#(S) =A={pick two aces}find #(A) in each of the three Equally Likely OutcomesIfSis a finite sample space, then if each outcome is equally likely, we define the Probability ofAas thefraction of outcomes that are (A) =#(A)#(S).

6 Thus, computingP(A) means counting the number of outcomes in the eventAand the number ofoutcomes in the sample space and Toss a {heads}=#(A)#(S)= Toss a coin three {toss at least two heads in a row}=#(A)#(S)=3. Roll two {sum is 7}=#(A)#(S)=Because we always have 0 #(A) #(S),, we always have0 P(A) 1(1)andP(S) = 1(2)So, no we know that the range of the function we call the Probability is a subset of the interval [0,1].Toss a coin 4 times.#(S) = 16A={exactly 3 heads}#(A) = 4={HHHT, HHTH, HTHH, THHH}P(A) =416=14B={exactly 4 heads}#(B) = 1={HHHH}P(B) =116 Now let s define the setC={at least three heads}.

7 If you are asked the supply the Probability of C,your intuition is likely to give you an immediate (C) =.4 Let s have a look at this intuition. The eventsAandBhave no outcomes in common, they are mutuallyexclusive events, and thus,#(A B) = #(A) + #(B).If we take thisaddition principleand divide by #(S), then we obtain the following identityIfA B= , thenP(A B) =P(A) +P(B).(3)Using this property, we see thatP{at least 3 heads}=P{exactly 3 heads}+P{exactly 4 heads}=416+116= The Axioms of Probability1. For any eventA,0 P(A) 1.

8 (1)2. For the sample spaceS,P(S) = 1.(2)3. If the eventsAandBare mutually exclusive (A B= ), thenP(A B) =P(A) +P(B).(3)We are saying that any functionPthat accepts events as its domain and returns numbers as its rangeand satisfies (1), (2), and (3) can be called a example, if we toss abiasedcoin. We may want to say thatP{heads}=pwherepis not necessarily equal to 1/2. By necessity,P{tails}= 1 we iterate the procedure in Axiom 3, we can also state that if the events,A1,A2, ,An, are mutuallyexclusive, thenP(A1 A2 An) =P(A1) +P(A2) + +P(An).

9 (3 )For the random experiment, flip a coin repeated until heads appears, we can writeAj={the first headappears on thej-th toss}. We would like to say thatP{heads appears eventually}=P(A1) +P(A2) + +P(An) + .This would call for an extension of Axiom 3 to an infinite number of mutually exclusive events. This is thegeneral version of Axiom 3 we use when we want to use calculus in the theory of Probability :For{Aj;j 1}, are mutually exclusive, thenP j=1Aj = j=1P(Aj)(3 )56 Consequences of the Complement Rule. BecauseAandAcare mutually exclusiveP(A) +P(Ac) =P(A Ac) =P( ) = 1orP(Ac) = 1 P(A).

10 Toss a coin 4 {fewer than 3 heads}= 1 P{at least 3 heads}= 1 516= can extend this. IfA B, then theP(B\A) =P(B) P(A). Inclusion-Exclusion Rule. For any two eventsAandB,P(A B) =P(A) +P(B) P(A B)(P(A) +P(B) counts the outcomes inA Btwice, so removeP(A B).)Exercise that the inclusion-exclusion rule follows from the axioms. Hint:A B= (A Bc) BandA= (A B) (A Bc).Deal two {ace on the second card},B={ace on the first card}P(A B) =P(A) +P(B) P(A B)P{at least one ace}=113+113 ?To complete this computation, we will need to computeP(A B) =P{both cards are aces}.