M2S1 Lecture Notes

1 m2s1 Lecture NotesG. A. ayoungSeptember 2011iiContents1 DEFINITIONS, TERMINOLOGY, EVENTS AND THE SAMPLE SPACE .. IN SET THEORY .. EXCLUSIVE EVENTS AND PARTITIONS .. THE -FIELD .. THE PROBABILITY FUNCTION .. PROPERTIES OF P(.): THE AXIOMS OF PROBABILITY .. CONDITIONAL PROBABILITY .. THE THEOREM OF TOTAL PROBABILITY .. BAYES THEOREM .. COUNTING TECHNIQUES .. MULTIPLICATION PRINCIPLE .. FROM A FINITE POPULATION .. AND COMBINATIONS .. CALCULATIONS .. 102 RANDOM VARIABLES&PROBABILITY RANDOM VARIABLES & PROBABILITY MODELS.

2 DISCRETE RANDOM VARIABLES .. OF MASS FUNCTIONfX.. BETWEENFXANDfX.. OF DISCRETE CDFFX.. CONTINUOUS RANDOM VARIABLES .. OF CONTINUOUSFXANDfX.. EXPECTATIONS AND THEIR PROPERTIES .. INDICATOR VARIABLES .. TRANSFORMATIONS OF RANDOM VARIABLES .. TRANSFORMATIONS .. TRANSFORMATIONS .. GENERATING FUNCTIONS .. GENERATING FUNCTIONS .. PROPERTIES OF MGFS .. GENERATING FUNCTIONS .. JOINT PROBABILITY DISTRIBUTIONS .. CHAIN RULE FOR RANDOM VARIABLES .. EXPECTATION AND ITERATED EXPECTATION.

3 MULTIVARIATE TRANSFORMATIONS .. MULTIVARIATE EXPECTATIONS AND COVARIANCE .. EXPECTATION WITH RESPECT TO JOINT DISTRIBUTIONS .. COVARIANCE AND CORRELATION .. JOINT MOMENT GENERATING FUNCTION .. FURTHER RESULTS ON INDEPENDENCE .. ORDER STATISTICS .. 40iiiivCONTENTS3 DISCRETE PROBABILITY DISTRIBUTIONS414 CONTINUOUS PROBABILITY DISTRIBUTIONS455 MULTIVARIATE PROBABILITY THE MULTINOMIAL DISTRIBUTION .. THE DIRICHLET DISTRIBUTION .. THE MULTIVARIATE NORMAL DISTRIBUTION .. 526 PROBABILITY RESULTS&LIMIT BOUNDS ON PROBABILITIES BASED ON MOMENTS.

4 THE CENTRAL LIMIT THEOREM .. MODES OF STOCHASTIC CONVERGENCE .. IN DISTRIBUTION .. IN PROBABILITY .. IN QUADRATIC MEAN .. 597 STATISTICAL STATISTICAL SUMMARIES .. SAMPLING DISTRIBUTIONS .. HYPOTHESIS TESTING .. FOR NORMAL SAMPLES - THE Z-TEST .. TESTING TERMINOLOGY .. t-TEST.. FOR .. SAMPLE TESTS .. POINT ESTIMATION .. TECHNIQUES I: METHOD OF MOMENTS .. TECHNIQUES II: MAXIMUM LIKELIHOOD .. INTERVAL ESTIMATION .. QUANTITY .. A TEST STATISTIC.

5 71 CHAPTER 1 DEFINITIONS, TERMINOLOGY, EVENTS AND THE SAMPLE SPACED efinition a one-off or repeatable process or procedure for which(a) there is a well-defined set ofpossibleoutcomes(b) theactualoutcome is not known with , , is precisely one of the possible outcomes of , , of an experiment is the set of all possible : is a set in the mathematical sense, so set theory notation can be used. For example, ifthe sample outcomes are denoted 1,.., k, say, then ={ 1,.., k}={ i:i= 1,..,k},and i2 fori= 1,.., sample space of an experiment can be- a FINITE list of sample outcomes,{ 1.}

6 , k}- a (countably) INFINITE list of sample outcomes,{ 1, 2,..}- an INTERVAL or REGION of a real space,{ : 2A Rd}Definition ,E, is a designated collection of sample outcomes. EventEoccursif the actual outcome of the experiment is one of this collection. An event is, therefore, a subset ofthe sample space .Special Cases of EventsThe event corresponding to the collection ofallsample outcomes is .The event corresponding to a collection ofnoneof the sample outcomes is denoted;. The sets;and are also events, termed theimpossibleand thecertainevent respectively,and for any eventE,E.

7 12 CHAPTER 1. DEFINITIONS, TERMINOLOGY, OPERATIONS IN SET THEORYS ince events are subsets of , set theory operations are used to manipulate events in probabilitytheory. Consider eventsE,F . Then we can reasonably concern ourselves also with eventsobtained from the three basic set operations:UNIONE F EorFor both occur INTERSECTIONE F bothEandFoccur COMPLEMENTE0 Edoes not occur Properties of Union/Intersection operatorsConsider eventsE,F,G .COMMUTATIVITYE F=F EE F=F EASSOCIATIVITYE (F G) = (E F) GE (F G) = (E F) GDISTRIBUTIVITYE (F G) = (E F) (E G)E (F G) = (E F) (E G)DE MORGAN S LAWS(E F)0=E0 F0(E F)0=E0 MUTUALLY EXCLUSIVE EVENTS AND PARTITIONSD efinition exclusiveifE F=;, that is, if eventsEandFcannot both occur.

8 If the sets of sample outcomes represented byEandFaredisjoint(haveno common element), thenEandFare mutually ,..,Ek form apartitionof eventF if(a)Ei Ej=;fori6=j,i,j= 1,..,k(b)k i=1Ei=F,so that each element of the collection of sample outcomes corresponding to eventFis inone andonly oneof the collections corresponding to eventsE1,.. THE -FIELD3 Figure : Partition of In Figure , we have =6 i=1 EiFigure : Partition ofF In Figure , we haveF=6 i=1(F Ei), but, for example,F E6=;. THE -FIELDE vents are subsets of , but need all subsets of be events?

9 The answer is negative. But itsuffices to think of the collection of events as a subcollectionAof the set of all subsets of . Thissubcollection should have the following properties:(a) ifA,B2 AthenA B2 AandA B2A;(b) ifA2 AthenA02A;4 CHAPTER 1. DEFINITIONS, TERMINOLOGY, NOTATION(c); collectionAof subsets of which satisfies these three conditions is called afield. It followsfrom the properties of a field that ifA1,A2,..,Ak2A, thenk i= ,Ais closed under finite unions and hence under finite intersections also. To see this note thatifA1,A22A, thenA01,A022A=)A01 A022A=)(A01 A02)02A=)A1 is fine when is a finite set, but we require slightly more to deal with the common situationwhen is infinite.

10 We require the collection of events to be closed under the operation of takingcountable unions, not just finite collectionAof subsets of is called a fieldif it satisfies the followingconditions:(I);2A;(II) ifA1,A2,..2 Athen i=1Ai2A;(III) recap, with any experiment we may associate a pair ( ,A), where is the set of all possibleoutcomes (orelementary events) andAis a field of subsets of , which contains all the eventsin whose occurrences we may be interested. So, from now on, to call a setAan event is equivalentto asserting thatAbelongs to the field in THE PROBABILITY FUNCTIOND efinition an eventE , theprobability thatEoccurswill be writtenP(E).