MATH 2P82 MATHEMATICAL STATISTICS (Lecture Notes)

1 MATH 2P82 MATHEMATICAL STATISTICS (Lecture Notes) c Jan Vrbik23 Contents1 PROBABILITY REVIEW7 Basic 7 binomial 7 Multinomial 7 Random Experiments (Basic Definitions).. 7 Sample 8 Set 8 Boolean 8 Probability of 8 Probability 9 Important 9 Probability 9 Product 9 Conditional 9 Total-probability 10 Discrete Random 10 Bivariate (joint) 11 Conditional 11 Multivariate 11 Expected Value of a 11 Expected values related 12 Moments (univariate).. 12 Moments (bivariate or joint ).

2 12 Variance ofaX+bY+ 13 Moment generating 13 Main 13 Probability generating 13 Conditional expected 14 Common discrete 14 Negative 15 Multivariate 15 Continuous Random 16 Univariate probability density function (pdf).. 16 Distribution 16 Bivariate (multivariate) 16 Marginal 16 Conditional 17 Mutual 17 Expected 17 Common Continuous 18 Transforming Random 192 Transforming Random Variables21 Univariate 21 Distribution-Function(F) 21 Probability-Density-Function(f) 23 Bivariate 24 Distribution-Function 24 Pdf (Shortcut) 253 Random Sampling31 Sample 31 Central Limit 31 Sample 32 Sampling fromN( , ).

3 33 Sampling without 35 Bivariate 364 OrderStatistics37 Univariate 37 Sample 38 Bivariate 40 Special 415 Estimating Distribution 45 Cram r-Rao 50 Method of 51 One 52 Two 53 Maximum-likelihood 53 One 5556 Confidence Intervals57CI for mean .. 57 58 Large-sample 58 Difference of two 58 Proportion(s).. 59 Variance(s).. 60 607 Testing Hypotheses61 Tests concerning mean(s).. 62 Concerning variance(s).. 63 Concerning proportion(s).. 63 Contingency 63 Goodness 638 Linear Regression and Correlation65 Simple 65 Maximum likelihood 65 Least-squares 65 Normal 66 Statistical properties of the 67 Confidence 70 Multiple 71 Various standard 739 AnalysisofVariance75 One-way 75 Two-way 76No 77 With 7810 Nonparametric Tests79 Sign 79 Signed-rank 79 Rank-sum 81 Run 81(Sperman s) rank correlation 8367 Chapter 1 PROBABILITY REVIEWB asic CombinatoricsNumber of permutations ofndistinct objects:n!

4 Not all distinct, such as, for exampleaaabbc:6!3!2!1! 63,2,1 orN!n1!n2!n3!..nk! Nn1,n2,n3, .., nk in general, whereN=kPi=1niwhich is the total word length (multinomial coef-ficient).Selectingrout ofnobjects (without duplication), counting all possible arrange-ments:n (n 1) (n 2) .. (n r+1)=n!(n r)! (number of permutations).Forget theirfinal arrangement:Pnrr!=n!(n r)!r! (number of combinations). This will also be called thebinomial we can duplicate (any number of times), and count the arrangements:nrBinomial expansion(x+y)n=nXi=0 ni xn iyiMultinomial expansion(x+y+z)nXi,j,k 0i+j+k=n ni, j, k xiyjzk(x+y+z+w)n=Xi,j,k,c 0i+j+k+c=n ni, j, k, c Experiments (Basic Definitions)Sample spaceis a collection of all possible outcomes of an individual (complete) outcomes are calledsimple the sample space (A, B, C.)

5 Set TheoryThe old notion of:is (are) now called:Universal set Sample spaceElements of (its individual points )Simple events (complete outcomes)Subsets of EventsEmpty set Null eventWe continue to use the wordintersection(notation:A B, representingthe collection of simple events common to bothAandB),union(A B,simpleeventsbelongingtoeitherAorBorbot h), andcomplement(A,simple eventsnotinA). One should be able to visualize these using Venn diagrams, but whendealing with more than 3 events at a time, one can tackle problems only with thehelp ofBoolean AlgebraBoth and (individually) isdistributiveover union:A (B C.)

6 =(A B) (A C) ..Similarly, union is distributive over intersection:A (B C ..)=(A B) (A C) ..Trivial rules:A =A, A = ,A A=A, A = ,A =A,A A=A, A A= ,A A= , A= , whenA B(Ais asubsetofB,meaning that every element ofAalsobelongs toB),we get:A B=A(the smaller event) andA B=B(the biggerevent).DeMorgan Laws:A B=A B,andA B=A B,or in generalA B C ..=A B C ..and vice versa ( ).AandBare called (mutually)exclusiveordisjointwhenA B= (nooverlap).Probability of EventsSimple eventscan be assigned aprobability(relative frequency of its occurrencein alongrun).

7 It s obvious that each of these probabilities must be a non-negativenumber. Tofind a probability ofany othereventA(not necessarily simple), wethen add the probabilities of the simple eventsAconsists of. This immediatelyimplies that probabilities must follow a few basic rules:Pr(A) 0Pr( )=0Pr( )=1(the relative frequency of all is obviously1).We should mention thatPr(A)=0does not necessarily imply thatA= .9 Probability rulesPr(A B)=Pr(A)+Pr(B)butonlywhenA B= (disjoint). This implies thatPr(A)=1 Pr(A)as a special also implies thatPr(A B)= Pr(A) Pr(A B).

8 For anyAandB(possibly overlapping) we havePr(A B)=Pr(A)+Pr(B) Pr(A B)Can be extended to:Pr(A B C)=Pr(A)+Pr(B)+Pr(C) Pr(A B) Pr(A C) Pr(B C)+Pr(A B C).IngeneralPr(A1 A2 A3 .. Ak)=kXi=1Pr(Ai) kXi<jPr(Ai Aj)+kXi<j<cPr(Ai Aj Ac) .. Pr(A1 A2 A3 .. Ak)The formula computes the probability thatat least oneof theAievents probability of gettingexactly oneof theAievents is similarly computedby:kXi=1Pr(Ai) 2kXi<jPr(Ai Aj)+3kXi<j<cPr(Ai Aj Ac) .. kPr(A1 A2 A3 .. Ak)Important resultProbability of any (Boolean) expression involving eventsA, B, C.

9 Can bealwaysconverted to a linear combination of probabilities of the individual events and theirsimple (non-complemented)intersections(A B, A B C,etc.) treeis a graphical representation of a two-stage (three-stage) random experiment.(effectivelyits sample space - each complete path being a simple event).The individual branch probabilities (usually simple tofigure out), are the socalledconditional rulePr(A B)=Pr(A) Pr(B|A)Pr(A B C)=Pr(A) Pr(B|A) Pr(C|A B)Pr(A B C D)=Pr(A) Pr(B|A) Pr(C|A B) Pr(D|A B C).

10 Conditional probabilityThe general definition:Pr(B|A) Pr(A B)Pr(A)All basic formulas of probability remain , :Pr(B|A)=1 Pr(B|A),Pr(B C|A)=Pr(B|A)+Pr(C|A) Pr(B C|A), formulaApartitionrepresents chopping the sample space into several smaller events, sayA1,A2,A3,..,Ak,so that they(i)don t overlap ( are all mutually exclusive):Ai Aj= for any1 i, j k(ii)cover the whole ( no gaps ):A1 A2 A3 .. Ak= .For any partition, and an unrelated evenB,we havePr(B)=Pr(B|A1) Pr(A1)+Pr(B|A2) Pr(A2)+..+Pr(B|Ak) Pr(Ak)Independenceof two events is a very natural notion (we should be able to tell from the experi-ment): when one of these events happens, it does not effect the probability of theother.