Probability and Distributions (Hogg Chapter One)

1 Probability and Distributions (Hogg Chapter One)STAT 405-01: Mathematical Statistics I Fall Semester 2015 Contents0 Administrata .. Outline ..21 Review of Probability Set Theory and Logic .. Defining and Assigning Probabilities .. Basic Rules of Probability ..42 Random The cumulative distribution function .. Transformations .. of the cdf and pmf .. of the pdf ..93 Expectation Mean, Variance and Moments .. The Moment Generating Function .. 11 Copyright 2015, John T. Whelan, and all that4 Important Existence of Lower Moments .. Markov s Inequality .. Chebyshev s Inequality .. Jensen s Inequality .. 14 Tuesday 25 August 2015 Read Sections of Hogg0 Administrata Introductions!

2 Mathematical Diagnostic (not graded; intended as a peda-gogical guide for me). Syllabus Instructor s name (Whelan) rhymes with wailin . Text: Hogg, McKean, and Craig,Introduction to Mathe-matical Statistics, 7th edition. Other useful books:1 Casella and Berger,Statistical Inference, 2nd is a standard first-year graduate text in statis-tics. It covers roughly the same material, but witha little more sophistication (more possible pathologiesare mentioned) but also more of a practical philosophy. Jaynes, Probability Theory: the Logic of Science. Thisis a sort of Bayesian manifesto and as such doesn toverlap much with the traditional approach, but it sgot a lot of interesting bits in it, such as a demon-stration that you can derive Probability as an obviousextension of logic.

3 Coursewebsite: ~whelan/STAT-405/ Will contain links to notes and problem sets; coursecalendar is probably the most useful. Course calendar:tentativetimetable for course. Course work: Please read the relevant sections of the textbookbeforeclass so as to be prepared for class discussions. There will be quasi-weekly homeworks. Collaborationis allowed an encouraged, but please turn in your ownwork, as obviously identical homeworks may not re-ceive credit. There will be two prelim exams, in class, and one cu-mulative final exam. Grading:5% Class Participation20% Problem Sets20% First Prelim Exam20% Second Prelim Exam35% Final ExamYou ll get a separate grade on the quality point scale( , is the B+ range) for each of these fivecomponents; course grade is weighted Outline1.

4 Random Variables ( Chapter One)2. multivariate Distributions ( Chapter Two)3. Specific Probability Distributions (binomial, Poisson, nor-mal, 2( Chapter Three)4. Statistical Inference ( Chapter Four)5. Central limit theorem ( Chapter Five)Warning: the material in this class is rather advanced. Pleasemake sure you re familiar with what you covered in AppliedStatistics or Engineering Statistics, as well as multi-variable Review of Probability TheorySections of Hogg build up Probability theory in a some-what formal and axiomatic way, and in particular they developthe formalism of set theory which is used to combine you should already be familiar with these principles fromProbability or Applied Statistics, and since most of this courseis concerned with random variables rather than abstract prob - ability , we ll just take a quick refresher, and make a few pointsby way of Set Theory and LogicAt its heart, Probability theory applies to each event a realnumber between zero and one.)

5 There are two different, math-2ematically equivalent, ways to understand what s meant by event : one based on set theory and the other based on books tend to define things in the set theory way,in which there is asample space,C, consisting of all of thepossibleoutcomesof an experiment, with an individual out-come labelledc, so thatc C. Then an eventCis some set ofoutcomes, , a subset ofC,C the application of Probability theory, though, it s easier tothink of events as being logical propositions, statements aboutthe outcome of an experiment which could be true or false. Theidea connecting the two is that for each outcome of the experi-ment, a given statement is either true or false. The set associatedwith an eventCis the set of all outcomes in which the state-ment in question is true.

6 (We ll be making these statementsin the context of a repeatable experiment, but in fact the en-tire mathematical formalism can be extended to any true/falsestatements that can be made about the world.)There are basic operations in set theory used to combineevents into other events, and each one of them has an analoguein the formalism of logic: ThecomplementCcof the eventCis the set of all out-comes which are not inC. In terms of logic, this is notC , written as C,C orC. It is a statement which is trueifCis false and false ifCis true. TheunionC1 C2is the set of all outcomes which are inC1orC2, or both. In logic, this is C1orC2 (where wemean an inclusive or ), writtenC1 C2, a statement whichis true if eitherC1orC2or both are true. TheintersectionC1 C2is the set of all outcomes whichare in bothC1andC2.

7 In logic, this is C1andC2 , writtenC1 C2, a statement which is true if can connect the two ideas using a truth table OutcomeC1C2C1 C2C1 C2 C1C1 C2 TTTTFC1 Cc2 TFFTFCc1 C2 FTFTTCc1 Cc2 FFFFTIn set theory, two events are equal if the sets of outcomes theycontain are identical; in logic, this corresponds to one eventbeing true whenever the other is true and false whenever theother is false. Finally, it s useful to define the null event =Ccwhich contains no outcomes and therefore is always Defining and Assigning ProbabilitiesMathematically speaking, Probability is a number between 0ans 1 which is assigned to each event. , the eventChasprobabilityP(C). If we think about the logical definition ofevents, then we have P(C) = 1 means the statement corresponding toCis defi-nitely true.

8 P(C) = 0 means the statement corresponding toCis defi-nitely false. 0< P(C)<1 means the statement corresponding toCcould be true or standard numerical interpretation of the probabilityP(C) isin terms of a repeatable experiment with some random that we repeat the same experiment over and over againmany times under identical conditions. In each iteration of theexperiment (each game of craps, sequence of coin flips, opinionsurvey, etc), a given outcome or event will represent a statementthat is either true or false. Over the long run, the fraction ofexperiments in which the statement is true will be approximately3given by the Probability of the corresponding outcome or we write the number of repetitions of the experiment asN,and the number of experiments out of thoseNin whichCistrue as #C, thenlimN #CN=P(C)( )You can test this proposition on the optional numerical exer-cise on this week s problem set.

9 This interpretation of proba-bility is sometimes called the frequentist interpretation, sinceit involves the relative frequency of outcomes in repeated ex-periements. It s actually a somewhat more limited interpreta-tion than the Bayesian interpretation, in which the probabilityof an event corresponds to a quantitative degree of certainty thatthe corresponding statement is true. (Devore somewhat pejora-tively calls this subjective Probability .) These finer points arebeyond the scope of this course, but if you re interested, you maywant to look up , Probability Theory: The Logic of Scienceby E. T. Basic Rules of ProbabilityIt s a standard approach to develop a formal theory of prob - ability starting from a few axioms, and derives other sensibleresults from those.

10 This is an interesting intellectual exercise,but for our purposes, it s enough to note certain simple prop-erties which make sense for our understanding of Probability asthe likelihood that a statement is true:1. For any eventC, 0 P(C) (C) = 1 andP( ) = 0 (something always happens) (Cc) = 1 P(C) (the Probability that a statement is falseis one minus the Probability that it s IfC1andC2are disjoint events,P(C1 C2) =P(C1)+P(C2)One useful non-trivial result concerns the Probability of theunion of any two events. SinceC1 C2= (C1 Cc2) (C1 C2) (Cc1 C2), the union of three disjoint events,P(C1 C2) =P(C1 Cc2) +P(C1 C2) +P(Cc1 C2) ( )On the other hand,C1= (C1 Cc2) (C1 C2) andC2=(C1 C2) (Cc1 C2), soP(C1) =P(C1 Cc2) +P(C1 C2)( )P(C2) =P(C1 C2) +P(Cc1 C2)( )which means thatP(C1) +P(C2) =P(C1 Cc2) + 2P(C1 C2) +P(Cc1 C2)=P(C1 C2) +P(C1 C2)( )soP(C1 C2) =P(C1) +P(C2) P(C1 C2)( )Another important concept is conditional Probability , butwe ll postpone consideration of this until we talk about con-ditional Distributions for random Random VariablesIn this course we ll primarily be interested in random random variable (or rv for short)Xassigns exactly one valueX(c) to each outcomecin the sample spaceC.)