Statistical Theory - Pomona College

1 Statistical TheoryLecture NotesAdolfo J. Rumbosc Draft date December 18, 20092 Contents1 Introduction to Statistical inference .. Introductory Example .. : Concepts and Terminology ..72 Estimating the Mean of a Distribution .. Interval Estimate for Proportions .. Interval Estimates for the Mean .. 2 Distribution .. Distribution .. from a normal distribution .. of the Sample Variance from a Normal Dis-tribution .. Distribution of .. 393 Hypothesis Chi Square Goodness of Fit Test.

2 Multinomial Distribution .. Pearson Chi-Square Statistic .. of Fit Test .. The Language and Logic of Hypothesis Tests .. Hypothesis Tests in General .. Likelihood Ratio Test .. The Neyman Pearson Lemma .. 734 Evaluating Mean Squared Error .. Cr amer Rao Theorem .. 80A Pearson Chi Square Statistic87B The Variance of the Sample Variance9334 CONTENTSC hapter Introduction to Statistical inferenceThe main topic of this course is Statistical inference .

3 Loosely speaking, statisti-cal inference is the process of going from information gained from a sample toinferences about a population from which the sample is taken. There are twoaspects of Statistical inference that we ll be studying in this course: estimationand hypothesis testing. In estimation, we try to determine parameters from apopulation based on quantities, referred to as statistics, calculated from datain a sample. The degree to which the estimates resemble the parameters be-ing estimated can be measured by ascertaining the probability that a certainrange of values around the estimate will contain the actual parameter.

4 The useof probability is at the core of Statistical inference ; it involves the postulationof a certain probability model underlying the situation being studied and cal-culations based on that model. The same procedure can in turn be used todetermine the degree to which the data in the sample support the underlyingmodel; this is the essence of hypothesis we delve into the details of the Statistical Theory of estimation andhypothesis testing, we will present a simple example which will serve to illustrateseveral aspects of the An Introductory ExampleI have a hot air popcorn popper which I have been using a lot lately.

5 It is asmall appliance consisting of a metal, cylindrical container with narrow ventsat the bottom, on the sides of the cylinder, through which hot air is vents are slanted in a given direction so that the kernels are made tocirculate at the bottom of the container. The top of the container is coveredwith a hard-plastic lid with a wide spout that directs popped and unpoppedkernels to a container placed next to the popper. The instructions call forone quarter cup of kernels to be placed at the bottom of the container andthe device to be plugged in.

6 After a short while of the kernels swirling in hot56 CHAPTER 1. INTRODUCTIONair, a few of the kernels begin to pop. Pressure from the circulating air andother kernels popping an bouncing off around inside the cylinder forces kernelsto the top of the container, then to the spout, and finally into the you start eating the popcorn, you realize that not all the kernels also notice that there are two kinds of unpopped kernels: those that justdidn t pop and those that were kicked out of the container before they couldget warm enough to pop.

7 In any case, after you are done eating the poppedkernels, you cannot resit the temptation to count how many kernels did not shows the results of 27 popping sessions performed under nearly thesame conditions. Each popping session represents a random of Uppopped Kernels132211332495176877815913910110111 2412111136714143153516521735186519442052 21492218235624131255526592737 Table : Number of Unpopped Kernels out of 1/4 cup of popcorn1 Arandom experimentis a process or observation, which can be repeated indefinitelyunder the same conditions, and whose outcomes cannot be predicted with certainty beforethe experiment is INTRODUCTION TO Statistical INFERENCE7number of unpopped kernels is a random variable2which we obtain from theoutcome of each experiment.

8 Denoting the number of unpopped kernels in agiven run by , we may postulate that follows a Binomial distribution withparameters and , where is the probability that a given kernel will not pop(either because it was kicked out of the container too early, or because it wouldjust not pop) and is the number of kernels contained in one-quarter cup. Wewrite binom( , )and have that ( = ) = (1 ) for = 0,1,2,.., ,where = ! !( )!, = 0,1, , .This is the underlying probability model that we may postulate for this situa-tion.

9 The probability of a failure to pop for a given kernel, , and the number ofkernels, , in one quarter cup are unknown parameters. The challenge beforeus is to use the data in Table on page 6 to estimate the parameter . No-tice that is also unknown, so we ll also have to estimate as well; however,the data in Table do not give enough information do so. We will thereforehave to design a new experiment to obtain data that will allow us to estimate . This will be done in the next chapter. Before we proceed further, we willwill lay out the sampling notions and terminology that are at the foundation ofstatistical Sampling: Concepts and TerminologySuppose we wanted to estimate the number of popcorn kernels in one quartercup of popcorn.

10 In order to do this we can sample one quarter cup from abag of popcorn and count the kernels in the quarter cup. Each time we dothe sampling we get a value, , for the number of kernels. We postulate thatthere is a value, , which gives the mean value of kernels in one quarter cup ofpopcorn. It is reasonable to assume that the distribution of each of the , for = 1,2,3,.., is normal around with certain variance 2. That is, normal( , 2) for all = 1,2,3,..,so that each of the s has a density function, , given by ( ) =1 2 ( )22 2,for < <.