Theory of Statistics

1 James E. Gentle Theory of Statistics Theory of Statistics 2000 2013. c James E. Gentle Theory of Statistics 2000 2013. c James E. Gentle Preface: Mathematical Statistics After teaching mathematical Statistics for several years using chalk on a black- board (and, later, smelly dry erase markers on a whiteboard) mostly doing proofs of theorems, I decided to lecture from computer slides that provide an outline of the big picture . Rather than spend class time doing proofs that are given in standard texts, I decided that time would be better spent discussing the material from a different, higher-level perspective. While lecturing from canned slides, I cannot, however, ignore certain de- tails of proofs and minutiae of examples. But what details and which minutiae? To be effective, my approach depends on an understanding between students and the instructor, an understanding that is possibly implicit.

2 I lecture; but I. ask what is .. ? and why is .. ? ; and I encourage students to ask what is .. ? and why is .. ? . I adopt the attitude that there are many things that I. don't know, but if there's something that I wonder about, I'll admit ignorance and pursue the problem until I've attained some resolution. I encourage my students to adopt a similar attitude. I am completely dissatisfied with a class that sits like stumps on a log when I ask what is .. ? or why is .. ? during a lecture. What can I say? After writing class slides (in LATEX 2 , of course), mostly in bullet form, I. began writing text around the bullets, and I put the notes on the class website. Later I decided that a single document with a fairly extensive subject index (see pages 881 through 899) would be useful to serve as a companion for the study of mathematical Statistics .

3 The other big deal about this document is the internal links, which of course is not something that can be done with a hardcopy book. (One thing I must warn you about is that there is a (known). bug in the LATEX package hyperref; if the referenced point happens to occur at the top of a page, the link takes you to the previous page so if you don't see what you expect, try going to the next page.). Much of the present document reflects its origin as classroom notes; it contains incomplete sentences or sentence fragments, and it lacks connective material in some places. (The connective material was (probably!) supplied orally during the lectures.). Theory of Statistics 2000 2013. c James E. Gentle vi Preface Another characteristic of this document that results from the nature of its origin, as well as from the medium itself (electronic) is its length.

4 A long book doesn't use any more paper or kill any more trees than a short book. Usually, however, we can expect the density of importance in a short document to be greater than that in a long document. If I had more time I could make this book shorter by prioritizing its content, and I may do that someday. Several sections are incomplete and several proofs are omitted. Also, I plan to add more examples. I just have not had time to type up the material. I do not recommend that you print these notes. First of all, they are evolv- ing, so a printed version is just a snapshot. Secondly, the PDF file contains active internal links, so navigation is easy. (For versions without active links, I. try to be friendly to the reader by providing page numbers with most internal references.)

5 This document is directed toward students for whom the Theory of statis- tics is or will become an important part of their lives. Obviously, such students should be able to work through the details of hard proofs and derivations;. that is, students should master the fundamentals of mathematical Statistics . In addition, students at this level should acquire, or begin acquiring, a deep appreciation for the field, including its historical development and its rela- tion to other areas of mathematics and science generally; that is, students should master the fundamentals of the broader Theory of Statistics . Some of the chapter endnotes are intended to help students gain such an appreciation by leading them to background sources and also by making more subjective statements than might be made in the main body.

6 It is important to understand the intellectual underpinnings of our science. There are certain principles (such as sufficiency, for example) that guide our approaches to statistical inference. There are various general approaches (see page 235) that we follow. Within these general approaches there are a number of specific methods (see page 236). The student should develop an appreciation for the relations between principles, approaches, and methods. This book on mathematical Statistics assumes a certain amount of back- ground in mathematics. Following the final chapter on mathematical Statistics chapter 8, there is chapter 0 on statistical mathematics (that is, mathe- matics with strong relevance to statistical Theory ) that provides much of the general mathematical background for probability and Statistics .

7 The mathe- matics in this chapter is prerequisite for the main part of the book, and it is hoped that the reader already has command of the material; otherwise, chapter 0 can be viewed as providing just in time mathematics. chapter 0. grew (and is growing) recursively. Every time I add material, it seems that I. need to add some background for the new material. This is obviously a game one cannot win. probability Theory is the most directly relevant mathematical background, and it is assumed that the reader has a working knowledge of measure- Theory - based probability Theory . chapter 1 covers this Theory at a fairly rapid pace. Theory of Statistics 2000 2013. c James E. Gentle Preface vii The objective in the discussion of probability Theory in chapter 1, as in that of the other mathematical background, is to provide some of the most relevant material for Statistics , which is the real topic of this text.

8 chapter 2 is also on probability , but the focus is on the applications in Statistics . In that chapter , I. address some important properties of probability distributions that determine properties of statistical methods when applied to observations from those distributions. chapter 3 covers many of the fundamentals of Statistics . It provides an overview of the topics that will be addressed in more detail in Chapters 4. through 8. This document is organized in the order in which I cover the topics (more- or-less!). Material from chapter 0 may be covered from time to time during the course, but I generally expect this chapter to be used for reference as needed for the statistical topics being covered. The primary textbooks I have used in the past few years are Shao (2003), Lehmann and Casella (1998), and Lehmann (1986) (the precursor to Lehmann and Romano (2005)).

9 At various places in this document, references are given to the related sections of Shao (2003) ( MS2 ), Lehmann and Casella (1998) ( TPE2 ), and Lehmann and Romano (2005) ( TSH3 ). These texts state all of the important theorems, and in most cases, provide the proofs. They are also replete with examples. Full bibliographic citations for these references, as well as several other resources are given in the bibliography beginning on page 865. It is of course expected that the student will read the primary textbook, as well as various other texts, and to work through all proofs and examples in the primary textbook. As a practical matter, obviously, even if I attempted to cover all of these in class, there just is not enough class time to do it. The purpose of this evolving document is not just to repeat all of the material in those other texts.

10 Its purpose, rather, is to provide some additional background material, and to serve as an outline and a handy reference of terms and concepts. The nature of the propositions vary consider- ably; in some cases, a fairly trivial statement will be followed by a proof, and in other cases, a rather obtuse statement will not be supported by proof. In all cases, the student should understand why the statement is true (or, if it's not, immediately send me email to let me know of the error!). More importantly, the student should understand why it's relevant. Each student should read this and other texts and work through the proofs and examples at a rate that matches the individual student's understanding of the individual problem. What one student thinks is rather obtuse, another student comprehends quickly, and then the tables are turned when a different problem is encountered.