Springer Texts in Statistics - MIM

1 Springer Texts in StatisticsAdvisors:George Casella Stephen Fienberg Ingram OlkinSpringer Texts in Statistics Alfred: Elements of Statistics for the Life and Social Sciences Berger: An Introduction to Probability and Stochastic Processes Bilodeau and Brenner: Theory of Multivariate Statistics Blom: Probability and Statistics : Theory and Applications Brockwell and Davis: Introduction to Times Series and Forecasting, Second Edition Chow and Teicher: Probability Theory: Independence, Interchangeability, Martingales, Third Edition Christensen: Advanced Linear Modeling: Multivariate, Time Series, and Spatial Edition Christensen: Log-Linear Models and Logistic Regression, Second Edition Christensen: Plane Answers to Complex Questions: The Theory of Linear Models, Third Edition Creighton: A First Course in Probability Models and Statistical Inference Davis: Statistical Methods for the Analysis of Repeated Measurements Dean and Voss: Design and Analysis of Experiments du Toit, Steyn, and Stumpf: graphical Exploratory Data Analysis Durrett: Essentials of Stochastic Processes Finkelstein and Levin: Statistics for Lawyers Flury: A First Course in Multivariate Statistics Jobson: Applied Multivariate Data Analysis, Volume I: Regression and Experimental Design Jobson: Applied Multivariate Data Analysis, Volume II: Categorical and Multivariate Methods Kalbfleisch: Probability and Statistical Inference, Volume I: Probability, Second Edition Kalbfleisch: Probability and Statistical Inference, Volume II: Statistical Inference, Second Edition Karr: Probability Keyfitz: Applied Mathematical Demography, Second Edition Kiefer: Introduction to Statistical Inference Kokoska and Nevison: Statistical Tables and Formulae Kulkarni.

2 Modeling, Analysis, Design, and Control of Stochastic Systems Lange: Applied Probability Lehmann: Elements of Large-Sample Theory Lehmann: Testing Statistical Hypotheses, Second Edition Lehmann and Casella: Theory of Point Estimation, Second Edition Lindman: Analysis of Variance in Experimental Design Lindsey: Applying Generalized Linear Models Data: Nonparametric Regression and Response Surface Maximization, Second Edwards: Introduction to graphical Modelling, Second Edition (continued after index)Jun ShaoMathematical StatisticsSecond EditionJun ShaoDepartment of StatisticsUniversity of Wisconsin, MadisonMadison, WI BoardGeorge CasellaStephen FienbergIngram OlkinDepartment of StatisticsDepartment of StatisticsDepartment of StatisticsUniversity of FloridaCarnegie Mellon UniversityStanford UniversityGainesville, FL 32611-8545 Stanford, CA 94305 USAUSAUSAWith 7 of Congress Cataloging-in-Publication DataShao, Statistics / Jun Shao.

3 2nd cm. ( Springer Texts in Statistics )Includes bibliographical references and 0-387-95382-5 (alk. paper)1. Mathematical Statistics . I. Title. II. dc212003045446 ISBN 0-387-95382-5 All rights reserved. This work may not be translated or copied in whole or in part without theNew York, , 10013, USA), except for brief excerpts in connection with reviews or , computer software, or by similar or dissimilar methodology now known or hereafterdeveloped is are not identified as such, is not to be taken as an expression of opinion as to whether ornot they are subject to proprietary in the United States of in connection with any form ofISBN-13 978-0-387-95382-3 Printed on acid-free (corrected printing as of 4th printing, 2007) 2003 Springer Science+Business Media, permission of the publisher ( Springer Science+Business Media, LLC., 233 Spring St.,Pittsburgh, PA 15213-3890 The use in this publication of trade names, trademarks, service marks, and similar terms, even ifinformation storage and retrieval, electronicTo Guang, Jason, and AnniePreface to the FirstEditionThis book is intended for a course entitledMathematical Statisticsofferedat the Department of Statistics , University of Wisconsin-Madison.

4 Thiscourse, taught in a mathematically rigorous fashion, covers essential ma-terials in statistical theory that a first or second year graduate studenttypically needs to learn as preparation for work on a degree in statis-tics. The course is designed for two 15-week semesters, with three lecturehours and two discussion hours in each week. Students in this course areassumed to have a good knowledge of advanced calculus. A course in realanalysis or measure theory prior to this course is often 1 provides a quick overview of important concepts and resultsin measure-theoretic probability theory that are used as tools in math-ematical Statistics . Chapter 2 introduces some fundamental concepts instatistics, including statistical models, the principle of sufficiency in datareduction, and two statistical approaches adopted throughout the book:statistical decision theory and statistical inference. Each of Chapters 3through 7 provides a detailed study of an important topic in statistical de-cision theory and inference: Chapter 3 introduces the theory of unbiasedestimation; Chapter 4 studies theory and methods in point estimation un-der parametric models; Chapter 5 covers point estimation in nonparametricsettings; Chapter 6 focuses on hypothesis testing; and Chapter 7 discussesinterval estimation and confidence sets.

5 The classical frequentist approachis adopted in this book, although the Bayesian approach is also introduced( , , , and ). Asymptotic (large sample) theory, a cru-cial part of statistical inference, is studied throughout the book, rather thanin a separate 85% of the book covers classical results in statistical theory thatare typically found in textbooks of a similar level. These materials are in theStatistics Department s qualifying examination syllabus. This partof the book is influenced by several standard textbooks, such as Casella andviiviiiPreface to the First EditionBerger (1990), Ferguson (1967), Lehmann (1983, 1986), and Rohatgi (1976).The other 15% of the book covers some topics in modern statistical theorythat have been developed in recent years, including robustness of the leastsquares estimators, Markov chain Monte Carlo, generalized linear models,quasi-likelihoods, empirical likelihoods, statistical functionals, generalizedestimation equations, the jackknife, and the addition to the presentation of fruitful ideas and results, this bookemphasizes the use of important tools in establishing theoretical , most proofs of theorems, propositions, and lemmas are providedor left as exercises.

6 Some proofs of theorems are omitted (especially inChapter 1), because the proofs are lengthy or beyond the scope of thebook (references are always provided). Each chapter contains a number ofexamples. Some of them are designed as materials covered in the discussionsection of this course, which is typically taught by a teaching assistant (asenior graduate student). The exercises in each chapter form an importantpart of the book. They provide not only practice problems for students,but also many additional results as complementary materials to the book is essentially based on (1) my class notes taken in 1983-84when I was a student in this course, (2) the notes I used when I was ateaching assistant for this course in 1984-85, and (3) the lecture notes Iprepared during 1997-98 as the instructor of this course. I would like toexpress my thanks to Dennis Cox, who taught this course when I wasa student and a teaching assistant, and undoubtedly has influenced myteaching style and textbook for this course.

7 I am also very grateful tostudents in my class who provided helpful comments; to Mr. Yonghee Lee,who helped me to prepare all the figures in this book; to the Springer -Verlagproduction and copy editors, who helped to improve the presentation; andto my family members, who provided support during the writing of , WisconsinJun ShaoJanuary 1999 Preface to the SecondEditionIn addition to correcting typos and errors and making a better presentation,the main effort in preparing this new edition is adding some new materialto Chapter 1 (Probability Theory) and a number of new exercises to eachchapter. Furthermore, two new sections are created to introduce semipara-metric models and methods ( ) and to study the asymptotic accuracyof confidence sets ( ). The structure of the book remains the Chapter 1 of the new edition, moment generating and characteristicfunctions are treated in more detail and a proof of the uniqueness theoremis provided; some useful moment inequalities are introduced; discussionson conditional independence, Markov chains, and martingales are added,as a continuation of the discussion of conditional expectations; the con-cepts of weak convergence and tightness are introduced; proofs to some keyresults in asymptotic theory, such as the dominated convergence theoremand monotone convergence theorem, the L evy-Cram er continuity theorem,the strong and weak laws of large numbers, and Lindeberg s central limittheorem, are included; and a new section ( ) is created to introduceEdgeworth and Cornish-Fisher expansions.

8 As a result, Chapter 1 of thenew edition is self-contained for important concepts, results, and proofs inprobability theory with emphasis in statistical the original book was published in 1999, I have been using it asa textbook for a two-semester course in mathematical Statistics . Exerciseproblems accumulated during my teaching are added to this new exercises that are too trivial have been the original book, indices on definitions, examples, theorems, propo-sitions, corollaries, and lemmas are included in the subject index. In thenew edition, they are in a separate index given in the end of the book (priorto the author index). A list of notation and a list of abbreviations, whichare appendices of the original book, are given after the to the Second EditionThe most significant change in notation is the notation for a the text of the new edition, ak-dimensional vector is denoted byc=(c1,..,ck), whether it is treated as a column or a row vector (which is notimportant if matrix algebra is not considered).

9 When matrix algebra isinvolved, any vectorcis treated as ak 1 matrix (a column vector) andits transposec is treated as a 1 kmatrix (a row vector). Thus, forc= (c1,..,ck),c c=c21+ +c2kandcc is thek kmatrix whose (i,j)thelement would like to thank reviewers of this book for their constructive com-ments, the Springer -Verlag production and copy editors, students in myclasses, and two teaching assistants, Mr. Bin Cheng and Dr. HanshengWang, who provided help in preparing the new edition. Any remainingerrors are of course my own responsibility, and a correction of them maybe found on my web page , WisconsinJun ShaoApril, 2003 ContentsPreface to the First EditionviiPreface to the Second EditionixChapter 1. Probability Probability Spaces and Random Elements .. -fields and measures .. Measurable functions and distributions .. Integration and Differentiation .. Integration .. Radon-Nikodym derivative .. Distributions and Their Characteristics.

10 Distributions and probability densities .. Moments and moment inequalities .. Moment generating and characteristic functions .. Conditional Expectations .. Conditional expectations .. Independence .. Conditional distributions .. Markov chains and martingales .. Asymptotic Theory .. Convergence modes and stochastic orders .. Weak convergence .. Convergence of transformations .. The law of large numbers .. The central limit theorem .. Edgeworth and Cornish-Fisher expansions .. Exercises .. 74 Chapter 2. Fundamentals of Populations, Samples, and Models .. Populations and samples .. Parametric and nonparametric models .. Exponential and location-scale families .. Statistics , Sufficiency, and Completeness .. Statistics and their distributions .. Sufficiency and minimal sufficiency .. Complete Statistics .. Statistical Decision Theory.