Mathematical Statistics - ETH Z

1 Mathematical StatisticsSara van de GeerSeptember 20102 Contents1 Some notation and model assumptions .. Estimation .. Comparison of estimators: risk functions .. Comparison of estimators: sensitivity .. Confidence intervals .. confidence sets and tests .. Intermezzo: quantile functions .. How to construct tests and confidence sets .. An illustration: the two-sample problem .. normality .. nonparametric test .. of Student s test and Wilcoxon s test .. How to construct estimators .. estimators .. method of moments .. methods .. 232 Decision Decisions and their risk.

2 Admissibility .. Minimaxity .. Bayes decisions .. Intermezzo: conditional distributions .. Bayes methods .. Discussion of Bayesian approach (to be written) .. Integrating parameters out (to be written) .. Intermezzo: some distribution theory .. multinomial distribution .. Poisson distribution .. distribution of the maximum of two random variables Sufficiency .. Rao-Blackwell .. Factorization Theorem of Neyman .. Exponential families .. Canonical form of an exponential family .. Minimal sufficiency .. 533 Unbiased What is an unbiased estimator?

3 UMVU estimators .. Statistics .. The Cramer-Rao lower bound .. Higher-dimensional extensions .. Uniformly most powerful tests .. example .. tests and exponential families .. tests .. tests .. 774 Equivariant Equivariance in the location model .. Equivariance in the location-scale model (to be written) .. 865 Proving admissibility and Minimaxity .. Admissibility .. Inadmissibility in higher-dimensional settings (to be written) .. 956 Asymptotic Types of convergence .. order symbols .. implications of convergence .. Consistency and asymptotic normality.

4 Linearity .. -technique .. M-estimators .. of M-estimators .. normality of M-estimators .. Plug-in estimators .. of plug-in estimators .. normality of plug-in estimators .. Asymptotic relative efficiency .. Asymptotic Cramer Rao lower bound .. Cam s 3rdLemma .. Asymptotic confidence intervals and tests .. likelihood .. ratio tests .. Complexity regularization(to be written).. 1397 Literature141 CONTENTS5 These notes in English will closely followMathematische Statistik, by unsch (2005), but are as yet Statistikcan be usedas supplementary reading material in rigor and clarity often bite each other.

5 At some places, not allsubtleties are fully presented. A snake will indicate 1 IntroductionStatistics is about the Mathematical modeling of observable phenomena, usingstochastic models, and about analyzing data: estimating parameters of themodel and testing hypotheses. In these notes, we study various estimation andtesting procedures. We consider their theoretical properties and we investigatevarious notions of Some notation and model assumptionsThe data consist of measurements (observations)x1,..,xn, which are regardedas realizations of random variablesX1,..,Xn. In most of the notes, theXiare real-valued:Xi R(fori= 1.)

6 ,n), although we will also consider someextensions to vector-valued and Foucault developed methods for estimating thespeed of light (1849, 1850), which were later improved by Newcomb and Michel-son. The main idea is to pass light from a rapidly rotating mirror to a fixedmirror and back to the rotating mirror. An estimate of the velocity of lightis obtained, taking into account the speed of the rotating mirror, the distancetravelled, and the displacement of the light as it returns to the rotating 1 The data are Newcomb s measurements of the passage time it took light totravel from his lab, to a mirror on the Washington Monument, and back to 1.

7 introduction distance: measurements on 3 consecutive daysfirst measurement: seconds= 24828 nanosecondsThe dataset has the deviations from 24800 measurements on 3 different days:llllllllllllllllllll0510152025 4002040day 1t1X1llllllllllllllllllllll202530354045 4002040day 2t2X2llllllllllllllllllllllllll404550556 065 4002040day 3t3X3 All measurements in one plot:lllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllll010203040 5060 40 SOME NOTATION AND MODEL ASSUMPTIONS9 One may estimate the speed of light using the mean, or the median, orHuber s estimate (see below). This gives the following results (for the 3 daysseparately, and for the three days combined):MeanMedianHuberDay 1 Day 2 Day 1 The question which estimate is the best one is one of the topics of these collection of observations will be denoted byX={X1.}

8 ,Xn}. Thedistribution ofX, denoted by IP, is generally unknown. A statistical model isa collection of assumptions about this unknown will usually assume that the observationsX1,..,Xnare independent andidentically distributed ( ). Or, to formulate it differently,X1,.., copies from some population random variable, which we denote common distribution, that is: the distribution ofX, is denoted byP. ForX R, the distribution function ofXis written asF( ) =P(X ).Recall that the distribution functionFdetermines the distributionP(and viseversa).Further model assumptions then concern the modeling ofP. We write sucha model asP P, wherePis a given collection of probability measures, theso-called model following example will serve to illustrate the concepts that are to real-valued.

9 The location model isP:={P ,F0(X ) :=F0( ), R, F0 F0},( )whereF0is a given collection of distribution functions. Assuming the expec-tation exist, we center the distributions inF0to have mean zero. ThenP ,F0has mean . We call a location parameter. Often, only is the parameter ofinterest, andF0is a so-called nuisance 1. INTRODUCTIONThe classF0is for example modeled as the class of all symmetric distributions,that isF0:={F0(x) = 1 F0( x), x}.( )This is an infinite-dimensional collection: it is not parametrized by a finitedimensional parameter. We then callF0an infinite-dimensional finite-dimensional model is for exampleF0:={ ( / ) : >0},( )where is the standard normal distribution , the location model isXi= + i, i= 1.

10 ,n,with 1,.., and, under model ( ), symmetrically but otherwise un-known distributed and, under model ( ),N(0, 2)-distributed with unknownvariance EstimationA parameter is an aspect of the unknown distribution. An estimatorTis somegiven functionT(X) of the observationsX. The estimator is constructed toestimate some unknown parameter, Example , one may consider the following estimators of : The average 1:=1nN i= that 1minimizes the squared lossn i=1(Xi ) can be shown that 1is a good estimator if the model ( ) holds. When( ) is not true, in particular when there areoutliers(large, wrong , obser-vations) (Ausreisser), then one has to apply a morerobustestimator.