Chapter 2. Order Statistics - 國立臺灣大學

1 1 Chapter 2. Order Statistics1 The Order StatisticsFor a sample of independent observationsX1, X2, .. , Xnon a distributionF, the orderedsample valuesX(1) X(2) X(n),or, in more explicit notation,X(1:n) X(2:n) X(n:n),are called the Order Statistics . IfFis continuous, then with probability 1 the Order statisticsof the sample take distinct values (and conversely).There is an alternative way to visualize Order Statistics that, although it does notnecessarily yield simple expressions for the joint density, does allow simple derivation of manyimportant properties of Order Statistics . It can be called the quantile function function(orinverse distribution function, if you wish) is defined byF 1(y) = inf{x:F(x) y}.

2 (1)Now it is well known that ifUis a Uniform(0,1) random variable, thenF 1(U) has distri-bution functionF. Moreover, if we envisionU1, .. , Unas being iid Uniform(0,1) randomvariables andX1, .. , Xnas being iid random variables with common distributionF, then(X(1), .. , X(n))d= (F 1(U(1)), .. , F 1(U(n))),(2)whered= is to be read as has the same distribution as. The Quantiles and Sample QuantilesLetFbe a distribution function (continuous from the right, as usual). The proof ofFisright continuous can be obtained from the following fact:F(x+hn) F(x) =P(x < X x+hn),where{hn}is a sequence of real numbers such that 0< hn 0 asn.

3 It follows fromthe continuity property of probabaility (P(limnAn) = limnP(An) if limAnexists.) thatlimn [F(x+hn) F(x)] = 0,and hence thatFis right-continuous. LetDbe the set of all discontinuity points ofFandnbe a positive integer. SetDn={x D:P(X=x) 1n}.2 SinceF( ) F( ) = 1, the number of elements inDncannot exceedn. ClearlyD= nDn,and it follows thatDis countable. Or, the set of discontinuity points of a distributionfunctionFis countable. We then conclude that every distribution functionFadmits thedecompositionF(x) = Fd(x) + (1 )Fc(x),(0 1),whereFdandFcare both continuous function such thatFdis a step function andFciscontinuous.

4 Moreover, the above decomposition is denote the Lebesgue measure onB, the -field of Borel sets inR. It follows fromthe Lebesgue decomposition theorem that we can writeFc(x) = Fs(x)+(1 )Fac(x) where0 1,Fsis singular with respect to , andFacis absolutely continuous with respect to . On the other hand, the Radon-Nikodym theorem implies that there exists a nonnegativeBorel-measurable function onRsuch thatFac(x) = x f d ,wherefis called the Radon-Nikodym derivative. This says that every distribution functionFadmits a unique decompositionF(x) = 1Fd(x) + 2Fs(x) + 3 Fac(x),(x R),where i 0 and 3i=1 i= 0< p <1, thepthquantileorfractileofFis defined as (p) =F 1(p) = inf{x:F(x) p}.

5 This definition is motivated by the following observation: IfFis continuous and strictly increasing,F 1is defined byF 1(y) =xwheny=F(x). IfFhas a discontinuity atx0, suppose thatF(x0 )< y < F(x0) =F(x0+). In thiscase, although there exists noxfor whichy=F(x),F 1(y) is defined to be equal tox0. Now consider the case thatFis not strictly increasing. Suppose thatF(x) < yforx < a=yfora x b> yforx > bThen any valuea x bcould be chosen forx=F 1(y). The convention in this caseis to defineF 1(y) = we prove that ifUis uniformly distributed over the interval (0,1), thenX=F 1X(U) has cumulative distribution functionFX(x).

6 The proof is straightforward:P(X x) =P[F 1X(U) x] =P[U FX(x)] =FX(x).Note that discontinuities ofFbecome converted into flat stretches ofF 1and flat stretchesofFinto discontinuities ofF particular, 1/2=F 1(1/2) is called themedianofF. Note that psatisfiesF( (p) ) p F( (p)).The functionF 1(t), 0< t <1, is called theinversefunction ofF. The followingproposition, giving useful properties ofFandF 1, is easily 1 LetFbe a distribution function. The functionF 1(t),0< t <1, is nondecreasingand left-continuous, and satisfies(i)F 1(F(x)) x, < x < ,(ii)F(F 1(t)) t,0< t < (iii)F(x) tif and only ifx F 1(t).

7 Corresponding to a sample{X1, X2, .. , Xn}of observations onF, the samplepthquantileis defined as thepth quantile of the sample distribution functionFn, that is, asF 1n(p). Regarding the samplepth quantile as an estimator of p, we denote it by pn, orsimply by pwhen the Order stastistics is equivalent to the sample distribution functionFn, its roleis fundamental even if not always explicit. Thus, for example, the sample mean may beregarded as the mean of the Order Statistics , and the samplepth quantile may be expressedas pn= Xn,npifnpis an integerXn,[np]+1ifnpis not an Functions of Order StatisticsHere we consider Statistics which may be expressed asfunctionsof Order Statistics .

8 A varietyof short-cut procedures for quick estimates of location or scale parameters, or for quick testsof related hypotheses, are provided in the form oflinearfunctions of Order Statistics , that isstatistics of the formn i=1cniX(i:n).4We term such Statistics L-estimates. For example, thesample rangeX(n:n) X(1:n)belongsto this class. Another example is given by the -trimmed 2[n ]n [n ] i=[n ]+1X(i:n),which is a popular competitor of Xforrobustestimation of location. The asymptotic dis-tribution theory of L- Statistics takes quite different forms, depending on the character of thecoefficients{cni}.

9 The representations of Xand pnin terms of Order Statistics are a bit artificial. Onthe other hand, for many useful Statistics , the most natural and efficient representations arein terms of Order Statistics . Examples are theextreme valuesX1:nandXn:nand thesamplerangeXn:n X1 General PropertiesTheorem 1 (1)P(X(k) x) = ni=kC(n, i)[F(x)]i[1 F(x)]n ifor < x < .(2)The density ofX(k)is given bynC(n 1, k 1)Fk 1(x)[1 F(x)]n kf(x).(3)The joint density ofX(k1)andX(k2)is given byn!(k1 1)!(k2 k1 1)!(n k2)![F(x(k1))]k1 1[F(x(k2)) F(x(k1))]k2 k1 1[1 F(x(k2))]n k2f(x(k1))f(x(k2))fork1< k2andx(k1)< x(k2).(4)The joint pdf of all the Order Statistics isn!

10 F(z1)f(z2) f(zn)for < z1< <zn< .(5)DefineV=F(X). ThenVis uniformly distributed over(0,1).Proof. (1) The event{X(k) x}occurs if and only if at leastkout ofX1, X2, .. , Xnareless than or equal tox.(2) The density ofX(k)is given bynC(n 1, k 1)Fk 1(x)[1 F(x)]n kf(x). It can beshown by the fact thatddpn i=kC(n, i)pi(1 p)n i=nC(n 1, k 1)pk 1(1 p)n ,k 1 smallest observations are xandn klargest are> (k)falls into asmall interval of lengthdxaboutxisf(x) Conditional Distribution of Order StatisticsIn the following two theorems, we relate the conditional distribution of Order Statistics (con-ditioned on another Order statistic) to the distribution of Order Statistics from a populationwhose distribution is a truncated form of the original population distribution functionF(x).