Example: marketing

6 Finite Sample Theory of Order Statistics and Extremes

6 Finite Sample Theory of Order Statistics and Extremes The ordered values of a Sample of observations are called the Order Statistics of the Sample , and the smallest and the largest called the Extremes . Order Statistics and Extremes are among the most important functions of a set of random variables that we study in probability and Statistics . There is natural interest in studying the highs and lows of a sequence, and the other Order Statistics help in understanding concentration of probability in a distribution, or equivalently, the diversity in the population represented by the distribution. Order Statistics are also useful in statistical inference, where estimates of parameters are often based on some suitable functions of the Order Statistics . In particular, the median is of very special importance. There is a well developed Theory of the Order Statistics of a fixed number n of observations from a fixed distribution, as also an asymptotic Theory where n goes to infinity.

6 Finite Sample Theory of Order Statistics and Extremes The ordered values of a sample of observations are called the order statistics of the sample, and the

Tags:

  Statistics, Samples, Order, Theory, Extreme, Sample theory of order statistics and extremes

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of 6 Finite Sample Theory of Order Statistics and Extremes

1 6 Finite Sample Theory of Order Statistics and Extremes The ordered values of a Sample of observations are called the Order Statistics of the Sample , and the smallest and the largest called the Extremes . Order Statistics and Extremes are among the most important functions of a set of random variables that we study in probability and Statistics . There is natural interest in studying the highs and lows of a sequence, and the other Order Statistics help in understanding concentration of probability in a distribution, or equivalently, the diversity in the population represented by the distribution. Order Statistics are also useful in statistical inference, where estimates of parameters are often based on some suitable functions of the Order Statistics . In particular, the median is of very special importance. There is a well developed Theory of the Order Statistics of a fixed number n of observations from a fixed distribution, as also an asymptotic Theory where n goes to infinity.

2 We will discuss the case of fixed n in this chapter. Distribution Theory for Order Statistics when the observations are from a discrete distribution is complex, both notationally and algebraically, because of the fact that there could be several observations which are actually equal. These ties among the Sample values make the distribution Theory cumbersome. We therefore concentrate on the continuous case. Principal references for this chapter are the books by David (1980). Reiss (1989), Galambos (1987), Resnick (2007), and Leadbetter, Lindgren, and Rootz en (1983). Specific other references are given in the sections. Basic Distribution Theory Definition Let X1 .X2 , , Xn be any n real valued random variables. Let X(1) X(2) . X(n) denote the ordered values of X1 .X2 , , Xn . Then, X(1) , X(2) , , X(n) are called the Order Statistics of X1 .X2 , , Xn . Remark: Thus, the minimum among X1 .X2 , , Xn is the first Order statistic, and the max- imum the nth Order statistic.

3 The middle value among X1 .X2 , , Xn is called the median. But it needs to be defined precisely, because there is really no middle value when n is an even integer. Here is our definition. Definition Let X1 .X2 , , Xn be any n real valued random variables. Then, the median of X1 .X2 , , Xn is defined to be Mn = X(m+1) if n = 2m + 1 (an odd integer), and Mn = X(m) if n = 2m (an even integer). That is, in either case, the median is the Order statistic X(k) where k is the smallest integer n2 . Example Suppose .3, .53, .68, .06, .73, .48, .87, .42, .89, .44 are ten independent observations from the U [0, 1] distribution. Then, the Order Statistics are .06, .3, .42, .44, .48, .53, .68, .73, .87, .89. Thus, X(1) = .06, X(n) = .89, and since n2 = 5, Mn = X(5) = .48. An important connection to understand is the connection Order Statistics have with the empir- ical CDF, a function of immense theoretical and methodological importance in both probability and Statistics .

4 Definition Let X1 , X2 , , Xn be any n real valued random variables. The empirical CDF. of X1 .X2 , , Xn , also called the empirical CDF of the Sample , is the function # {Xi : Xi x}. Fn (x) = , n 189. , Fn (x) measures the proportion of Sample values that are x for a given x. Remark: Therefore, by its definition, Fn (x) = 0 whenever x < X(1) , and Fn (x) = 1 whenever x X(n) . It is also a constant , namely, nk , for all x-values in the interval [X(k) , X(k+1) ). So Fn satisfies all the properties of being a valid CDF. Indeed, it is the CDF of a discrete distribution, 1. which puts an equal probability of n at the Sample values X1 , X2 , , Xn . This calls for another definition. 1. Definition Let Pn denote the discrete distribution which assigns probability n to each Xi . Then, Pn is called the empirical measure of the Sample . Definition Let Qn (p) = Fn 1 (p) be the quantile function corresponding to Fn . Then, Qn =. Fn 1 is called the quantile function of X1 , X2 , , Xn , or the empirical quantile function.]

5 We can now relate the median and the Order Statistics to the quantile function Fn 1 . Proposition Let X1 , X2 , , Xn be n random variables. Then, i (a)X(i) = Fn 1 ( );. n 1. (b)Mn = Fn 1 ( ). 2. We now specialize to the case where X1 , X2 , , Xn are independent random variables with a common density function f (x) and CDF F (x), and work out the fundamental distribution Theory of the Order Statistics X(1) , X(2) , , X(n) . Theorem (Joint Density of All the Order Statistics ) Let X1 , X2 , , Xn be indepen- dent random variables with a common density function f (x). Then, the joint density function of X(1) , X(2) , , X(n) is given by f1,2, ,n (y1 , y2 , , yn ) = n!f (y1 )f (y2 ) f (yn )I{y1 <y2 < <yn } . Proof: A verbal heuristic argument is easy to understand. If X(1) = y1 , X(2) = y2 , , X(n) = yn , then exactly one of the Sample values X1 , X2 , , Xn is y1 , exactly one is y2 , etc., but we can put any of the n observations at y1 , any of the other n 1 observations at y2 , etc.

6 , and so the density of X(1) , X(2) , , X(n) is f (y1 )f (y2 ) f (yn ) n(n 1) 1 = n!f (y1 )f (y2 ) f (yn ), and obviously if the inequality y1 < y2 < < yn is not satisfied, then at such a point the joint density of X(1) , X(2) , , X(n) must be zero. Here is a formal proof. The multivariate transformation (X1 , X2 , , Xn ) (X(1) , X(2) , , X(n) ). is not one-to-one, as any permutation of a fixed (X1 , X2 , , Xn ) vector has exactly the same set of Order Statistics X(1) , X(2) , , X(n) . However, fix a specific permutation { (1), (2), , (n)}. of {1, 2, , n} and consider the subset A = {(x1 , x2 , , xn ) : x (1) < x (2) < < x (n) }. Then, the transformation (x1 , x2 , , xn ) (x(1) , x(2) , , x(n) ) is one-to-one on each such A , and indeed, then x(i) = x (i) , i = 1, 2, , n. The Jacobian matrix of the transformation is 1, for each such A . A particular vector (x1 , x2 , , xn ) falls in exactly one A , and there are n! such regions A , as we exhaust all the n!

7 Permutations { (1), (2), , (n)} of {1, 2, , n}. By a modification of the Jacobian density theorem, we then get X. f1,2, ,n (y1 , y2 , , yn ) = f (x1 )f (x2 ) f (xn ).. 190. X. = f (x (1) )f (x (2) ) f (x (n) ).. X. = f (y1 )f (y2 ) f (yn ).. = n!f (y1 )f (y2 ) f (yn ).. Example (Uniform Order Statistics ). Let U1 , U2 , , Un be independent U [0, 1] vari- ables, and U(i) , 1 i n, their Order Statistics . Then, by our theorem above, the joint density of U(1) , U(2) , , U(n) is f1,2, ,n (u1 , u2 , , un ) = n!I0<u1 <u2 < <un <1 . Once we know the joint density of all the Order Statistics , we can find the marginal density of any subset, by simply integrating out the rest of the coordiantes, but being extremely careful in using the correct domain over which to integrate the rest of the coordinates. For example, if we want the marginal density of just U(1) , that is of the Sample minimum, then we will want to integrate out u2 , , un , and the correct domain of integration would be, for a given u1 , a value of U(1) , in (0,1), u1 < u2 < u3 < < un < 1.

8 So, we will integrate down in the Order un , un 1 , , u2 , to obtain Z 1Z 1 Z 1. f1 (u1 ) = n! dun dun 1 du3 du2. u1 u2 un 1. = n(1 u1 )n 1 , 0 < u1 < 1. Likewise, if we want the marginal density of just U(n) , that is of the Sample maximum, then we will want to integrate out u1 , u2 , , un 1 , and now the answer will be Z un Z un 1 Z u2. fn (un ) = n! du1 du2 dun 1. 0 0 0. = nun 1. n ,0 < un < 1. However, it is useful to note that for the special case of the minimum and the maximum, we could have obtained the densities much more easily and directly. Here is why. First take the maximum. Consider its CDF; for 0 < u < 1: Y. n P (U(n) u) = P ( ni=1 {Xi u}) = P (Xi u). i=1. = un , d n and hence, the density of U(n) is fn (u) = du [u ] = nun 1 , 0 < u < 1. Likewise, for the minimum, for 0 < u < 1, the tail CDF is: P (U(1) > u) = P ( ni=1 {Xi > u}) = (1 u)n , and so the density of U(1) is d f1 (u) = [1 (1 u)n ] = n(1 u)n 1 , 0 < u < 1. du 191. Density of Minimum, Median, and Maximum of U[0,1] Variables; n = 15.

9 14. 12. 10. 8. 6. 4. 2. x 1. For a general r, 1 r n, the density of U(r) works out to a Beta density: n! fr (u) = ur 1 (1 u)n r , 0 < u < 1, (r 1)!(n r)! which is the Be(r, n r + 1) density. As a rule, if the underlying CDF F is symmetric about its median, then the Sample median will also have a density symmetric about the median of F ; see the Exercises. When n is even, one has to be careful about this, because there is no universal definition of Sample median when n is even. Additionally, the density of the Sample maximum will generally be skewed to the right, and that of the Sample minimum skewed to the left. For general CDFs, the density of the Order Statistics usually will not have a simple formula in terms of elementary functions; but approximations for large n are often possible. This will be treated in a later chapter. Although such approximations for large n are often available, they may not be very accurate unless n is very large; see Hall (1979).

10 We have plotted above the density of the minimum, median, and maximum in the U [0, 1] case when n = 15. The minimum and the maximum clearly have skewed densities, while the density of the median is symmetric about .5. More Advanced Distribution Theory Example (Density of One and Two Order Statistics ). The joint density of any subset of the Order Statistics X(1) , X(2) , , X(n) can be worked out from their joint density, which we derived in the preceding section. The most important case in applications is the joint density of two specific Order Statistics , say X(r) and X(s) , 1 r < s n, or the density of a specific one, say X(r) . A verbal heuristic argument helps in understanding the formula for the joint density of X(r). and X(s) , and also the density of a specific one X(r) . First consider the density of just X(r) . Fix u. To have X(r) = u, we must have exactly one observation at u, another r 1 below u, and n r above u. This will suggest that the density of 192.


Related search queries