Prediction Theory 1 Introduction - University of …

1 Prediction Theory1 IntroductionBest linear Unbiased Prediction (BLUP) was developed for animal breeding by Dr. CharlesRoy Henderson around 1949. The methods were first applied to genetic evaluation of dairy siresin the northeastern United States in 1970. BLUP is used in nearly all countries and all livestockspecies for genetic evaluation of individual : Prediction is the estimation of the realized value of a random variable (fromdata) that has been sampled from a population with a known variance-covariance General linear Mixed Equation of the Modely=Xb+Zu+ewhereyis anN 1 vector of observations,bis ap 1 vector of unknown constants,uis aq 1 vector of unknown effects of random variables,eis anN 1 vector of unknown residual effects,X,Zare known matrices of orderN pandN qrespectively, that relate elements ofbanduto elements elements ofbare considered to be fixed effects while the elements ofuare the randomeffects from populations of random effects with known variance-covariance structures.

2 Bothbandumay be partitioned into one or more factors depending on the Expectations and Covariance StructureThe expectations of the random variables areE(u) =01E(e) =0E(y) =E(Xb+Zu+e)=E(Xb) +E(Zu) +E(e)=XE(b) +ZE(u) +E(e)=Xb+Z0+0=Xband the variance-covariance structure is typically represented asV(ue)=(G 00 R),whereGandRare known, positive definite matrices. Consequently,V ar(y) =V ar(Xb+Zu+e)=V ar(Zu+e)=ZV ar(u)Z +V ar(e) +ZCov(u,e) +Cov(e,u)Z =ZGZ +R,andCov(y,u) =ZGCov(y,e) =RIfuis partitioned intosfactors asu =(u 1u 2 u s),thenV ar(u) =V ar = G11G12 G1sG 12G22 1sG 2s Gss .EachGijis assumed to be PredictorsThe problem is to predict the functionK b+M u,provided thatK bis an estimable Best PredictorThebest predictor, for any type of model, requires knowledge of the distribution of the randomvariables as well as the moments of that distribution.

3 Then, the best predictor is the conditionalmean of the predictor given the data vector, (K b+M u|y)which is unbiased and has the smallest mean squared error of all predictors (Cochran 1951).The computational form of the predictor depends on the distribution ofy. The computationalform could be linear or nonlinear. The wordbestmeans that the predictor has the smallestmean squared error of all predictors ofK b+M Best linear PredictorThe best predictor may be linear OR nonlinear. Nonlinear predictors are often difficult tomanipulate or to derive a feasible solution. The predictor could be restricted to class of linearfunctions ofy. Then, the distributional form ofydoes not need to be known, and only the first(means) and second (variances) moments ofymust be known.

4 If the first moment isXbandthe second moment isV ar(y) =V, then thebest linear predictorisE(K b+M u) =K b+C V 1(y Xb)whereC =Cov(K b+M u,y).Whenyhas a multivariate normal distribution, then the best linear predictor (BLP) is thesame as the best predictor (BP). The BLP has the smallest mean squared error of alllinearpredictors ofK b+M Best linear Unbiased PredictorIn general, the first moment ofy, namelyXb, is not known, butV, the second moment, iscommonly assumed to be known. Then predictors can be restricted further to those that arelinear and also unbiased. Thebest linear unbiased predictorisK b+C V 1(y X b)where b= (X V 1X) X V 1y,andCandVare as predictor is the same as the BLP except that bhas replacedbin the formula.

5 Notethat bis the GLS estimate ofb. Of alllinear, unbiasedpredictors, BLUP has the smallestmean squared error. However, ifyis not normally distributed, then nonlinear predictors ofK b+M ucould potentially exist that have smaller mean squared error than Derivation of Predictand and PredictorDEFINITION:The predictand is the function to be predicted, in this caseK b+M :The predictor is the function to predict thepredictand, a linear function ofy, y, for Requiring UnbiasednessEquate the expectations of the predictor and the predictand to determine what needs to be truein order for unbiasedness to hold. That is,E(L y) =L XbE(K b+M u) =K bthen to be unbiased for all possible vectorsb,L X=K orL X K = Variance of Prediction ErrorThe Prediction error is the difference between the predictor and the predictand.

6 The covariancematrix of the Prediction errors isV ar(K b+M u L y) =V ar(M u L y)=M V ar(u)M+L V ar(y)L M Cov(u,y)L L Cov(y,u)M=M GM+L VL M GZ L L ZGM=V ar(PE) Function to be MinimizedBecause the predictor is required to be unbiased, then the mean squared error is equivalentto the variance of Prediction error. Combine the variance of Prediction error with a LaGrangeMultiplier to force unbiasedness to obtain the matrixF, whereF=V ar(PE) + (L X K ) .Minimization of the diagonals ofFis achieved by differentiatingFwith respect to the un-knowns,Land , and equating the partial derivatives to null matrices. F L= 2VL 2 ZGM+X =0 F =X L K=0 Let =.5 , then the first derivative can be written asVL=ZGM X then solve forLasV 1VL=L=V 1 ZGM V 1X.

7 Substituting the above forLinto the second derivative, then we can solve for asX L K=0X (V 1 ZGM V 1X ) K=0X V 1X =X V 1 ZGM K = (X V 1X) (X V 1 ZGM K)Substituting this solution for into the equation forLgivesL =M GZ V 1+K (X V 1X) X V 1 M GZ V 1X(X V 1X) X V b= (X V 1X) X V 1y,then the predictor becomesL y=K b+M GZ V 1(y X b)which is the BLUP ofK b+M u, and bis a GLS solution forb. A special case for this predictorwould be to letK =0andM =I, then the predictand isK b+M u=u, andL y= u=GZ V 1(y X b).Hence the predictor ofK b+M uis(K M )times the predictor of(b u ) which is( b u ) .55 Variances of PredictorsLetP= (X V 1X) b=PX V 1ythen u=GZ V 1(y XPX V 1y)=GZ V 1 WyforW= (I XPX V 1).From the results on generalized inverses ofX,XPX V 1X=X,and therefore,WX= (I XPX V 1)X=X XPX V 1X=X X= variance of the predictor is,V ar( u) =GZ V 1W(V ar(y))W V 1ZG=GZ V 1 WVW V 1ZG=GZ V 1ZG GZ V 1 XPX V covariance between band uisCov( b, u) =PX V 1V ar(y)W V 1ZG=PX W V 1ZG=0becauseX W=0 Therefore, the total variance of the predictor isV ar(K b+M u) =K PK+M GZ V 1 ZGM M GZ V 1 XPX V Variance of Prediction ErrorThe main results areV ar( b b) =V ar( b) +V ar(b) Cov( b,b) Cov(b, b)=V ar( b)= ar( u u) =V ar( u) +V ar(u) Cov( u,u) Cov(u, u),6whereCov( u,u) =GZ V 1 WCov(y,u)=GZ V 1 WZG=GZ (V 1 V 1 XPX V 1)ZG=V ar( u)so thatV ar( u u) =V ar( u) +G 2V ar( u)=G V ar( u).

8 Also,Cov( b, u u) =Cov( b, u) Cov( b,u)=0 PX V Mixed Model EquationsThe covariance matrix ofyisVwhich is of usually too large to allowVtobe inverted. The BLUP predictor has the inverse ofVin the formula, and therefore, wouldnot be practical whenNis large. Henderson(1949) developed the mixed model equations forcomputing BLUP ofuand the GLS ofb. However, Henderson did not publish a proof of theseproperties until 1963 with the help of S. R. Searle, which was one year after Goldberger (1962).Take the first and second partial derivatives ofF,(V XX 0)(L )=(ZGMK)Recall thatV=V ar(y) =ZGZ +R,and letS=G(Z L M)which when re-arranged givesM=Z L G 1S,then the previous equations can be re-written as R X ZX 0 0Z 0 G 1 L S = 0KM.

9 Take the first row of these equations and solve forL, then substitute the solution forLintothe other two R 1X R 1ZS7and (X R 1X X R 1ZZ R 1X Z R 1Z+G 1)( S)=(KM).Let a solution to these equations be obtained by computing a generalized inverse of(X R 1X X R 1ZZ R 1X Z R 1Z+G 1)denoted as(CxxCxzCzxCzz),then the solutions are( S)= (CxxCxzCzxCzz)(KM).Therefore, the predictor isL y=(K M )(CxxCxzCzxCzz)(X R 1yZ R 1y)=(K M )( b u),where band uare solutions to(X R 1X X R 1ZZ R 1X Z R 1Z+G 1)( b u)=(X R 1yZ R 1y).The equations are known asHenderson s Mixed Model Equationsor MME. The equations areof order equal to the number of elements inbandu, which is usually much less than the numberof elements iny, and therefore, are more practical to solve.

10 Also, these equations require theinverse ofRrather thanV, both of which are of the same order, butRis usually diagonal orhas a more simple structure thanV. Also, the inverse ofGis needed, which is of order equalto the number of elements inu. The ability to compute the inverse ofGdepends on the modeland the definition MME are a useful computing algorithm for obtaining BLUP ofK b+M u. Please keepin mind that BLUP is a statistical procedure such that if the conditions for BLUP are met,then the predictor has the smallest mean squared error of all linear , unbiased predictors. Theconditions are that the model is the true model and the variance-covariance matrices of therandom variables are known without the strictest sense, all models approximate an unknown true model, and the variance-covariance parameters are usually guessed, so that there is never a truly BLUP analysis of data,except possibly in simulation Equivalence ProofsThe equivalence of the BLUP predictor to the solution from the MME was published by Hen-derson in 1963.