Prior distribution - Department of Statistics

1 Prior distributionAndrew GelmanVolume 3, pp 1634 1637inEncyclopedia of Environmetrics(ISBN 0471 899976)Edited byAbdel H. El-Shaarawi and Walter W. Piegorsch John Wiley & Sons, Ltd, Chichester, 2002 Prior distributionTheprior distributionis a key part of Bayesian infer-ence (seeBayesian methods and modeling) and rep-resents the information about an uncertain parameter that is combined with the probability distribution ofnew data to yield theposterior distribution ,whichin turn is used for future inferences and decisionsinvolving . The existence of a Prior distribution forany problem can be justified by axioms of decisiontheory; here we focus on how to set up a Prior distri-bution for any given application.

2 In general, can andwill be a vector, but for simplicity we will focus hereon Prior distributions for parameters one at a key issues in setting up a Prior distributionare: what information is going into the Prior distri-bution; the properties of the resulting posterior well-identified parameters and large samplesizes, reasonable choices of Prior distributions willhave minor effects on posterior inferences. This def-inition of well identified and large sample sizemight seem to be circular, but in practice one cancheck the dependence on Prior distributions by asensitivity analysis: comparing posterior inferencesunder different reasonable choices of Prior distribu-tion (and, for that matter, different reasonable choicesof probability models for data).

3 If the sample size is small, or available data pro-vide only indirect information about the parameters ofinterest, the Prior distribution becomes more impor-tant. In many cases, however, models can be set uphierarchically, so that clusters of parameters haveshared Prior distributions, which can themselves beestimated from illustrate with an example from a model in phar-macokinetics, the study of the absorption, distribu-tion, and elimination of drugs from the body. Forthis particular study, about 20 measurements wereavailable on six young adult males, and a modelwas fit with 15 parameters per person (which welabel klfor personkand parameterl), along withtwo variance parameters, 21and 22, indicating thescale of measurement/modeling error.

4 The data (con-centrations of a compound in blood and exhaled airover time) are only indirectly informative of the indi-vidual level parameters, which refer to equilibriumconcentrations, volumes, and metabolic rates insidethe is a nice example to use here because dif-ferent principles for assigning Prior distributions arerelevant for different parameters in the model, as wenow Prior DistributionsWe first consider the variance parameters 21and 22, which are actually quite well identified in theposterior distribution . For these, a noninformativeuniform Prior distribution works fine.

5 (A uniform dis-tribution on the log standard deviations was used, butenough information was available from the data thatthe choice of noninformative Prior distribution wasessentially irrelevant, and one could just as well haveassigned a uniform Prior distribution on the variancesor the standard deviations.) The uniform Prior distri-bution here isimproper that is, the function usedas a Prior probability density has an infinite integraland is thus not, strictly speaking, a probability den-sity at all. However, when formally combined withthe data likelihood it yields an acceptable proper pos-terior Informative Prior DistributionsAt the other extreme, fairly precise scientific infor-mation is available on some of the parameters klinthe model.

6 For example, parameter 8 represents themass of the liver as a fraction of lean body mass;from previous medical studies, the liver is known tobe about of lean body mass for young adultmales, with little variation. The Prior distribution forlog k,8(for personskD1,..,6) is assumed nor-mal with mean 8and standard deviation 8; 8was given a normal Prior distribution with meanlog and standard deviation log ,and 8was given an inverse 2prior distribution with scalelog and two degrees of freedom. This setup setsthe parameters k,8approximately to their Prior esti-mate, , with some variation allowed distributionModerately Informative Hierarchical PriorDistributionsFinally, some of the physiological parameters klarenot well estimated by the data thus, they requireinformative Prior distributions but scientific infor-mation on them is limited.

7 For example, in thisparticular study, parameter 14 represents the maxi-mum rate of metabolism of a certain compound; thebest available estimate of this parameter for healthyhumans is , but this estimate is quite crudeand could easily be off by a factor of 10 or maximum rate of metabolism is not expectedto vary greatly between persons, but there is muchuncertainty about the numerical value of the param-eter. This information is encoded in ahierarchi-calprior distribution : log k,14 N 14, 214 , with 14 Nflog ,[log 10 ]2gand 14 Inv 2f2,[log 2 ]2g. Thus, the parameters k,14for the dif-ferent personskare expected to vary by about a factorof 2, with their overall level estimated at about ,with a multiplicative uncertainty of about a factor of10.

8 To express this subtle statement of Prior uncer-tainty, the hierarchical Prior distribution is Would Happen if Noninformative PriorDistributions Were Used for All the Parameters inthis Example?In our parameterization, noninformative Prior distri-butions on the parameters klcorrespond to setting kD1for each parameterk, thus allowing eachperson s parameters to be estimated from that per-son s own data. If noninformative Prior distributionswere assigned to all the individual parameters, thenthe model would fit the data very closely but withscientifically unreasonable parameters for example,a person with a liver weighing 10 k.

9 This sort of diffi-culty is what motivates a researcher to specify a priordistribution using external of the theoretical work on Prior distributions hasbeen on two topics: first, determining the conditionsthat must be satisfied by the Prior and data distribu-tions so that the posterior distribution is well behaved;and second, setting up rules for noninformative priordistributions that satisfy various invariance strands of research are related, in that Prior dis-tributions set up based on invariance rules alone willmake sense only if they lead to reasonable the most well known theoretical result isthat, for variance parameters in a linear regres-sion model, the uniform Prior distribution for log is acceptable when applied to the lowest level vari-ance component butnotacceptable for higher levelvariance components.

10 For these, a uniform Prior dis-tribution on is acceptable, in the sense that theposterior distribution will be proper if sample sizesare moderate or large. By not acceptable , we refer toprior distributions that yield improper posterior dis-tributions, even when the data really supply enoughinformation to estimate the parameter theoretical results do not giverecommendedmodels, but rather are useful inruling outcertain nat-ural seeming models with poor statistical , one might seek to avoid theoreticalconsiderations entirely and simply pick a subjective Prior distribution that best represents one s scientificknowledge about the set of uncertain parameters inthe problem .