Transcription of Posterior Predictive Distribution
1 Posterior Predictive DistributionIRecall that for a fixed value of , our dataXfollow thedistributionp(X| ).IHowever, the true value of is uncertain, so we shouldaverage over the possible values of to get a better idea ofthe Distribution the sample, the uncertainty in is representedby the prior distributionp( ). So for some new data valuexnew, averaging overp( ) gives theprior predictivedistribution:p(xnew) = p(xnew, ) d = p(xnew| )p( ) d Posterior Predictive DistributionIAftertaking the sample, we have abetter representationofthe uncertainty in via our posteriorp( |x). So theposteriorpredictive distributionfor a new data pointxnewis:p(xnew|x) = p(xnew| ,x)p( |x) d = p(xnew| )p( |x) d (sincexnewis independent of the sample datax)IThis reflects how we would predict new data to behave / the data wedid observefollow this pattern closely, itindicates we have chosen our model and prior Predictive DistributionExample 2 again:X1,..,Xniid Poisson( ), Gamma( , ) |x= Gamma( xi+ ,n+ ) Posterior Predictive Distribution is:p(xnew|x) = 0p(xnew| )p( |x) d = 0[ xnewe (xnew)!]
2 ][(n+ )Pxi+ ( xi+ ) Pxi+ 1e (n+ ) ]d Posterior Predictive DistributionSop(xnew|x) =(n+ )Pxi+ ( xi+ ) (xnew+ 1) 0 xnew+Pxi+ 1e (n+ +1) d =(n+ )Pxi+ ( xi+ ) (xnew+ 1) (xnew+ xi+ )(n+ + 1)xnew+Pxi+ = (xnew+ xi+ ) ( xi+ ) (xnew+ 1)(n+ n+ + 1)Pxi+ (1n+ + 1)xnewwhich is anegative binomial with meanPxi+ n+ and variancePxi+ (n+ )2(n+ + 1). Posterior Predictive DistributionI The Posterior Predictive Distribution has the same mean asthe Posterior Distribution , but agreatervariance(additional sampling uncertainty since we are drawing anewdatavalue).ISeeRexample (Prussian army data).More about Posterior Predictive DistributionIExample 1(a) again:X1,..,Xniid N( , 2), for |xisnormal with mean post= / 2+n x/ 21/ 2+n/ 2and variance 2post= 2 2 2+n | N( , 2), so the Posterior predictivedistribution is:p(xnew|x) = p(xnew| )p( |x) d .More about Posterior Predictive DistributionISometimes the form ofp(xnew|x) can be derived directly, butit is often easier to sample fromp(xnew|x) using Monte Carlomethods:IForj= 1.
3 ,J, sample1. [j]fromp( |x) [j]fromp(xnew| [j])IThenx [1],..,x [J]are an iid sample fromp(xnew|x).ISeeRexample with lead Predictive Distribution in RegressionExample 3: In the regression setting, we have shown that theposterior Predictive Distribution for a new response vectory check model fit, we can generate samples from theposterior Predictive Distribution (lettingX = the observedsampleX) and plot the values against they-values from theoriginal an observedyifalls far from the center of the posteriorpredictive Distribution , thisi-th observation is an this occurs for manyy-values, we would doubt theadequacy of the (small automobile data set).
