catalog - stats.org.uk

1 ACatalogofNoninformativePriors RuoyongYangParexelInternational RevereDrive Suite Northbrook IL ruoyong yang parexel comJamesO BergerISDSDukeUniversityDuahrm NC berger stat duke eduDecember Abstract PRELIMINARYDRAFT Thisdraftofthecatalogisincomplete withmuchremainingtobe lledinand oradded Wearecirculatingthisdraftinthehopesthatr eaderswillknowofrelevantinformationthats houldbeadded Pleasesendsuchinformationtotheauthorsabo ve Avarietyofmethodsofderivingnoninformativ epriorshavebeendeveloped andappliedtoawidevarietyofstatisticalmod els Inthispaperweprovideacatalogofmanyofther esultingpriors andlistknownpropertiesofthepriors EmphasisisgiventoreferencepriorsandtheJe reysprior althoughotherapproachesarealsoconsidered Keywordsandphrases Je reysprior referenceprior maximaldatainforma tionprior Contents Introduction Motivation ApproachestoDevelopmentofNoninformativeP riors ThisresearchwassupportedbyNSFgrantsDMS andDMS atPurdueUniversity OrganizationandNotation AR Behrens FisherProblem Beta Binomial BivariateBinomial Box CoxPowerTransformedLinearModel Cauchy Dirichlet ExponentialRegressionModel FDistribution Gamma GeneralizedLinearModel InverseGamma InverseNormalorGaussian LinearCalibration Location ScaleParameterModels LogitModel MixedModel

2 MixtureModel Multinomial NegativeBinomial NeymanandScottExample NonlinearRegressionModel Normal Pareto Poisson ProductofNormalMeans RandomE ectsModels RatioofExponentialMeans RatioofNormalMeans RatioofPoissonMeans SequentialAnalysis Stress StrengthSystem SumofSquaresofNormalMeans TDistribution Uniform Weibull Introduction MotivationTheliteratureonnoninformativep riorshasgrownenormouslyoverrecentyears Therehavebeenseveralexcellentbooksorrevi ewarticlesthathavebeenconcernedwithdiscu ssingorcomparingdi erentapproachestodevelopingnoninformativ epriors e g KassandWasserman buttherehasbeennosystematice orttocatalogthenoninformativepriorsthath avebeendeveloped Sinceuseofnoninformativepriorsisbecoming routineinBayesianpractice preparationofsuchacatalogseemedinorder Althoughgeneraldiscussionisnotthepurpose ofthiscatalog itisusefultoreviewthenumerousreasonsthat noninformativepriorsareimportanttoBayesi ananalysis i Frequently elicitationofsubjectivepriordistribution sisimpossible becauseoftimeorcostlimitations orresistanceorlackoftrainingofclients Automaticordefaultpriordistribu tionsarethenneeded ii Thestatisticalanalysisisoftenrequiredtoa ppearobjective Ofcourse trueobjectivityisvirtuallyneverattainabl e

3 Andthepriordistributionisusuallytheleast oftheproblemsintermsofobjectivity butuseofasubjectivelyelicitedpriorsigni cantlyreducestheappearanceofobjectivity Noninformativepriorsnotonlypreservethisa ppearance butcanbearguedtoresultinanalysesthatarem oreobjectivethanmostclassicalanalyses iii Subjectiveelicitationcaneasilyresultinpo orpriordistributions becauseofsystematicelicitationbiasandthe factthatelicitationtypicallyyieldsonlyaf ewfeaturesoftheprior withtherestoftheprior e g itsfunctionalform beingchoseninaconvenient butpossiblyinappropriate way Itisthusgoodpracticetocompareanswersfrom asubjectiveanalysiswithanswersfromanonin formativeprioranalysis Iftherearesubstantialdi erences itisimportanttocheckthatthedi erencesareduetofeaturesofthepriorthatare trusted andnotduetoeitherunelicited convenience featuresoftheprior orsuspectelicitations iv Inhighdimensionalproblems thebestonecantypicallyhopeforistodevelop subjectivepriorsforthe important parameters withtheunimportantor nuisance parametersbeinggivennoninformativepriors v Goodnoninformativepriorscanbesomewhatmag icalinmultiparameterproblems As anexample theJe reyspriorseemstoalmostalwaysyieldaproper posteriordistribution

4 Thisis magical inthatthecommonconstant oruniform priorwillmuchmorefrequentlyfailtoyieldap roperposterior Evenbetter thereferencepriorapproachhasrepeatedlyyi eldedmultiparameterpriorsthatovercomelim itationsoftheJe reysprior andyieldsurprisinglygoodperformancefroma lmostanyperspective Thepointhereisthat inmultiparameterproblems inappropriateaspectsofpriors evenproperones canaccumulateacrossdimensionsinverydetri mentalways referencepriorsseemto magically avoidsuchinappropriateaccumulation vi Bayesiananalysiswithnoninformativepriors isbeingincreasinglyrecognizedasamethodfo rclassicalstatisticianstoobtaingoodclass icalprocedures Forinstance thefrequentist matchingapproachtodevelopingnoninformati vepriorsisbasedonensuringthatonehasBayes iancrediblesetswithgoodfrequentistproper ties anditturnsoutthatthisisproba blythebestwayto ndgoodfrequentistcon dencesets ApproachestoDevelopmentofNoninformativeP riorsWedonotattemptathoroughdiscussionof thevariousapproaches See e g KassandWasserman forsuchdiscussion Weprimarilywilljustde nethevariousapproaches andgiverelevantreferences TheUniformPrior Bythis wejustmeantheconstantdensity withtheconstanttypicallybeingchosentobe unlesstheconstantcanbechosentoyieldaprop erdensity Thischoicewas ofcourse popularizedbyLaplace

5 TheJe reysPrior Thisisde nedas pdet I whereI istheFisherinfor mationmatrix ThiswasproposedinJe reys asasolutiontotheproblemthattheuniformpri ordoesnotyieldananalysisinvarianttochoic eofparameterization Notethat inspeci csituations Je reysoftenrecommendednoninformativepriors thatdi eredfromtheformalJe reysprior TheReferencePrior ThisapproachwasdevelopedinBernardo andmodi edformultiparameterproblemsinBergerandBe rnardo c Theapproachcannotbesimplydescribed butitcanberoughlythoughtofastryingtomodi fytheJe reyspriorbyreducingthedependenceamongpar ametersthatisfrequentlyinducedbytheJe reysprior therearemanywell knownexamplesinwhichtheJe reysprioryieldspoorperformance eveninconsistency becauseofthisdependence TheMaximalDataInformationPrior MDIP ThisapproachwasdevelopedinZellner basedonaninformationargument Itisgivenby expfRp xj logp xj dxg wherep xj isthedatadensityfunction OrganizationandNotationThecatalogisorgan izedaroundstatisticalmodels withthemodelsbeinglistedinalpha beticalorder Eachmodel entryiskeptasself containedaspossible Listedforeachare i themodeldensity ii variousnoninformativepriors and iii certainoftheresultingposteriorsandtheirp roperties Category iii informationisoftenverylimited Notationisstandard Thisinclude jD

6 JAj determinantofA isadensitywithrespecttod ImportantNotation Noninformativepriorsthatareproper i e integrateto aregiveninboldtype Othersareimproper Thedistinctionisimportantfortestingprobl ems whereproperdistributionsaretypicallyneed ed forestimationandprediction impropernoninformativepriorsaretypically ne AR TheAR model inwhichthedataX X XT followthemodelXt Xt t wherethe tarei i d N Theexpressionsbelowarefor known If isunknown multiplyby d or d Prior Marginal PosteriorUniform Je reyshT T fE X gi Reference expf E log PTi X i gallareproperReference p ifj j j jp ifj j Nonasymptoticreferenceprior Symmetrizedreferenceprior recommendedfortypicaluse SeeBergerandYang forcomparisonofthenoninformativepriors Behrens FisherProblemLetx xnbei i d observationsfromN andy ynbei i d observationsfromN theparametersofinterestare and Liseo computedtheJe reyspriorandreferencepriorforthisproblem asPrior Marginal PosteriorUniform Je reys properReference Independentofthegrouporderingoftheparame ters BetaTheBe densityisf xj x x I x TheFisherinformationmatrixisI B PG PG PG PG PG PG CA wherePG x P i x i isthePolyGammafunction TheJe reysprioristhusthesquarerootoftheFisheri nformationmatrix BinomialTheB n p p densityisf xjn p B nx CApx p n x Case Priorsforp givennPrior

7 Pjn Marginal PosteriorUniform Be pjx n x Je reys p p Be pjx n x ReferenceMDIP pp p p properNovickandHall s p p Be pjx n x SeeNovickandHall Notethispriorisuniformin logp p Case PriorsfornPrior n Marginal PosteriorUniform Je reysn Reference nUniversal nlog n loglog n loglog log n DiscussedbyAlbaandMendoza Theproductinthedenominatorisuptothelastt ermforwhichloglog log n SeeRissanen BivariateBinomialf r sjp q m B mr CApr p m r B rs CAqs q r s forr mands r PolsonandWasserman computetheFisherinformationmatrixofthisd istributionasI p q mdiag fp p g pfq q g Prior p qjm Marginal PosteriorUniform Je reys p q q properReference p p q q Reference p q q pq CrowderandSweeting s p p q q Parameterofinterestisporq Parameterofinterestis pqor p q pq SeeCrowderandSweeting Box CoxPowerTransformedLinearModelGivenobser vationsfy yng themodelisz y i lnyi xti i where isa k vectorofregressioncoe cients xiisavectorofcovariates and i N truncatedat xti Je reyspriorwasobtainedbyPericchi p k wherep issomeunspeci edpriorfor BoxandCox proposedtheprior g k wheregisthegeometricmeanofthey s Basedonthesocalleddata translatedparameterization z xti i ln xti i correspondingto orln Wixley proposedthefollowingtwopriors g k p wheregisthegeometricmeanofthey sandp issomeunspeci edpriorfor

8 ThispriorisapartfromthepriorofBoxandCox onlybyafactorp ItisalsothepriorusedinBoxandTiao k p k p wheregisthegeometricmeanofthey sandp issomeunspeci edpriorfor Thispriorwillgiveaverycloseresultantpost eriordistributiontothatofBoxandCox CauchyTheC densityisf xj x Thisisalocation scaleparameterproblem seethatsectionforpriors PosterioranalysiscanbefoundinSpiegelhalt er andHowladerandWeiss DirichletTheD density wherePki xi xi and k t i foralli isgivenbyf xj Qki i kYi x i i where Pki i TheFisherinformationmatrixisI k BBBB PG PG PG PG PG k PG CCCCA wherePG x P i x i isthePolyGammafunction TheJe reyspriorisjI k j ExponentialRegressionModelSeeYeandBerger Yij N x xia where R x a xandaknownconstants i k j m thexi sareknownnonnegativeregressorswithxi xjfori jandthevariance isanunknownconstant Itisassumedthatxi xjfori j Prior Marginal PosteriorUniform