11 | TRANSFORMING DENSITY FUNCTIONS

1 11 | TRANSFORMINGDENSITYFUNCTIONSIt canbe expedient to useatransformationfunctionto transformoneprobability densityfunctioninto thistopic,it is helpfulto recapitulatethemethod of integrationby substitutionof a Substitutionofa newVariableImaginethata newcomerto integrationcomesacrossthefollowing:Zp 202xcosx2dxAssumingthatthenewcomerdoesn' tnoticethattheintegrandis thederivative of sinx2,oneway to proceedwouldbe to substitutea newvariableyforx2:Lety=x2 Replacethelimitsx= 0 andx=p 2byy= 0 andy= 2 Replace2xcosx2by 2pycosyNotethatx=pyandhencedxdy=12pyands o replacedxbydy2pyTheoriginalproblemis thereby transformedinto thefollowingintegration:Z 20cosy dy=hsinyi 20= 1 TheGeneralCaseIt is instructive to developthegeneralcasealongsidetheabove example:General CaseAboveExampleZbaf(x)dxZp 202xcosx2dxChoosea transformationfunctiony(x)y(x) =x2 Noteitsinversex(y)x(y) =pyReplacethelimitsbyy(a) andy(b)0 and 2 Replacef(x) byf x(y) 2pycosyReplacedxbydxdydy12pydyResultisZy (b)y(a)f x(y) dxdydyZ 20cosy dy{ {ApplicationtoProbability DENSITY FunctionsTheprevioussectioninformallylea dsto thegeneralformulaforintegrationby substitutionof a newvariable:Zbaf(x)dx=Zy(b)y(a)f x(y) dxdydy(11:1)Thisformulahasdirectapplicat iontotheprocessof transformingprobability densityfunctions: : :SupposeXis a randomvariablewhoseprobability DENSITY functionisf(x).}}

2 Byde nition:P(a6X < b) =Zbaf(x)dx(11:2)Any functionof a randomvariableis itselfa randomvariableand,ifyis takenas sometransformationfunction,y(X) willbe a (X).NoticethatifX=athederivedrandomvaria bleY=y(a) andifX=b,Y=y(b).Moreover,(subjectto certainassumptionsabouty) ifa6X < btheny(a)6Y < y(b)andP y(a)6Y < y(b) = P(a6X < b). Hence,by ( )and( ):P y(a)6Y < y(b) = P(a6X < b) =Zbaf(x)dx=Zy(b)y(a)f x(y) dxdydy(11:3)Noticethattheright-handinteg randf x(y) dxdyis expressedwhollyin (y):P y(a)6Y < y(b) =Zy(b)y(a)g(y)dyThisdemonstratesthatg(y) is theprobability DENSITY illustratedby thefollowing guresin which thefunctionf(x) (ontheleft)is transformedbyy(x) (centre)into thenewfunctiong(y) (right):XYf(x)y(b)g(y)y(a)ababy(a)y(b)xx y{ {ObservationsandConstraintsThecrucialste pis ( ).}}

3 Oneimaginesnotinga sequenceof valuesof a randomvariableXandforeach valuein therangeatobusinga transformationfunctiony(x) to computea valuefora (x), thevalueofYmustbe in therangey(a) toy(b)andtheprobability ofYbeingin thisrangeis clearlythesameas theprobability ofXbeingin summary:theshadedregionin theright-hand gurehasthesameareaas theshadedregionin theleft-hand conditionsthatany probability DENSITY functionf(x) hastosatisfy: f(x) mustbe singlevaluedforallx f(x)>0 forallx Z+1 1f(x)dx= 1 Oftenthefunctionusefullyappliesover some niteinterval ofxandis usedin thespeci cationof aprobability DENSITY function:f(x) =(2xcosx2;if 06x <p 20;otherwiseByinspection,f(x) is singlevaluedandnon-negative and,given ,theintegralfrom 1to +1is onthespeci cationof a probability DENSITY functionresultin implicitconstraints onany transformationfunctiony(x), mostimportantly: Throughouttheusefulrangeofx, bothy(x) anditsinversex(y) mustbe de nedandmustbe single-valued.)

4 Throughoutthisrange,dxdymustbe de nedandeitherdxdy> changesigntherewouldbe valuesofxforwhichy(x) wouldbe multivalued(aswouldbe thecaseif thegraphofy(x) wereanS-shapedcurve).A consequenceof theconstraints is thatany practicaltransformationfunctiony(x) musteitherincreasemonotonicallyover theusefulrangeofx(inwhich caseforanya < b,y(a)< y(b)) or decreasemonotonically(inwhich caseforanya < b,y(a)> y(b)).Notingtheseconstraints,it is customaryfortherelationshipbetweena probability densityfunctionf(x), theinversex(y) of a transformationfunction,andthederivedprob abilitydensity functiong(y) to be written:g(y) =f x(y) dxdy (11:4){ {ExampleITake a particularrandomvariableXwhoseprobabilit y DENSITY functionf(x) is:f(x) =8<:x2;if 06x <20;otherwiseSupposethetransformationfun ctiony(x) is:y(x) = 1 p4 x22 Notethattheusefulpartof therangeofxis 0 to 2 and,over thisrange,y(x)increasesmonotonicallyfrom 0 to (X), thederived randomvariable,andletg(y) be theprobability DENSITY functionassociatedwithY.}}

5 Whatis thefunctiong(y)?Theproblemis illustratedby thefollowing gures:XY22211g(y)f(x)y(x)0000202012xxyFi rst,derivex(y) theinverseof thefunctiony(x).Given:y= 1 p4 x224(y 1)2= 4 x2So:4y2 8y+ 4 = 4 x2x2= 4y(2 y)x= 2py(2 y){ {Accordingly:f x(y) =py(2 y)anddxdy=2(1 y)py(2 y)From( ):g(y) =f x(y) dxdy = 2(1 y)Asillustratedin the gures,thefunctiony(x) transformsonetriangulardistributionf(x)i ntoanotherg(y).Thetwo trianglesareoppositewaysroundandthetrans formationfunctiony(x) hasto ensurethatalthoughlow valuesofXarerelativelyrare,low :y(x) stays low formostof therangeofxso thateven whenxis wellover one,thevalueofyis wellundera QuestionIn theexample,a probability DENSITY functionanda transformationfunctionweregivenandthereq uirement was to determinewhatnewprobability DENSITY probability DENSITY functionsaregivenandtherequirement isto nda functionwhich transformsoneinto theparticularfunctionsusedin thepreviousexampleandposethequestionas :f(x) =8<:x2;if 06x <20;otherwiseandg(y) =(2(1 y);if 06y <10;otherwisedeterminethefunctiony(x) which willtransformf(x) intog(y).)}}

6 Fromtherelationshipg(y) =f x(y) dxdy :2(1 y) =x2dxdyor:xdxdy= 4(1 y)Thisdi erentialequationis readilysolvedandyields:x22= 4y 2y2+cSinceX= 0 hasto transformintoY= 0, theconstantc= 0.{ {Continuing:x2= 4(2y y2)Hencetheinversefunctionx(y) is:x(y) = 2py(2 y)A littlemoreprocessingis requiredto determiney(x):y2 2y+ 1 = 1 x24 Hence:(y 1)2= 1 x24 Thisleadsto:y= 1 r1 x24 Choiceof signis important. Note,again,thatX= 0 hasto transformintoY= 0 andhenceminus is thesolution:y(x) = 1 p4 x22 Transforminga UniformDistributionIt wouldbe unusualto wishto transforma triangulardistributionbutthereis a goodreasonforwantingto be ableto transforma uniformdistributioninto a uniformdistributionby computeris a well-understood processanda typicalprogramminglanguagewillbe suppliedwitha libraryprocedureto generatea randomvariablewhich is distributeddi erentlyis to is verycommonto startwitha distributionwhich is Uniform(0,1)which is to say thattheprobability DENSITY functionf(x) is:f(x) =(1;if 06x <10.)}}

7 OtherwiseOver theusefulrangeofx, therelationshipg(y) =f x(y) dxdy simpli esto:g(y) = dxdy (11:5){ {ExampleIITake a randomvariableXwhoseprobability DENSITY functionf(x) is Uniform(0,1)andsupposethatthetransformat ionfunctiony(x) is:y(x) = 1 lnx( >0)Notethattheusefulpartof therangeofxis 0 to 1 and,over thisrange,y(x)decreasesmonotonicallyfrom 1to (X) andletg(y) be theprobability DENSITY functionassociatedwithY. Whatis thefunctiong(y)?Theproblemis illustratedby thefollowing gures(inwhich = 2):XY112f(x)y(x)g(y)000010102xxyFirst,de rivex(y) theinverseof thefunctiony(x).Given:y= 1 lnxx=e yAccordingly:dxdy= :e yGiventhat >0 thisderivativedxdyis ( ):g(y) = dxdy = :e yAsillustratedin the gures,thefunctiony(x) transformsthedistributionf(x) which isUniform(0,1)intog(y) which is theexponentialdistribution.}}

8 { {ExampleIII |IntroductionSupposeraindropsfallin a uniformlydistributedway onto thesurfaceof a circularpondwhich a randomvariablewhosevaluexis thedistanceof a raindrop(shownatDinthe gure)fromthecentreof theprobability DENSITY functionf(x)associatedwithX?Considera narrow annularconcentricstripof radiusxandwidth x. Theareaof thisstripis 2 x x. Theareaof thepondas a wholeis :P(x6X < x+ x) =2 x x :12 Theprobability DENSITY functionf(x) is therefore2xor,morestrictly:f(x) =(2x;if 06x <10;otherwiseNote,as a check, thatf(x) is singlevaluedandnon-negative anditsintegralfrom 1to+1is anothertriangulardistributionandleadstot heunsurprisingresultthatmoreraindropsfal lcloseto theedgeof thepondthanfallcloseto |TransformationThevalueof therandomvariableXdescribed in theprevioussectioncorrespondedto thedistanceof a randomraindropfromthecentreof interestedin thesquareof thedistancefromthecentreof thepondandhow thisderivedvalueis investigatethis,take therandomvariableXandapplyto it thetransformationfunctiony(x) speci edas.)}}

9 Y(x) =x2 Notethattheusefulpartof therangeofxis 0 to 1 and,over thisrange,y(x) increasesmonotonicallyfrom0 to (X) andletg(y) be theprobability DENSITY functionassociatedwithY. Whatis thefunctiong(y)?{ {Theproblemis illustratedby thefollowing gures:XY211f(x)y(x)g(y)0000120101xxyFirs t,derivex(y) theinverseof thefunctiony(x):x(y) =pyAccordingly:f x(y) = 2pyanddxdy=12pyFrom( ):g(y) =f x(y) dxdy = 1 Asillustratedin the gures,thefunctiony(x) transformsthetriangulardistributionf(x)i nto thedistributiong(y) which is Uniform(0,1).Transforminga UniformDistributionintoa NormalDistributionIt wouldbe veryusefulif therewereaneasyway of transforminga uniformdistributioninto a a randomvariablewhosedistributionis Uniform(0,1)andYis a randomvariablewhosedistributionis Normal(0,1).}}

10 Theassociatedprobability DENSITY FUNCTIONS (f(x) andg(y) respectively)are:f(x) =(1;if 06x <10;otherwiseandg(y) =1p2 e 12y2 Thegoalis to determinea functiony(x) which willtransformf(x) intog(y). Giventhatf(x) is Uniform(0,1),relationship( )above leadsto thedi erentialequation:dxdy=1p2 e 12y2(11:6)Unfortunatelythisdi erentialequationis intractable.{ {GlossaryThefollowingtechnicaltermhasbee nintroduced:transformationfunctionExerci ses|XI1. Although( )cannotbe solvedanalytically, it (x) is incorporatedinto Excelas thebuilt-infunctionNORMSINV. Itsuseis illustratedin theExcelworksheetwhich is worksheetlike :(a) Firstsetupa columnof stepsof rst51of thesevaluesappearonthefacingpage(thework sheetrunsto a secondpagewhich is notshown).)}}