11 | TRANSFORMING DENSITY FUNCTIONS

1 11 | TRANSFORMINGDENSITYFUNCTIONSIt canbe expedient to useatransformationfunctionto transformoneprobability densityfunctioninto thistopic,it is helpfulto recapitulatethemethod of integrationby substitutionof a Substitutionofa newVariableImaginethata newcomerto integrationcomesacrossthefollowing:Zp 202xcosx2dxAssumingthatthenewcomerdoesn' tnoticethattheintegrandis thederivative of sinx2,oneway to proceedwouldbe to substitutea newvariableyforx2:Lety=x2 Replacethelimitsx= 0 andx=p 2byy= 0 andy= 2 Replace2xcosx2by 2pycosyNotethatx=pyandhencedxdy=12pyands o replacedxbydy2pyTheoriginalproblemis thereby transformedinto thefollowingintegration:Z 20cosy dy=hsinyi 20= 1 TheGeneralCaseIt is instructive to developthegeneralcasealongsidetheabove example:General CaseAboveExampleZbaf(x)dxZp 202xcosx2dxChoosea transformationfunctiony(x)y(x) =x2 Noteitsinversex(y)x(y) =pyReplacethelimitsbyy(a) andy(b)0 and 2 Replacef(x) byf x(y) 2pycosyReplacedxbydxdydy12pydyResultisZy (b)y(a)f x(y) dxdydyZ 20cosy dy{ {ApplicationtoProbability DENSITY FunctionsTheprevioussectioninformallylea dsto thegeneralformulaforintegrationby substitutionof a newvariable:Zbaf(x)dx=Zy(b)y(a)f x(y) dxdydy(11:1)Thisformulahasdirectapplicat iontotheprocessof transformingprobability densityfunctions: : :SupposeXis a randomvariablewhoseprobability DENSITY functionisf(x).}}

2 Byde nition:P(a6X < b) =Zbaf(x)dx(11:2)Any functionof a randomvariableis itselfa randomvariableand,ifyis takenas sometransformationfunction,y(X) willbe a (X).NoticethatifX=athederivedrandomvariableY=y(a) andifX=b,Y=y(b).Moreover,(subjectto certainassumptionsabouty) ifa6X < btheny(a)6Y < y(b)andP y(a)6Y < y(b) = P(a6X < b). Hence,by ( )and( ):P y(a)6Y < y(b) = P(a6X < b) =Zbaf(x)dx=Zy(b)y(a)f x(y) dxdydy(11:3)Noticethattheright-handintegrandf x(y) dxdyis expressedwhollyin (y):P y(a)6Y < y(b) =Zy(b)y(a)g(y)dyThisdemonstratesthatg(y) is theprobability DENSITY illustratedby thefollowing guresin which thefunctionf(x) (ontheleft)is transformedbyy(x) (centre)into thenewfunctiong(y) (right):XYf(x)y(b)g(y)y(a)ababy(a)y(b)xxy{ {ObservationsandConstraintsThecrucialstepis ( ).Oneimaginesnotinga sequenceof valuesof a randomvariableXandforeach valuein therangeatobusinga transformationfunctiony(x) to computea valuefora (x), thevalueofYmustbe in therangey(a) toy(b)andtheprobability ofYbeingin thisrangeis clearlythesameas theprobability ofXbeingin summary:theshadedregionin theright-hand gurehasthesameareaas theshadedregionin theleft-hand conditionsthatany probability DENSITY functionf(x) hastosatisfy: f(x) mustbe singlevaluedforallx f(x)>0 forallx Z+1 1f(x)dx= 1 Oftenthefunctionusefullyappliesover some niteinterval ofxandis usedin thespeci cationof aprobability DENSITY function:f(x) =(2xcosx2;if 06x <p 20.)}}

3 OtherwiseByinspection,f(x) is singlevaluedandnon-negative and,given ,theintegralfrom 1to +1is onthespeci cationof a probability DENSITY functionresultin implicitconstraints onany transformationfunctiony(x), mostimportantly: Throughouttheusefulrangeofx, bothy(x) anditsinversex(y) mustbe de nedandmustbe single-valued. Throughoutthisrange,dxdymustbe de nedandeitherdxdy> changesigntherewouldbe valuesofxforwhichy(x) wouldbe multivalued(aswouldbe thecaseif thegraphofy(x) wereanS-shapedcurve).A consequenceof theconstraints is thatany practicaltransformationfunctiony(x) musteitherincreasemonotonicallyover theusefulrangeofx(inwhich caseforanya < b,y(a)< y(b)) or decreasemonotonically(inwhich caseforanya < b,y(a)> y(b)).Notingtheseconstraints,it is customaryfortherelationshipbetweena probability densityfunctionf(x), theinversex(y) of a transformationfunction,andthederivedprob abilitydensity functiong(y) to be written:g(y) =f x(y) dxdy (11:4){ {ExampleITake a particularrandomvariableXwhoseprobabilit y DENSITY functionf(x) is:f(x) =8<:x2;if 06x <20;otherwiseSupposethetransformationfun ctiony(x) is:y(x) = 1 p4 x22 Notethattheusefulpartof therangeofxis 0 to 2 and,over thisrange,y(x)increasesmonotonicallyfrom 0 to (X), thederived randomvariable,andletg(y) be theprobability DENSITY functionassociatedwithY.}}

4 Whatis thefunctiong(y)?Theproblemis illustratedby thefollowing gures:XY22211g(y)f(x)y(x)0000202012xxyFi rst,derivex(y) theinverseof thefunctiony(x).Given:y= 1 p4 x224(y 1)2= 4 x2So:4y2 8y+ 4 = 4 x2x2= 4y(2 y)x= 2py(2 y){ {Accordingly:f x(y) =py(2 y)anddxdy=2(1 y)py(2 y)From( ):g(y) =f x(y) dxdy = 2(1 y)Asillustratedin the gures,thefunctiony(x) transformsonetriangulardistributionf(x)i ntoanotherg(y).Thetwo trianglesareoppositewaysroundandthetrans formationfunctiony(x) hasto ensurethatalthoughlow valuesofXarerelativelyrare,low :y(x) stays low formostof therangeofxso thateven whenxis wellover one,thevalueofyis wellundera QuestionIn theexample,a probability DENSITY functionanda transformationfunctionweregivenandthereq uirement was to determinewhatnewprobability DENSITY probability DENSITY functionsaregivenandtherequirement isto nda functionwhich transformsoneinto theparticularfunctionsusedin thepreviousexampleandposethequestionas :f(x) =8<:x2;if 06x <20;otherwiseandg(y) =(2(1 y);if 06y <10;otherwisedeterminethefunctiony(x) which willtransformf(x) intog(y).)}}

5 Fromtherelationshipg(y) =f x(y) dxdy :2(1 y) =x2dxdyor:xdxdy= 4(1 y)Thisdi erentialequationis readilysolvedandyields:x22= 4y 2y2+cSinceX= 0 hasto transformintoY= 0, theconstantc= 0.{ {Continuing:x2= 4(2y y2)Hencetheinversefunctionx(y) is:x(y) = 2py(2 y)A littlemoreprocessingis requiredto determiney(x):y2 2y+ 1 = 1 x24 Hence:(y 1)2= 1 x24 Thisleadsto:y= 1 r1 x24 Choiceof signis important. Note,again,thatX= 0 hasto transformintoY= 0 andhenceminus is thesolution:y(x) = 1 p4 x22 Transforminga UniformDistributionIt wouldbe unusualto wishto transforma triangulardistributionbutthereis a goodreasonforwantingto be ableto transforma uniformdistributioninto a uniformdistributionby computeris a well-understood processanda typicalprogramminglanguagewillbe suppliedwitha libraryprocedureto generatea randomvariablewhich is distributeddi erentlyis to is verycommonto startwitha distributionwhich is Uniform(0,1)which is to say thattheprobability DENSITY functionf(x) is:f(x) =(1;if 06x <10.)}}

6 OtherwiseOver theusefulrangeofx, therelationshipg(y) =f x(y) dxdy simpli esto:g(y) = dxdy (11:5){ {ExampleIITake a randomvariableXwhoseprobability DENSITY functionf(x) is Uniform(0,1)andsupposethatthetransformat ionfunctiony(x) is:y(x) = 1 lnx( >0)Notethattheusefulpartof therangeofxis 0 to 1 and,over thisrange,y(x)decreasesmonotonicallyfrom 1to (X) andletg(y) be theprobability DENSITY functionassociatedwithY. Whatis thefunctiong(y)?Theproblemis illustratedby thefollowing gures(inwhich = 2):XY112f(x)y(x)g(y)000010102xxyFirst,de rivex(y) theinverseof thefunctiony(x).Given:y= 1 lnxx=e yAccordingly:dxdy= :e yGiventhat >0 thisderivativedxdyis ( ):g(y) = dxdy = :e yAsillustratedin the gures,thefunctiony(x) transformsthedistributionf(x) which isUniform(0,1)intog(y) which is theexponentialdistribution.{ {ExampleIII |IntroductionSupposeraindropsfallin a uniformlydistributedway onto thesurfaceof a circularpondwhich a randomvariablewhosevaluexis thedistanceof a raindrop(shownatDinthe gure)fromthecentreof theprobability DENSITY functionf(x)associatedwithX?}}}}

7 Considera narrow annularconcentricstripof radiusxandwidth x. Theareaof thisstripis 2 x x. Theareaof thepondas a wholeis :P(x6X < x+ x) =2 x x :12 Theprobability DENSITY functionf(x) is therefore2xor,morestrictly:f(x) =(2x;if 06x <10;otherwiseNote,as a check, thatf(x) is singlevaluedandnon-negative anditsintegralfrom 1to+1is anothertriangulardistributionandleadstot heunsurprisingresultthatmoreraindropsfal lcloseto theedgeof thepondthanfallcloseto |TransformationThevalueof therandomvariableXdescribed in theprevioussectioncorrespondedto thedistanceof a randomraindropfromthecentreof interestedin thesquareof thedistancefromthecentreof thepondandhow thisderivedvalueis investigatethis,take therandomvariableXandapplyto it thetransformationfunctiony(x) speci edas:y(x) =x2 Notethattheusefulpartof therangeofxis 0 to 1 and,over thisrange,y(x) increasesmonotonicallyfrom0 to (X) andletg(y) be theprobability DENSITY functionassociatedwithY.)

8 Whatis thefunctiong(y)?{ {Theproblemis illustratedby thefollowing gures:XY211f(x)y(x)g(y)0000120101xxyFirs t,derivex(y) theinverseof thefunctiony(x):x(y) =pyAccordingly:f x(y) = 2pyanddxdy=12pyFrom( ):g(y) =f x(y) dxdy = 1 Asillustratedin the gures,thefunctiony(x) transformsthetriangulardistributionf(x)i nto thedistributiong(y) which is Uniform(0,1).Transforminga UniformDistributionintoa NormalDistributionIt wouldbe veryusefulif therewereaneasyway of transforminga uniformdistributioninto a a randomvariablewhosedistributionis Uniform(0,1)andYis a randomvariablewhosedistributionis Normal(0,1).Theassociatedprobability DENSITY FUNCTIONS (f(x) andg(y) respectively)are:f(x) =(1;if 06x <10;otherwiseandg(y) =1p2 e 12y2 Thegoalis to determinea functiony(x) which willtransformf(x) intog(y). Giventhatf(x) is Uniform(0,1),relationship( )above leadsto thedi erentialequation:dxdy=1p2 e 12y2(11:6)Unfortunatelythisdi erentialequationis intractable.)}}

9 { {GlossaryThefollowingtechnicaltermhasbee nintroduced:transformationfunctionExerci ses|XI1. Although( )cannotbe solvedanalytically, it (x) is incorporatedinto Excelas thebuilt-infunctionNORMSINV. Itsuseis illustratedin theExcelworksheetwhich is worksheetlike :(a) Firstsetupa columnof stepsof rst51of thesevaluesappearonthefacingpage(thework sheetrunsto a secondpagewhich is notshown).Headthiscolumnxas shown.(b) Setupa secondcolumnheadedy(x).Each valueis theresultof applyingthefunctionNORMSINVto thecorrespondingvalueofx. valuesofxbecauseandy(0)= 1andy(1)= +1. Therangeof theUniformdistributionis 0 to 1 andthismapsinto therangeof theNormaldistributionwhich is 1to +1.(c) Usethechartwizardtosetuptheplotof thetransformationfunction:y(x)againstxov er therangeof athowrapidlythefunctionapproaches 1and+1asxtendsto zeroor one.(d) SetupthecolumnheadedRange, whose12 theHistogramtool in Excel: : :(e) Check theDataAnalysiscommandis notthere,choosetheAdd-Inscommandand,viat hat,pick choosetheDataAnalysiscommandand,viathat, (x) as theInputRangeandspecifytherangecontainin gthe12valuesundertheheadingRangeas ,as theOutputRangeitself,specifythesinglecel ltwo placesto theright of thecellwithRangein thetopleft-handcellof thetablewhich theHistogramtool shouldthenproducealongwiththelower chart.}}

10 (f) Tidyupthechartandaddcomments to a neatappearanceroughlyas guresin thecolumnheadedFrequencyshouldbe halfa row wouldthenmoreclearlyindicatethatit is thenumber of valuesfoundbetween 0:25 and+0 at thetopis thenumber of valuesfoundlessthan 2:75andthevalue0 at thebottomis thenumber of valuesfoundmore than+2:75 (hencethewordMore).Thenumbersagainstthex -axisof Uniformdistributionhasbeentransformedint o a Normaldistribution.{ {2. Replacethe99valuesin thecolumnheadedxby=RAND(). Thenewvalueswillbe distributedUniform(0,1)andthevaluesin they(x) columnwillcontinueto bedistributedNormal(0,1)buttheywillnolon gerbe chartandinvoke theHistogramtool resultswillnotbe quiteso convincingas its predecessorbutit shouldnotbe verydi Extendthetwo maincolumnsso thatinsteadof 99pairsof theHistogramtool (remember to extendtheInputRange). Reworktheoriginalworksheet(ontheprevious page)butreplacethetransformationfunction NORMSINVby 12lnxandinvoke theHistogramtool thattheresultsarein reasonableaccordancewithExampleII Giventheprobability DENSITY FUNCTIONS :f(x) =(1;if 06x <10;otherwiseandg(y) =8<:y2;if 06y <20;otherwisedeterminethefunctiony(x) which willtransformf(x) intog(y).)}}