Random Variables and Probability Distributions - Springer

1 ARandom Variablesand Probability Distribution Functions and Random The Multivariate Normal Distribution Functions and ExpectationThe distribution functionFof a Random variableXis defined byF(x)=P[X x]( )for all realx. The following properties are direct consequences of ( ) nondecreasing, ,F(x) F(y)ifx right continuous, ,F(y) F(x)asy (x) 1andF(y) 0asx andy , , any function that satisfies properties 1 3 is the distribution function ofsome Random of the commonly encountered distribution functionsFcan be expressedeither asF(x)= x f(y)dy( )orF(x)= j:xj xp(xj),( )where{x0,x1,x2.}

2 }is a finite or countably infinite set. In the case ( )weshallsay that the Random variableXiscontinuous. The functionfis called theprobabilitydensity function(pdf) ofXand can be found from the relationf(x)=F (x). Springer International Publishing Switzerland Brockwell, Davis,Introduction to Time Series and Forecasting, Springer Texts in Statistics, DOI A Random Variables and Probability DistributionsIn case ( ), the possible values ofXare restricted to the set{x0,x1,..},andweshall say that the Random variableXisdiscrete.

3 The functionpis called theprobabilitymass function(pmf) ofX,andFis constant except for upward jumps of sizep(xj)atthe pointsxj. Thusp(xj)is the size of the jump inFatxj, ,p(xj)=F(xj) F(x j)=P[X=xj],whereF(x j)=limy xjF(y). Examples of Continuous Distributions (a)The normal distribution with mean and variance say that a randomvariableXhas the normal distribution with mean and variance 2 written moreconcisely asX N , 2 ifXhas the pdf given byn x; , 2 =(2 ) 1/2 1e (x )2/(2 2) <x< .It follows then thatZ=(X )/ N(0,1)and thatP[X x]=P Z x = x ,where (x)= x (2 ) 1/2e 12z2dzis known as thestandard normal distribu-tion function.

4 The significance of the termsmeanandvariancefor the parameters and 2is explained below (see ).(b)The uniform distribution on[a,b].The pdf of a Random variable uniformly dis-tributed on the interval[a,b]is given byu(x;a,b)= 1b a,ifa x b,0,otherwise.(c)The exponential distribution with parameter .The pdf of an exponentially dis-tributed Random variable with parameter >0ise(x; )= 0,ifx<0, e x,ifx corresponding distribution function isF(x)= 0,ifx<0,1 e x,ifx 0.(d)The gamma distribution with parameters and .The pdf of a gamma-distributedrandom variable isg(x; , )= 0,ifx<0,x 1 e x/ ( ),ifx 0,where the parameters and are both positive and is the gamma functiondefined as ( )= 0x 1e Distribution Functions and Expectation355 Note thatfis the exponential pdf when =1and that when is a positive integer ( )=( 1)!

5 With0!defined to be1.(e)The chi-squared distribution with degrees of each positive integer , the chi-squared distribution with degrees of freedom is defined to be thedistribution of the sumX=Z21+ +Z2 ,whereZ1,..,Z are independent normally distributed Random Variables withmean 0 and variance 1. This distribution is the same as the gamma distributionwith parameters = /2and = Examples of Discrete Distributions (f)The binomial distribution with parameters n and pmf of a binomiallydistributed Random variableXwith parametersnandpisb(j;n,p)=P[X=j]= nj pj(1 p)n j,j=0,1.

6 ,n,wherenis a positive integer and0 p 1.(g)The uniform distribution on{1,2,..,k}.The pmf of a Random variableXuni-formly distributed on{1,2,..,k}isp(j)=P[X=j]=1k,j=1, ,k,wherekis a positive integer.(h)The Poisson distribution with parameter .A Random variableXissaidtohaveaPoisson distribution with parameter >0ifp(j; )=P[X=j]= jj!e ,j=0,1,..We shall see in that is the mean ofX.(i)The negative binomial distribution with parameters and Random variableXis said to have a negative binomial distribution with parameters >0andp [0,1]if it has pmfnb(j; ,p)= j k=1k 1+ k (1 p)jp ,j=0,1.

7 ,where the product is defined to be 1 ifj= all Random Variables can be neatly categorized as either continuous or example, consider the time you spend waiting to be served at a checkout counterand suppose that the Probability of finding no customers ahead of you is12. Then thetime you spend waiting for service can be expressed asW= 0,with probability12,W1,with probability12,356 Appendix A Random Variables and Probability DistributionswhereW1is a continuous Random variable. If the distribution ofW1is exponential withparameter 1, then the distribution function ofWisF(x)= 0,ifx<0,12+12 1 e x =1 12e x,ifx distribution function is neither continuous (since it has a discontinuity atx=0)nor discrete (since it increases continuously forx>0).

8 It is expressible as amixture,F=pFd+(1 p)Fc,withp=12, of a discrete distribution functionFd= 0,x<0,1,x 0,and a continuous distribution functionFc= 0,x<0,1 e x,x distribution function can in fact be expressed in the formF=p1Fd+p2Fc+p3 Fsc,where0 p1,p2,p3 1,p1+p2+p3=1,Fdis discrete,Fcis continuous, andFscissingular continuous(continuous but not of the ). Distribution functionswith a singular continuous component are rarely Expectation, Mean, and VarianceTheexpectationof a functiongof a Random variableXis defined byE(g(X))= g(x)dF(x),where g(x)dF(x):= g(x)f(x)dxin the continuous case, j=0g(xj)p(xj)in the discrete case,andgis any function such thatE|g(x)|<.

9 (IfFis the mixtureF=pFc+(1 p)Fd,thenE(g(X))=p g(x)dFc(x)+(1 p) g(x)dFd(x).) ThemeanandvarianceofXare defined as =EXand 2=E(X )2, respectively. They are evaluated bysettingg(x)=xandg(x)=(x )2in the definition ofE(g(X)).It is clear from the definition that expectation has thelinearity propertyE(aX+b)=aE(X)+bfor any real constantsaandb(provided thatE|X|< ). Random Vectors357 Example Normal DistributionIfXhas the normal distribution with pdfn x; , 2 as defined in Example (a) above,thenE(X )= (x )n x; , 2 dx= 2 n x: , 2 dx= shows, with the help of the linearity property ofE,thatE(X)= , , that the parameter isin fact the mean of the normal distribution defined inExample (a).

10 Similarly,E(X )2= (x )2n x; , 2 dx= 2 (x )n x; , 2 by parts and using the fact thatfis a pdf, we find that the variance ofXisE(X )2= 2 n x; , 2 dx= 2. Example Poisson DistributionThe mean of the Poisson distribution with parameter (see Example (h) above) isgiven by = j=0j jj!e = j=1 j 1(j 1)!e = e e = .A similar calculation shows that the variance is also equal to (see Problem ). and parameters associated with a Random variableXwill belabeled with the subscriptXwhenever it is necessary to identify the particular randomvariable to which they refer.