Gaussian Probability Density Functions: Properties and ...

1 Gaussian Probability Density Functions: Properties and Error CharacterizationMaria Isabel RibeiroInstitute for Systems and RoboticsInstituto Superior TcnicoAv. Rovisco Pais, 11049-001 Lisboa M. Isabel Ribeiro, 2004 February 2004 Contents1 Normal random variables22 Normal random for second order .. of constant Probability .. 143 Properties224 Covariance matrices and error ellipsoid241 chapter 1 Normal random variablesA random variableXis said to be normally distributed with mean and variance 2if its Probability Density function (pdf) isfX(x) =1 2 exp[ (x )22 2], < x < .( )Whenever there is no possible confusion between the random variableXand thereal argument,x, of the pdf this is simply represented byf(x)omitting the explicitreference to the random variableXin the subscript. The Normal or Gaussiandistribution ofXis usually represented by,X N( , 2),or also,X N(x , 2).The Normal or Gaussian pdf ( ) is a bell-shaped curve that is symmetric aboutthe mean and that attains its maximum value of1 2 ' atx= asrepresented in Figure for = 2and 2= Gaussian pdfN( , 2)is completely characterized by the two parameters and 2, the first and second order moments, respectively, obtainable from thepdf as =E[X] = xf(x)dx,( ) 2=E[(X )2] = (x )2f(x)dx( )2 6 4 (x)Figure : Gaussian or Normal pdf, N(2, )The mean, or the expected value of the variable, is the centroid of the pdf.

2 Inthis particular case of Gaussian pdf, the mean is also the point at which the pdf ismaximum. The variance 2is a measure of the dispersion of the random variablearound the fact that ( ) is completely characterized by two parameters, the first andsecond order moments of the pdf, renders its use very common in characterizingthe uncertainty in various domains of application. For example, in robotics, it iscommon to use Gaussian pdf to statistically characterize sensor measurements,robot locations, map pdfs represented in Figure have the same mean, = 2, and 21> 22> 23showing that the larger the variance the greater the dispersion around themean. 6 4 (x) 32 22 12 Figure : Gaussian pdf with different variances ( 21= 32, 22= 22, 23= 1)3 Definition square-root of the variance, , is usually known a real numberxa R, the Probability that the random variableX N( , 2)takes values less or equalxais given byP r{X xa}= xa f(x)dx= xa 1 2 exp[ (x )22 2]dx,( )represented by the shaded area in Figure 6 4 Figure : Probability evaluation using pdfTo evaluate the Probability in ( ) theerror function,erf(x), which is relatedwithN(0,1),erf(x) =1 2 x0exp y2/2dy( )plays a key role.

3 In fact, with a change of variables, ( ) may be rewritten asP r{X xa}= erf( xa )f or xa +erf(xa )f or xa stating the importance of the error function, whose values for variousxare dis-played in Table various aspects of robotics, in particular when dealing with uncertainty inmobile robot localization, it is common the evaluation of the Probability that a4xerf xxerf xxerf xxerf : erf - Error functionrandom variableY(more generally a random vector representing the robot loca-tion) lies in an interval around the mean value . This interval is usually definedin terms of the standard deviation, , or its the error function, ( ),the Probability that the random variableXliesin an interval whose width is related with the standard deviation, isP r{|X | }= (1) = ( )P r{|X | 2 }= (2) = ( )P r{|X | 3 }= (3) = ( )In other words, the Probability that a Gaussian random variable lies in the in-terval[ 3 , + 3 ]is equal to Figure represents the situation( )corresponding to the Probability ofXlying in the interval[ , + ].

4 Another useful evaluation is the locus of values of the random variableXwhere the pdf is greater or equal a given pre-specified valueK1, ,1 2 exp[ (x )22 2] K1 (x )22 2 K( )5 6 4 (x)2 Figure : Probability of X taking values in the interval[ , + ], =2, = ln( 2 K1). This locus is the line segment K x + Kas represented in Figure x f(x) Figure : Locus of x where the pdf is greater or equal thanK16 chapter 2 Normal random vectorsA random vectorX= [X1, X2, .. Xn]T Rnis Gaussian if its pdf isfX(x) =1(2 )n/2| |1/2exp{ 12(x mX)T 1(x mX)}( )where mX=E(X)is the mean vector of the random vectorX, X=E[(X mX)(X mX)T]is the covariance matrix, n=dimXis the dimension of the random vector,also represented asX N(mX, X).In ( ), it is assumed thatxis a vector of dimensionnand that 1exists. If is simply non-negative definite, then one defines a Gaussian vector through thecharacteristic function, [2].The mean vectormXis the collection of the mean values of each of the randomvariablesXi,mX=E =.

5 7 The covariance matrix is symmetric with elements, X= TX== E(X1 mX1)2E(X1 mX1)(X2 mX2)..E(X1 mX1)(Xn mXn)E(X2 mX2)(X1 mX1)E(X2 mX2) (X2 mX2)(Xn mXn)..E(Xn mXn)(X1 mX1)..E(Xn mXn)2 .The diagonal elements of are the variance of the random variablesXiand thegeneric element ij=E(Xi mXi)(Xj mXj)represents the covariance of thetwo random to the scalar case, the pdf of a Gaussian random vector is completelycharacterized by its first and second moments, the mean vector and the covariancematrix, respectively. This yields interesting Properties , some of which are listedin chapter studying the localization of autonomous robots, the random vectorXplays the role of the robot s location. Depending on the robot characteristics andon the operating environment, the location may be expressed as: a two-dimensional vector with the position in a 2D environment, a three-dimensional vector (2d-position and orientation) representing a mo-bile robot s location in an horizontal environment, a six-dimensional vector (3 positions and 3 orientations) in an underwatervehicleWhen characterizing a 2D-laser scanner in a statistical framework, each rangemeasurement is associated with a given pan angle corresponding to the scanningmechanism.

6 Therefore the pair (distance, angle) may be considered as a randomvector whose statistical characterization depends on the physical principle of thesensor above examples refer quantities, ( , robot position, sensor measure-ments) that are not deterministic. To account for the associated uncertainties, weconsider them as random vectors. Moreover, we know how to deal with Gaussianrandom vectors that show a number of nice Properties ; this (but not only) pushesus to consider these random variables as been governed by a Gaussian many cases, we have to deal with low dimension Gaussian random vec-tors (second or third dimension), and therefore it is useful that we particularize8the n-dimensional general case to second order and present and illustrate following section particularizes some results for a second order Particularization for second orderIn the first two above referred cases, the Gaussian random vector is of order twoor three. In this section we illustrate the case when n= [XY] 2,be a second-order Gaussian random vector, with mean,E[Z] =E[XY]=[mXmY]( )and covariance matrix, =[ 2X XY XY 2Y]( )where 2 Xand 2 Yare the variances of the random variablesXandYand XYisthe covariance ofXandY, defined covariance XYof the two random variablesXandYis thenumber XY=E[(X mX)(Y mY)]( )wheremX=E(X)andmY=E(Y).

7 Expanding the product ( ), yields, XY=E(XY) mXE(Y) mYE(X) +mXmY( )=E(XY) E(X)E(Y)( )=E(X) mXmY.( )Definition coefficientof the variablesXandYis defined as = XY X Y( )9 Result correlation coefficient and the covariance of the variablesXandYsatisfy the following inequalities ,| | 1,| XY| X Y.( )Proof:[2] Consider the mean value ofE[a(X mX) + (Y mY)]2=a2 2X+ 2a XY+ 2 Ywhich is a positive quadratic for anya, and hence, the discriminant is negative, , XY 2X 2Y 0from where ( ) to the previous definitions, the covariance matrix ( ) is rewrittenas =[ 2X X Y X Y 2Y].( )For this second-order case, the Gaussian pdf particularizes as, withz= [x y]T R2,f(z) =12 det exp[ 12[x mXy mY] 1[x mXy mY]T]( )=12 X Y 1 2exp[ 12(1 2)((x mX)2 2X 2 (x mX)(y mY) X Y+(y mY)2 2Y)]where the role played by the correlation coefficient is this stage we present a set of definitions and Properties that, even thoughbeing valid for any two random variables,XandY, also apply to the case whenthe random variables (rv) are IndependenceTwo random variablesXandYare called independent if the joint pdf,f(x, y)equals the product of the pdf of each random variable,f(x),f(y), ,f(x, y) =f(x)f(y)In the case of Gaussian random variables, clearlyXandYare independentwhen = 0.

8 This issue will be further explored UncorrelatednessTwo random variblesXandYare called uncorrelated if their covariance iszero, , XY=E[(X mX)(Y mY)] = 0,which can be written in the following equivalent forms: = 0, E(XY) =E(X)E(Y).Note thatE(X+Y) =E(X) +E(Y)but, in general,E(XY)6=E(X)E(Y). However, whenXandYare uncorre-lated,E(XY) =E(X)E(Y)according to Definition OrhthogonalityTwo random variablesXandYare called orthognal ifE(XY) = 0,which is represented asX YProperty uncorrelated, thenX mX Y uncorrelated andmX= 0andmY= 0, thenX two random variablesXandYare independent, then they areuncorrelated, ,f(x, y) =f(x)f(y) E(XY) =E(X)E(Y)but the converse is not, in general, :From the definition of mean value,E(XY) = xyf(xy)dxdy= xf(x)dx yf(y)dy=E(X)E(Y).11 Property two Gaussian random variablesXandYare uncorrelated, theyare also independent, , for Normal or Gaussian random variables,independencyis equivalent to uncorrelatedness.

9 IfX N( X, X)andY N( Y, Y)f(xy) =f(x)f(y) E(XY) =E(X)E(Y) = Variance of the sum of two random variablesLetXandYbe tworandom variables, jointly distributed, with meanmXandmYand correlationcoefficient and letZ=X+ ,E(Z) =mZ=E(X) +E(Y) =mX+mY( ) 2Z=E[(Z mZ)2)] = 2X+ 2 X Y+ 2Y.( )Proof:Evaluating the second term in ( ) yields: 2Z=E[(X mX) + (Y mY)]2=E[(X mX)2] + 2E[(X mX)(Y mY)] +E[(Y mY)2]from where the result immediately Variance of the sum of two uncorrelated random variablesLetXandYbe two uncorrelated random variables, jointly distributed, withmeanmXandmYand letZ=X+ , 2Z= 2X+ 2Y( ) , if two random variables are uncorrelated, then the variance of their sumequals the sum of their regain the case of two jointly Gaussian random varaibles,XandY, withpdf represented by ( ) to analyze, in the plots of Gaussian pdfs, the influenceof the correlation coefficient in the bell-shaped represents four distinct situations with zero mean and null corre-lation betweenXandY, , = 0, but with different values of the standarddeviations Xand Y.

10 It is clear that, in all cases, the maximum of the pdf is ob-tained for the mean value. As = 0, , the random variables are uncorrelated,12 505 (X)=0, E(Y)=0, X=1, Y=1, =0yf(x,y) 505 (X)=0, E(Y)=0, X= , Y= , =0yf(x,y)a)b) 505 (X)=0, E(Y)=0, X=1, Y= , =0yf(x,y) 505 (X)=0, E(Y)=0, X=2, Y= , =0yf(x,y)c)d)Figure : Second-order Gaussian pdfs, withmX=mY= 0, = 0a) X=1, Y= 1, b) X= , Y= , c) X= 1, Y= , d) X= 2, Y= 113the change in the standard deviations Xand Yhas independent effects in eachof the components. For example, in Figure ) the spread around the mean isgreater along thexcoordinate. Moreover, the locus of constant value of the pdfis an ellipse with its axis parallel to thexandyaxis. This ellipse has equal axislength, , is a circumference, when both random variables,XandYhave thesame standard deviation, X= examples in Figure show the influence of the correlation coefficienton the shape of the pdf. What happens is that the axis of the ellipse referredbefore will no longer be parallel to the axisxandy.