Probability and Stochastic Processes - National Sun …

1 Wireless Information Transmission System of Communications EngineeringNational Sun Yat-senUniversityProbability and Stochastic Processes2 Table of Contents Probability Random Variables, Probability Distributions, and Probability Densities Statistical Averages of Random Variables Some Useful Probability Distributions Upper Bounds on the Tail Probability Sums of Random Variables and the Central Limit Theorem Stochastic Processes Statistical Averages Power Density Spectrum Response of a Linear Time-Invariant System to a Random Input Signal Discrete-Time Stochastic Signals and Systems CyclostationaryProcesses3 Sample spaceor certain eventof a die experiment: The six outcomes are the sample pointsof the experiment. An eventis a subsetof S, and may consist of any number of sample points. For example: The complementof the event A, denoted by , consists of all the sample points in Sthat are not in A:{}6,5,4,3,2,1=S{ }4,2=AA{}6,5,3,1=AProbability4 Two events are said to be mutually exclusiveif they have no sample points in common that is, if the occurrence of one event excludes the occurrence of the other.

2 For example: The union(sum) of two events in an event that consists of all the sample points in the two events. For example:{ }{ }6,3,1 ;4,2==BA{ }{}SAACBDC==== 6,3,2,13,2,1events. exclusivemutually are and AAProbability5 The intersectionof two events is an event that consists of the points that are common to the two events. For example: When the events are mutually exclusive, the intersection is the null event, denoted as . For example:{}3,1==CBE =AA Probability6 Associated with each event Acontained in Sis its probabilityP(A). Three postulations: P(A) 0. The Probability of the sample space is P(S)=1. Suppose that Ai , i=1, 2, .., are a (possibly infinite) number of events in the sample space Ssuch that Then the Probability of the union of these mutually exclusive events satisfies the condition:,..2,1 ;= =jiAAji = iiiiAPAP)( Probability7 Joint eventsand joint probabilities(two experiments) If one experiment has the possible outcomes Ai , i=1,2.

3 ,n, and the second experiment has the possible outcomes Bj, j=1,2,..,m, then the combined experiment has the possible joint outcomes(Ai ,Bj), i=1,2,..,n, j=1,2,..,m. Associated with each joint outcome (Ai ,Bj) is the joint probabilityP (Ai ,Bj) which satisfies the condition: Assuming that the outcomes Bj, j=1,2,..,m, are mutually exclusive, it follows that: If all the outcomes of the two experiments are mutually exclusive, then:1),(0 jiBAP ==mjijiAPBAP1)(),(()( )111,1nmnijiijiPABPA= ==== Probability8 Conditional probabilities The conditional Probability of the event Agiven the occurrence of the event B is defined as:provided P(B)>0. )(),()|(BPBAPBAP=)()|()()|(),(APABPBPBAP BAP==If two events and are mutually exclusive, , then ( | ) BABPA B == .1)|( and have we, ofsubset a is If==BAPBBAAB.

4 And of occurrence ussimultaneo thedenotes ),(is,That . ofy probabilit theas dinterprete is ),(BABAPBABAP Probability9 Bayes theorem: P(Ai) represents their a priori probabilitiesand P(Ai|B) is the a posteriori probabilityof Aiconditioned on having observed the received signal , 1, 2,.., , are mutually exclusive events such that and is an arbitrary event with nonzero Probability , then( ,) ( |) ()iniiiAinASBPA BPPA BPB===== 1( | )( )( | )( )iinjjjB A PAPB A PA= ( )()() ( )11,|nnjjjjjPBPBAPB A PA==== Probability10 Statistical independence When the events Aand Bsatisfy the relation P(A,B)=P(A)P(B),they are said to be statistically independent. Three statistically independent events A1, A2, and A3must satisfy the following conditions:).()|( then , ofoccurrence on the dependnot does of occurrence theIfAPBAPBA=)()()()|(),(BPAPBPBAPBAP==) ()()(),,()()(),()()(),()()(),(3213213232 31312121 APAPAPAAAPAPAPAAPAPAPAAPAPAPAAP====Proba bility11 The functionX(s) is called a random variable.

5 Example 1: If we flip a coin, the possible outcomes are head (H) and tail (T), so Scontains two points labeled H and T. Suppose we define a function X(s) such that:Thus we have mapped the two possible outcomes of the coin-flipping experiment into the two points ( +1,-1) on the real line. Example 2: Tossing a die with possible outcomes S={1,2,3,4,5,6}. A random variable defined on this sample space may be X(s)=s, in which case the outcomes of the experiment are mapped into the integers 1,..,6, or, perhaps, X(s)=s2, in which case the possible outcomes are mapped into the integers {1,4,9,16,25,36}.Given an experiment having a sample space andelements , we define a funciton ( ) whose domainis and whose range is a set of numbers on the real SXsS ==+=T)(s 1-H)(s 1)(sXRandom Variables, Probability Distributions, and Probability Densities12 Give a random variable X, let us consider the event {X x} where xis any real number in the interval (- , ).

6 We write the Probability of this event as P(X x) and denote it simply by F(x), , The function F(x) is called the Probability distribution functionof the random variable X. It is also called the cumulative distribution function(CDF). 1)(0 )( and 0)(= = FF( )(), Fx PX x-x= < < Random Variables, Probability Distributions, and Probability Densities13 Examples of the cumulative distribution functions of two discreterandom Variables, Probability Distributions, and Probability Densities14 An example of the cumulative distribution function of a continuousrandom Variables, Probability Distributions, and Probability Densities15 An example of the cumulative distribution function of a random variable of a mixed Variables, Probability Distributions, and Probability Densities16 The derivativeof the CDF F(x),denoted as p(x),is called the Probability density function (PDF) of the random variable X.

7 When the random variable is discreteor of a mixedtype, the PDF contains impulsesat the points of discontinuity of F(x): << = << =xxduupxFxdxxdFxp ,)()( ,)()( = ==niiixxxXPxp1)()()( Random Variables, Probability Distributions, and Probability Densities17()122121122 11212 21 Determining the Probability that a random variable falls in an interval , , where .() ()()( )() () ()( ) () Xxxx xPXx PXx Px XxFxFxPx X xPx X xFxFxp> = + < = + < < = ={}211212()The Probability of the event is simplythe area under the PDF in the range .xxx dxx Xxx Xx< < Random Variables, Probability Distributions, and Probability Densities18 Multiple random variables, joint Probability distributions, and joint Probability densities: (two random variables)( )()() ().

8 0,,, : thatNote1,),( PDFs. called are variables theofoneover gintegratin from obtained )( and )( PDFs The)(),( )(),( ),(),( :PDFJ oint ),(),(),( :CDFJ oint 122--121211221212121212212-x-12122112112 = = = = === == = xFxFFFxddxxxpxpxpxpdxxxpxpdxxxpxxFxxxxpu dduuupxXxXPxxFxmarginalRandom Variables, Probability Distributions, and Probability Densities19 Multiple random variables, joint Probability distributions, and joint Probability densities: (multidimensional random variables)()().0,..,,,, ).,..,,,(,..,,,, ),..,,(),..,,( ),..,,(..),..,,( PDFJ oint ..),..,,(.. ),..,,(),..,,( CDFJ oint . variablesrandom are 21 that Suppose4154141413221212121-2x-1212211211 2= = = == == nnnnnnnnnnxxnnnnixxxFxxxxFxxxFxxxpdxdxxx xpxxxFxxxxxxpduudduuuupxXxXxXPxxxF.

9 ,n,,, iXnRandom Variables, Probability Distributions, and Probability Densities20 Themeanor expected valueof X, which characterized by its PDF p(x), is defined as:This is the first momentof random variable X. The n-thmomentis defined as: Define Y=g(X), the expected value of Yis: = dxxxpmXEx)()(dxxpxXEnn)()( =[] ==dxxpxgXgEYE)()()()(Statistical Averages of Random Variables21 The n-thcentral momentof the random variable X is: When n=2, the central moment is called thevarianceof the random variable and denoted as : In the case of two random variables, X1and X2, with joint PDF p(x1,x2), we define the joint momentas:()[] = =dxxpmxmXEYE nxnx)()()([]2222222)()()()()(xxxxmXEXEXE dxxpmx = = = 2x 21212121),()(dxdxxxpxxXXEnknk =Statistical Averages of Random Variables22 The joint central momentis defined as: If k=n=1, the joint momentand joint central momentare called the correlationand the covarianceof the random variables X1and X2, respectively.

10 The correlationbetween Xiand Xjis given by the joint moment:[] = 212122112211),()()( )()(dxdxxxpmxmxmXmXEnknk =jijijijidxdxxxpxxXXE),()(Statistical Averages of Random Variables23 The covariancebetween Xiand Xjis given by the joint central moment: The n nmatrix with elements ijis called the covariance matrixof the random variables, Xi, i =1,2, ..,n.()()()()()jijijijijijijijijijijijij iijijiijjijiijjiijjiiijmmXXEmmmmmmdxdxxx pxxmmdxdxxxpxxdxdxxxpmmmxmxxxdxdxxxpmxmx mXmXEjj =+ = =+ = = )( ),( ),( ),( ),( ][ Statistical Averages of Random Variables24 Two random variables are said to be uncorrelatedif E(XiXj)=E(Xi)E(Xj)=mimj. Uncorrelated Covariance ij= 0. If XiandXjare statistically independent, they are uncorrelated. If XiandXjare uncorrelated, they are not necessarystatistically independently.