Probability, Statistics, and Random Processes for ...

1 probability , statistics , and Random Processes for electrical EngineeringThird EditionAlberto Leon-GarciaUniversity of TorontoUpper Saddle River, NJ 07458vContentsPrefaceixCHAPTER 1 probability Models in electrical and Computer Models as Tools in Analysis and Design Models Models Detailed Example: A Packet Voice Transmission System Examples of Book 16 Summary17 Problems18 CHAPTER 2 Basic Concepts of probability Random Experiments Axioms of probability Probabilities Using Counting probability of Events Experiments Randomness: Random Number Generators Points: Event Classes Points: Probabilities of Sequences of Events 75 Summary79 Problems80 CHAPTER 3 Discrete Random Notion of a Random Variable Random Variables and probability Mass Function Value and Moments of Discrete Random Variable probability Mass Function Discrete Random Variables of Discrete Random Variables 127 Summary129 Problems130**viContentsCHAPTER 4 One Random Cumulative Distribution Function probability Density Function Expected Value of Continuous Random Variables of a Random Variable Markov and Chebyshev Inequalities Methods Reliability Calculations Methods for Generating Random Variables 202 Summary213 Problems215 CHAPTER 5 Pairs of Random Random Variables of Discrete Random Variables Joint cdf of Joint pdf of Two Continuous Random Variables of Two Random Variables Moments and Expected Values of a Function

2 Of Two Random Variables probability and Conditional Expectation of Two Random Variables of Jointly Gaussian Random Variables Independent Gaussian Random Variables 284 Summary286 Problems288 CHAPTER 6 Vector Random Random Variables of Several Random Variables Values of Vector Random Variables Gaussian Random Vectors of Random Variables Correlated Vector Random Variables 342 Summary346 Problems348*ContentsviiCHAPTER 7 Sums of Random Variables and Long-Term of Random Variables Sample Mean and the Laws of Large Numbers 365 Weak Law of Large Numbers 367 Strong Law of Large Numbers Central Limit Theorem 369 Central Limit Theorem of Sequences of Random Variables Arrival Rates and Associated Averages Distribution s Using the Discrete Fourier Transform 392 Summary400 Problems402 CHAPTER and Sampling Distributions Estimation Likelihood Estimation Intervals Testing Decision Methods the Fit of a Distribution to Data 462 Summary469 Problems471 CHAPTER 9 Random of a Random Process a Random Process Processes .

3 Sum Process, Binomial Counting Process,and Random Walk and Associated Random Processes Random Processes , Wiener Process and Brownian Motion Random Processes , Derivatives, and Integrals of Random Processes Averages of Random Processes and Ergodic Theorems Series and Karhunen-Loeve Expansion Random Processes 550 Summary554 Problems557**viiiContentsCHAPTER 10 Analysis and Processing of Random Spectral Density of Linear Systems to Random Signals Random Processes Linear Systems Kalman Filter the Power Spectral Density Techniques for Processing Random Signals 628 Summary633 Problems635 CHAPTER 11 Markov Processes Markov Chains of States, Recurrence Properties, and Limiting Probabilities Markov Chains Markov Chains Techniques for Markov Chains 692 Summary700 Problems702 CHAPTER 12 Introduction to Queueing Elements of a Queueing System s Formula M/M/1 Queue Systems: M/M/c, M/M/c/c, And Queueing Systems Queueing Systems Analysis Using Embedded Markov Chains s Theorem: Departures From M/M/cSystems of Queues: Jackson s Theorem and Data Analysis of Queueing Systems Tables of Fourier Transforms and Linear Algebra 802 Index805M>M> **This chapter presents the basic concepts of probability theory.

4 In the remainder of thebook, we will usually be further developing or elaborating the basic concepts present-ed here. You will be well prepared to deal with the rest of the book if you have a goodunderstanding of these basic concepts when you complete the following basic concepts will be presented. First, set theory is used to specifythe sample space and the events of a Random experiment. Second, the axioms of prob-ability specify rules for computing the probabilities of events. Third, the notion of con-ditional probability allows us to determine how partial information about the outcomeof an experiment affects the probabilities of events. Conditional probability also allowsus to formulate the notion of independence of events and of experiments. Finally, weconsider sequential Random experiments that consist of performing a sequence ofsimple Random subexperiments.

5 We show how the probabilities of events in these exper-iments can be derived from the probabilities of the simpler subexperiments. Throughoutthe book it is shown that complex Random experiments can be analyzed by decompos-ing them into simple Random EXPERIMENTSA Random experiment is an experiment in which the outcome varies in an unpre-dictable fashion when the experiment is repeated under the same ran-dom experiment is specified by stating an experimental procedure and a set of one ormore measurements or a ball from an urn containing balls numbered 1 to 50. Note the number ofthe a ball from an urn containing balls numbered 1 to 4. Suppose that balls 1and 2 are black and that balls 3 and 4 are white. Note the number and color of the ball you a coin three times and note the sequence of heads and a coin three times and note the number of the number of voice packets containing only silence produced from agroup of Nspeakers in a 10-ms :E4:E3:E2:E1:21 Basic Concepts of probability Theory2 CHAPTER22 Chapter 2 Basic Concepts of probability TheoryExperimentA block of information is transmitted repeatedly over a noisy channel until anerror-free block arrives at the receiver.

6 Count the number of transmissions a number at Random between zero and the time between page requests in a Web the lifetime of a given computer memory chip in a specified the value of an audio signal at time ExperimentDetermine the values of an audio signal at times and ExperimentPick two numbers at Random between zero and a number Xat Random between zero and one, then pick a number Yatrandom between zero and system component is installed at time For let as longas the component is functioning, and let after the component specification of a Random experiment must include an unambiguous statementof exactly what is measured or observed. For example, Random experiments may consistof the same procedure but differ in the observations made, as illustrated by and A Random experiment may involve more than one measurement or observation,as illustrated by and A Random experiment may even involve acontinuum of measurements, as shown by Experiments and are examples of sequential experi-ments that can be viewed as consisting of a sequence of simple subexperiments.

7 Canyou identify the subexperiments in each of these? Note that in the second subex-periment depends on the outcome of the first Sample SpaceSince Random experiments do not consistently yield the same result, it is necessary todetermine the set of possible results. We define an outcomeorsample pointof a ran-dom experiment as a result that cannot be decomposed into other results. When weperform a Random experiment, one and only one outcome occurs. Thus outcomes aremutually exclusive in the sense that they cannot occur simultaneously. The samplespaceSof a Random experiment is defined as the set of all possible will denote an outcome of an experiment by where is an element or pointinS. Each performance of a Random experiment can then be viewed as the selection atrandom of a single point (outcome) from sample space Scan be specified compactly by using set notation.

8 It can be visu-alized by drawing tables, diagrams, intervals of the real line, or regions of the plane. Thereare two basic ways to specify a all the elements, separated by commas, inside a pair of a propertythat specifies the elements of the set:Note that the order in which items are listed does not change the set, ,and are the same , 2, 3, 0650, 1, 2, 36A=5x:x is an integer such that , 1, 2, 36,zz,E13E13E3,E4,E5,E6,E12, ,E3,E11,E12, 0t= :E13:E12 :E9:E8:E7:E6:Section Specifying Random Experiments23 Example sample spaces corresponding to the experiments in Example are given below using setnotation:See Fig. (a).See Fig. (b).See Fig. (c).See Fig. (d).for which for and for where is the time when the component experiments involving the same experimental procedure may have dif-ferent sample spaces as shown by Experiments and Thus the purpose of an ex-periment affects the choice of sample t0,X1t2= of functions X1t2S13=51x,y2 ,y2 and ,v22:-q6v16q and -q6v26q6S10=5v:-q6v6q6=1-q,q2S9=5t:t 06=30,q2S8=5t.

9 T 06=30,q2S7=5 , 14S6=51, 2, 3, 6S5=50, 1, 2, ,N6S4=50, 1, 2, 36S3=5 HHH, HHT, HTH, THH, TTH, THT, HTT, TTT6S2=511,b2,12,b2,13,w2,14,w26S1=51, 2, , 506(a) Sample space for Experiment (b) Sample space for Experiment (c) Sample space for Experiment (d) Sample space for Experiment spaces for Experiments and , E9, E12,24 Chapter 2 Basic Concepts of probability Theory1 The Cartesian product of the sets AandBconsists of the set of all ordered pairs (a,b), where the first ele-ment is taken from Aand the second from are three possibilities for the number of outcomes in a sample space. Asample space can be finite, countably infinite, or uncountably infinite. We call Sadiscrete sample spaceifSis countable; that is, its outcomes can be put into one-to-onecorrespondence with the positive integers. We call Sacontinuous sample spaceifSisnot countable.

10 Experiments and have finite discrete sample has a countably infinite discrete sample space. Experiments throughhave continuous sample an outcome of an experiment can consist of one or more observations ormeasurements, the sample space Scan be multi-dimensional. For example, the out-comes in Experiments and are two-dimensional, and those in Experi-ment are three-dimensional. In some instances, the sample space can be written asthe Cartesian product of other example,where Ris the set ofreal numbers, and where It is sometimes convenient to let the sample space include outcomes that areimpossible. For example, in Experiment it is convenient to define the samplespace as the positive real line, even though a device cannot have an infinite are usually not interested in the occurrence of specific outcomes, but rather inthe occurrence of some event ( , whether the outcome satisfies certain condi-tions).