Probability, Statistics, and Random Processes for ...

1 Probability, Statistics, and Random Processes for Electrical EngineeringThird EditionAlberto Leon-GarciaUniversity of TorontoUpper Saddle River, NJ 07458 Library of Congress Cataloging-in-Publication DataLeon-Garcia, , statistics, and Random Processes for electrical engineering / Alberto Leon-Garcia. -- 3rd bibliographical references and : 978-0-13-147122-1 (alk. paper)1. Electric engineering--Mathematics. 2. Probabilities. 3. Stochastic Processes . I. Leon-Garcia, Alberto. Probabilityand Random Processes for electrical engineering. II. '46213--dc222007046492 Vice President and Editorial Director, ECS:Marcia J. HortonAssociate Editor:Alice DworkinEditorial Assistant:William OpaluchSenior Managing Editor:Scott DisannoProduction Editor:Craig LittleArt Director:Jayen ConteCover Designer:Bruce KenselaarArt Editor:Greg DullesManufacturing Manager:Alan Fischer Manufacturing Buyer:Lisa McDowellMarketing Manager:Tim Galligan 2008 Pearson Education, Prentice HallPearson Education, Saddle River, NJ 07458 All rights reserved.

2 No part of this book may be reproduced, in any form or by any means, without permission inwriting from the Prentice HallTMis a trademark of Pearson Education, Inc. MATLAB is a registered trademark of The MathWorks, Inc. All other product or brand names are trademarks or registered trademarks of their respective author and publisher of this book have used their best efforts in preparing this book. These efforts include thedevelopment, research, and testing of the theories and programs to determine their effectiveness. The author andpublisher make no warranty of any kind, expressed or implied, with regard to the material contained in this book.

3 Theauthor and publisher shall not be liable in any event for incidental or consequential damages in connection with, orarising out of, the furnishing, performance, or use of this in the United States of America10987654321 ISBN 0-13-147122-8978-0-13-147122-1 Pearson Education Ltd.,LondonPearson Education Australia Pty. Ltd.,SydneyPearson Education Singapore, Pte. Education North Asia Ltd.,Hong KongPearson Education Canada, Inc.,TorontoPearson Educaci n de Mexico, de Education Japan,To ky oPearson Education Malaysia, Pte. Education,Upper Saddle River, New JerseyTOKAREN,CARLOS,MARISA, page intentionally left blank vContentsPrefaceixCHAPTER 1 Probability Models in Electrical and Computer Models as Tools in Analysis and Design Models Models Detailed Example: A Packet Voice Transmission System Examples of Book 16 Summary17 Problems18 CHAPTER 2 Basic Concepts of Probability Random Experiments Axioms of Probability Probabilities Using Counting Probability of Events Experiments Randomness: Random Number Generators Points: Event Classes Points.

4 Probabilities of Sequences of Events 75 Summary79 Problems80 CHAPTER 3 Discrete Random Notion of a Random Variable Random Variables and Probability Mass Function Value and Moments of Discrete Random Variable Probability Mass Function Discrete Random Variables of Discrete Random Variables 127 Summary129 Problems130**viContentsCHAPTER 4 One Random Cumulative Distribution Function Probability Density Function Expected Value of Continuous Random Variables of a Random Variable Markov and Chebyshev Inequalities Methods Reliability Calculations Methods for Generating Random Variables 202 Summary213 Problems215 CHAPTER 5 Pairs of Random Random Variables of Discrete Random Variables Joint cdf of Joint pdf of Two Continuous Random Variables of Two Random Variables Moments and Expected Values of a Function of Two Random Variables Probability and Conditional Expectation of Two Random Variables of Jointly Gaussian Random Variables Independent Gaussian Random Variables 284 Summary286 Problems288 CHAPTER 6 vector Random Random Variables of Several Random Variables Values of vector Random Variables Gaussian Random Vectors of Random Variables Correlated vector Random Variables

5 342 Summary346 Problems348*ContentsviiCHAPTER 7 Sums of Random Variables and Long-Term of Random Variables Sample Mean and the Laws of Large Numbers 365 Weak Law of Large Numbers 367 Strong Law of Large Numbers Central Limit Theorem 369 Central Limit Theorem of Sequences of Random Variables Arrival Rates and Associated Averages Distribution s Using the Discrete Fourier Transform 392 Summary400 Problems402 CHAPTER and Sampling Distributions Estimation Likelihood Estimation Intervals Testing Decision Methods the Fit of a Distribution to Data 462 Summary469 Problems471 CHAPTER 9 Random of a Random Process a Random Process Processes .

6 Sum Process, Binomial Counting Process,and Random Walk and Associated Random Processes Random Processes , Wiener Process and Brownian Motion Random Processes , Derivatives, and Integrals of Random Processes Averages of Random Processes and Ergodic Theorems Series and Karhunen-Loeve Expansion Random Processes 550 Summary554 Problems557**viiiContentsCHAPTER 10 Analysis and Processing of Random Spectral Density of Linear Systems to Random Signals Random Processes Linear Systems Kalman Filter the Power Spectral Density Techniques for Processing Random Signals 628 Summary633 Problems635 CHAPTER 11 Markov Processes Markov Chains of States, Recurrence Properties.

7 And Limiting Probabilities Markov Chains Markov Chains Techniques for Markov Chains 692 Summary700 Problems702 CHAPTER 12 Introduction to Queueing Elements of a Queueing System s Formula M/M/1 Queue Systems: M/M/c, M/M/c/c, And Queueing Systems Queueing Systems Analysis Using Embedded Markov Chains s Theorem: Departures From M/M/cSystems of Queues: Jackson s Theorem and Data Analysis of Queueing Systems Tables of Fourier Transforms and Linear Algebra 802 Index805M>M> **ixPrefaceThis book provides a carefully motivated, accessible, and interesting introduction toprobability, statistics, and Random Processes for electrical and computer engineers.

8 Thecomplexity of the systems encountered in engineering practice calls for an understand-ing of probability concepts and a facility in the use of probability tools. The goal of theintroductory course should therefore be to teach both the basic theoretical conceptsand techniques for solving problems that arise in practice. The third edition of thisbook achieves this goal by retaining the proven features of previous editions: Relevance to engineering practice Clear and accessible introduction to probability Computer exercises to develop intuition for randomness Large number and variety of problems Curriculum flexibility through rich choice of topics Careful development of Random process edition also introduces two major new features: Introduction to statistics Extensive use of MATLAB TO ENGINEERING PRACTICEM otivating students is a major challenge in introductory probability courses.

9 Instructorsneed to respond by showing students the relevance of probability theory to engineeringpractice. Chapter 1 addresses this challenge by discussing the role of probability modelsin engineering design. Practical current applications from various areas of electrical andcomputer engineering are used to show how averages and relative frequencies providethe proper tools for handling the design of systems that involve randomness. These ap-plication areas include wireless and digital communications, digital media and signalprocessing, system reliability, computer networks, and Web systems. These areas areused in examples and problems throughout the INTRODUCTION TO PROBABILITY THEORYP robability theory is an inherently mathematical subject so concepts must be presentedcarefully, simply, and gradually.

10 The axioms of probability and their corollaries are devel-oped in a clear and deliberate manner. The model-building aspect is introduced throughthe assignment of probability laws to discrete and continuous sample spaces. The notionof a single discrete Random variable is developed in its entirety, allowing the student toxPrefacefocus on the basic probability concepts without analytical complications. Similarly, pairsof Random variables and vector Random variables are discussed in separate most important Random variables and Random Processes are developed insystematic fashion using model-building arguments. For example, a systematic devel-opment of concepts can be traced across every chapter from the initial discussions oncoin tossing and Bernoulli trials, through the Gaussian Random variable, central limittheorem, and confidence intervals in the middle chapters, and on to the Wiener processand the analysis of simulation data at the end of the book.