PROBABILITY AND STOCHASTIC PROCESSES

1 PROBABILITY ANDSTOCHASTIC PROCESSESA Friendly Introductionfor Electrical and Computer EngineersPROBABILITY ANDSTOCHASTIC PROCESSESA Friendly Introductionfor Electrical and Computer EngineersRoy D. YatesRutgers, The State University of New JerseyDavid J. GoodmanRutgers, The State University of New JerseyJOHN WILEY & SONS, York Chichester Weinheim Brisbane SingaporeTorontoACQUISITIONS EDITORBill ZobristMARKETING MANAGERK atherine HepburnPRODUCTION EDITORKen SantorDESIGNERL aura BoucherThis book was set in Times Roman by the authors and printed and bound by Quebecor - Fairfield,Inc.

2 The cover was printed by Phoenix Color the cover: The cover art was developed by Ken Harris, a student in the ECE Department atRutgers University, and shows a bivariate Gaussian PROBABILITY density book is printed on acid-free paper. The paper in this book was manufactured by a mill whose forest management programs includesustained yield harvesting of its timberlands. Sustained yield harvesting principles ensure that thenumbers of trees cut each year does not exceed the amount of new 1999 John Wiley & Sons, Inc. All rights part of this publication may be reproduced, stored in a retrieval system or transmitted in anyform or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise,except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, withouteither the prior written permission of the Publisher, or authorization through payment of theappropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA01923, (508) 750-8400, fax (508) 750-4470.

3 Requests to the Publisher for permission should beaddressed to the Permissions Department, John Wiley & Sons, Inc., 605 Third Avenue, New York,NY 10158-0012, (212) 850-6011, fax (212) 850-6008, E-Mail: of Congress Cataloging - in - Publications DataYates RoyD Probabilityandstochasticprocesses afriendlyintroductionforelectrical computerengineers RoyD Yates DavidJ Goodman p cm Includesindex ISBN cloth alk paper Probabilities Stochasticprocesses I Goodman DavidJ II Title QA Y dc CIPP rinted in the United States of America10987654321To Theresa and LizPREFACEWhen we started teaching the courseProbability and STOCHASTIC Processesto Rutgersundergraduates in 1991, we never dreamed we would write a textbook on the bookshelves contain more than a dozen PROBABILITY texts.

4 Many of them directedat electrical engineering students. We respect most of them. However, we have yet tofind one that works well for Rutgers students. We discovered to our surprise that themajority of our students have a hard time learning the subject. Beyond meeting degreerequirements, the main motivation of most of our students is to learn how to solvepractical problems. For the majority, the mathematical logic of PROBABILITY theory is,in itself, of minor interest. What the students want most is an intuitive grasp of thebasic concepts and lots of practice working on students told us that the textbooks we assigned, for all their mathematicalelegance, didn t meet their needs.

5 To help them, we distributed copies of our lecturenotes, which gradually grew into this book. We also responded to student feedback byadministering a half-hour quiz every week. A quiz contains ten questions designed totest a student s grasp of the concepts presented that week. The quizzes provide rapidfeedback to students on whether they are catching on to the new material. This is es-pecially important in PROBABILITY theory because much of the math appears deceptivelysimple to engineering students. Reading a text and attending lectures, they feel theyunderstand everything presented to them.

6 However, when confronted with problems tosolve, they discover that it takes a lot of careful thought and practice to use the math -ematics correctly. Although the equations and formulas are simple, knowing whichone to use is difficult. This is a reversal from some mathematics courses, where theequations are given and the solutions are hard to meet the needs of our students, this book has several distinctive characteristics: The entire text adheres to a single model that begins with an experiment con-sisting of a procedure and observations. The mathematical logic is apparent to readers.

7 Every fact is identified clearly asa definition, an axiom, or a theorem. There is an explanation, in simple English,of the intuition behind every concept when it first appears in the text. The mathematics of discrete random variables are introduced separately fromthe mathematics of continuous random variables. STOCHASTIC PROCESSES fit comfortably within the unifying model of the text. Theyare introduced in Chapter 6, immediately after the presentations of discrete andcontinuous random variables. Subsequent material, including central limit the-orem approximations, laws of large numbers, and statistical inference, then useexamples that reinforce STOCHASTIC process concepts.

8 The text concludes with introductions to random signal processing and There is an abundance of exercises that put the theory to use. Many workedout example problems are embedded in the text. Each section concludes witha simple quiz to help students gauge their grasp of that section. An appendixincludes a complete solution for each quiz. At the end of each chapter, there areproblems that span a range of estimate that the material in this book represents about 125% of a one semesterundergraduate course. We suppose that every introduction to PROBABILITY to theory willspend about two thirds of a semester covering the material in the first five remainder of a course will be devoted to about half of the material in the finalsix chapters, with the selection depending on the preferences of the instructor and theneeds of the students.

9 Rutgers electrical and computer engineering students take thiscourse in the first semester of junior year. The following semester they use much ofthe material in Principles of have also used this book in an entry-level graduate course. That course coversthe entire book in one semester, placing more emphasis on mathematical derivationsand proofs than the undergraduate course. Although most of the early material in thebook is familiar in advance to many graduate students, the course as a whole bringsour diverse graduate student population up to a shared level of OF THE BOOKThe first five chapters carry the core material that is common to practically all in-troductory engineering courses in PROBABILITY theory.

10 Chapter 1 examines probabilitymodels defined on abstract sets. It introduces the set theory notation used throughoutthe book and states the three axioms of PROBABILITY and several theorems that follow di-rectly from the axioms. It defines conditional PROBABILITY , the Law of Total PROBABILITY ,Bayes theorem, and independence. The chapter concludes by presenting combinato-rial principles and formulas that are used later in the second and third chapters apply this material to models of discrete randomvariables, introducing expected values, functions of random variables, variance, co-variance, and conditional PROBABILITY mass functions.