AnIntroductionto StatisticalSignalProcessing

1 IAn Introduction toStatistical Signal ProcessingPr(f F) =P({ : F}) =P(f 1(F))f 1(F)fF-January 4, 2011iiAn Introduction toStatistical Signal ProcessingRobert M. GrayandLee D. DavissonInformation Systems LaboratoryDepartment of Electrical EngineeringStanford UniversityandDepartment of Electrical Engineering and Computer ScienceUniversity of Marylandc 2004 by Cambridge University Press. Copies of the pdf file maybedownloaded for individual use, but multiple copies cannot be made or printedwithout our FamiliesContentsPrefacepageixAcknowledge mentsxiiGlossaryxiii1 Introduction12 Spinning pointers and flipping Probability Discrete probability Continuous probability Elementary conditional Problems733 Random variables, vectors.

2 And Random Distributions of random Random vectors and random Distributions of random Independent random Conditional Statistical detection and Additive Binary detection in Gaussian Statistical Characteristic Gaussian random Simple random Directly given random Discrete time Markov Nonelementary conditional Problems1684 Expectation and Functions of random Functions of several random Properties of Conditional Jointly Gaussian Expectation as Implications for linear Correlation and linear Correlation and covariance The central limit Sample Convergence of random Weak law of large Strong law of large Asymptotically uncorrelated Problems2585 Second-order Linear filtering of random Linear systems I/O Power spectral Linearly filtered uncorrelated Linear White Time Mean square Linear estimation and Problems3496 A menagerie of Discrete time linear Sums of iid random

3 Independent stationary increment Second-order moments of isi Specification of continuous time isi Moving-average and autoregressive The discrete time Gauss Markov Gaussian random The Poisson counting Compound Composite random Exponential Thermal Random Problems400 Appendix A Set Examples of Mappings and Linear Linear system Problems431 Appendix B Sums and Double The Lebesgue integral443 Appendix C Common univariate distributions446 Appendix D Supplementary reading448 References453 Index457 PrefaceThe origins of this book lie in our earlier bookRandom Processes: A Math-ematical Approach for Engineers(Prentice Hall, 1986).

4 This book began asa second edition to the earlier book and the basic goal remains unchanged to introduce the fundamental ideas and mechanics of randomprocesses toengineers in a way that accurately reflects the underlying mathematics, butdoes not require an extensive mathematical background and does not bela-bor detailed general proofs when simple cases suffice to get the basic ideasacross. In the years since the original book was published, however, it hasevolved into something bearing little resemblance to its ancestor.

5 Numer-ous improvements in the presentation of the material have been suggestedby colleagues, students, teaching assistants, and reviewers, and by our ownteaching experience. The emphasis of the book shifted increasingly towardsexamples and a viewpoint that better reflected the title of the courses wetaught using the book for many years at Stanford University and at theUniversity of Maryland:An Introduction to Statistical Signal of the basic content of this course and of the fundamentals of randomprocesses can be viewed as the analysis of statistical signal processing sys-tems.

6 Typically one is given a probabilistic description for one random object,which can be considered as aninput signal. An operation is applied to theinput signal (signal processing) to produce a new random object, theoutputsignal. Fundamental issues include the nature of the basic probabilistic de-scription, and the derivation of the probabilistic description of the outputsignal given that of the input signal and the particular operation perusal of the literature in statistical signal processing, communications,control, image and video processing, speech and audio processing, medi-cal signal processing, geophysical signal processing, andclassical statisticalareas of time series analysis, classification and regression.

7 And pattern recog-nition shows a wide variety of probabilistic models for input processes andixxPrefacefor operations on those processes, where the operations might be determin-istic or random, natural or artificial, linear or nonlinear,digital or analog, orbeneficial or harmful. An introductory course focuses on thefundamentalsunderlying the analysis of such systems: the theories of probability, randomprocesses, systems, and signal the original book went out of print, the time seemed ripeto convertthe manuscript from the prehistoric troff format to the widely used LATEX format and to undertake a serious revision of the book in the process.

8 As therevision became more extensive, the title changed to match the course nameand content. We reprint the original preface to provide someof the originalmotivation for the book, and then close this preface with a description ofthe goals sought during the many subsequent to Random Processes: An Introduction for EngineersNothing in nature is random..A thing appears random onlythrough the incompleteness of our , Ethics II do not believe that God rolls to EinsteinLaplace argued to the effect that given complete knowledge of the physics of anexperiment, the outcome must always be predictable.

9 This metaphysical argumentmust be tempered with several facts. The relevant parameters may not be measur-able with sufficient precision due to mechanical or theoretical limits. For example,the uncertainty principle prevents the simultaneous accurate knowledge of both po-sition and momentum. The deterministic functions may be too complexto computein finite time. The computer itself may make errors due to power failures, lightning,or the general perfidy of inanimate objects. The experiment couldtake place in aremote location with the parameters unknown to the observer; for example, in acommunication link, the transmitted message is unknown a priori, forif it were not,there would be no need for communication.

10 The results of the experiment could bereported by an unreliable witness either incompetent or dishonest. For these andother reasons, it is useful to have a theory for the analysis and synthesis of pro-cesses that behave in a random or unpredictable manner. The goalis to constructmathematical models that lead to reasonably accurate prediction of the long-termaverage behavior of random processes. The theory should produce good estimatesof the average behavior of real processes and thereby correcttheoretical derivationswith measurable this book we attempt a development of the basic theory and applications ofrandom processes that uses the language and viewpoint of rigorous mathematicaltreatments of the subject but which requires only a typical bachelor s degree level ofelectrical engineering education including elementary discrete and continuous timelinear systems theory, elementary probability.