An Introduction to Stochastic Modeling - booksite.elsevier.com

1 An Introduction to Stochastic ModelingAn Introduction toStochastic ModelingFourth EditionMark A. PinskyDepartment of MathematicsNorthwestern UniversityEvanston, IllinoisSamuel KarlinDepartment of MathematicsStanford UniversityStanford, CaliforniaAMSTERDAM BOSTON HEIDELBERG LONDONNEW YORK OXFORD PARIS SAN DIEGOSAN FRANCISCO SINGAPORE SYDNEY TOKYOA cademic Press is an imprint of ElsevierAcademic Press is an imprint of Elsevier30 Corporate Drive, Suite 400, Burlington, MA 01803, USAThe Boulevard, Langford Lane, Kidlington, Oxford, OX5 1GB, UKc 2011 Elsevier Inc. All rights part of this publication may be reproduced or transmitted in any form or by any means, electronic ormechanical, including photocopying, recording, or any information storage and retrieval system, withoutpermission in writing from the publisher. Details on how to seek permission, further information about thePublisher s permissions policies and our arrangements with organizations such as the Copyright ClearanceCenter and the Copyright Licensing Agency, can be found at our book and the individual contributions contained in it are protected under copyright by the Publisher(other than as may be noted herein).

2 NoticesKnowledge and best practice in this field are constantly changing. As new research and experiencebroaden our understanding, changes in research methods, professional practices, or medical treatmentmay become and researchers must always rely on their own experience and knowledge in evaluating andusing any information, methods, compounds, or experiments described herein. In using such informationor methods they should be mindful of their own safety and the safety of others, including parties forwhom they have a professional the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume anyliability for any injury and/or damage to persons or property as a matter of products liability, negligence orotherwise, or from any use or operation of any methods, products, instructions, or ideas contained in thematerial of Congress Cataloging-in-Publication DataApplication Library Cataloguing-in-Publication DataA catalogue record for this book is available from the British : 978-0-12-381416-6 For information on all Academic Press publications,visit our by.

3 DiacriTech, IndiaPrinted in the United States of America10 11 12 138 7 6 5 4 3 2 1 ContentsPreface to the Fourth EditionxiPreface to the Third EditionxiiiPreface to the First EditionxvTo the InstructorxviiAcknowledgmentsxix1 and and Expected Distribution and of of Axiomatic Probability Major Discrete and Negative Binominal Poisson Multinomial Continuous Normal Exponential Uniform Gamma Beta Joint Normal Elementary Exponential Functions, Integrals, and Sums42viContents2 Conditional Probability and Conditional Discrete Dice Game Distributions: The Mixed Moments of a Random Distribution of a Random on a Continuous Random Markov Maximal Inequality for Nonnegative Martingales733 Markov Chains: Probability Matrices of a Markov Markov Chain Inventory Ehrenfest Urn Chains in Discrete Queueing Markov Step First Step General Absorbing Markov Special Markov Two-State Markov Chains Defined by IndependentRandom Random of Random Walks and Success General Random Success Runs Markov Look at First Step of Branching Mean and Variance of a Branching Processes and Generating Functions and Extinction Generating Functions and Sums ofIndependent Random Branching Processes159 Contentsvii4 The Long Run Behavior of Markov Transition Probability Stochastic of the Limiting History in the State and Continuous Sampling Replacement Replacement Classification of Markov of a Markov and Transient Basic Limit Theorem of Markov Markov Chains2155 Poisson Poisson Distribution and the Poisson Poisson Poisson Law of Rare Law of Rare Events and the Poisson of Theorem Associated with the Poisson

4 Uniform Distribution and Poisson Quota Poisson and Marked Poisson Poisson Poisson Processes2676 Continuous Time Markov Birth for the Poisson Birth Yule Death Linear Death Failure Under Static and Death Equations of Birth and Death Limiting Behavior of Birth and Death and Death Processes with Absorbing of Absorption into State Time Until Continuous Time Markov Poisson Process with a Markov Intensity3387 Renewal of a Renewal Process and Related Examples of Renewal Sketches of Renewal Poisson Process Viewed as a Renewal Asymptotic Behavior of Renewal Elementary Renewal Renewal Theorem for Continuous Asymptotic Distribution ofN(t) Limiting Distribution of Age and Excess and Variations on Renewal Renewal Renewal and Related Renewal Discrete Renewal Population Growth with Age Distribution3848 Brownian Motion and Related Motion and Gaussian Little Brownian Motion Stochastic Central Limit Theorem and the Invariance Maximum Variable and the Reflection Reflection Time to First Reach a Zeros of Brownian and Brownian Brownian Brownian Motion with Gambler s Ruin Brownian Ornstein Uhlenbeck Second Approach to Physical Brownian Position Long Run Measure and Integration4419 Queueing Queueing FormulaL= Sampling of Queueing Arrivals, Exponential Service Service Time and with Service System with Two-Server Overflow Priority Acyclic Queueing Basic Queues in Acyclic.

5 Time of Theorem Open General Open Network49210 Random Velocity Two-State Random The Telegraph Distribution Functions and Densities in theTwo-State Passage Time Random Finite Markov Chains and Random Velocity Constructive Approach of Random Velocity Random Evolution Existence-Uniqueness of the First-OrderSystem ( ) Single Hyperbolic Spectral Properties of the Transition Recurrence Properties of Random Law and Central Limit Transport in Higher The Rayleigh Problem of Random Three-Dimensional Rayleigh Model52311 Characteristic Functions and Their of the Characteristic Basic Properties of the Characteristic Formulas for Characteristic Reciprocity/Local Inversion and Parseval s Formula for General Random Continuity of the Continuity of the Central Limit s Formula and Representation ofn! of Stirling s deMoivre Laplace Theorem539 Further Reading541 Answers to Exercises543 Index557 Preface to the Fourth EditionSince the publication of the third edition in 1998, some new developments haveoccurred.

6 Samuel Karlin died in 2007, leaving a gap at the authorship level and thenew designation of the fourth edition, we have added two new chapters: Chapter 10 on random evo-lution and Chapter 11 on characteristic Evolutiondenotes a setof Stochastic models, which describe continuous motion with piecewise linear samplefunctions. Explicit formulas are available in the simplest cases. In the general case, onehas a central limit theorem, which is pursued more generally in Chapter 11, Char-acteristic Functions and Their Applications. Here the necessary tools from FourierAnalysis are developed and applied when necessary. Many theorems are proved infull detail, while other proofs are sketched in the spirit of the earlier Chapters 1 proofs may be found by consulting the intermediate textbooks listed in thesection on further reading. Instructors who have taught from the third edition maybe reassured that Chapters 1 9 of the new edition are identical to the correspondingchapters of the new express our thanks to Michael Perlman of the University of Washington andRussell Lyons of Indiana University for sharing their lists of errata from the third edi-tion.

7 We would also like to thank Craig Evans for useful advice on partial NoteSamuel Karlin earned his undergraduate degree from the Illinois Institute of Technol-ogy and his doctorate from Princeton University in 1947 at age 22. He served on thefaculty of Caltech from 1948 1956 before joining the faculty of Stanford University,where he spent the remainder of his career. Karlin made fundamental contributions tomathematical economics, bioinformatics, game theory, evolutionary theory, biomolec-ular sequence analysis, mathematical population genetics, and total authored 10 books and more than 450 articles. He was a member of theAmerican Academy of Arts and Sciences and the National Academy of 1989, he received the National Medal of Science for his broad and remarkableresearches in mathematical analysis, probability theory, and mathematical statisticsand in the application of these ideas to mathematical economics, mechanics, and genet-ics. He died on December 18, A. PinskyDepartment of MathematicsNorthwestern UniversityEvanston, IL 60208-2730 Preface to the Third EditionThe purposes, level, and style of this new edition conform to the tenets set forth in theoriginal preface.

8 We continue with our objective of introducing some theory and appli-cations of Stochastic processes to students having a solid foundation in calculus andin calculus-level probability, but who are not conversant with the epsilon delta def-initions of mathematical analysis. We hope to entice students toward the deeper studyof mathematics that is prerequisite to further work in Stochastic processes by showingthe myriad and interesting ways in which Stochastic models can help us understand thereal have removed some topics and added others. We added a small section onmartingales that includes an example suggesting the martingale concept as appropriatefor Modeling the prices of assets traded in a perfect market. A new chapter introducesthe Brownian motion process and includes several applications of it and its variantsin financial Modeling . In this chapter the Black Scholes formula for option pricingis evaluated and compared with some reported prices of options. A Poisson processwhose intensity is itself a Stochastic process is described in another new treatments have been updated.

9 The law of rare events is presented via aninequality that measures the accuracy of a Poisson approximation for the distributionof the sum of independent, not necessarily identically distributed, Bernoulli randomvariables. We have added the shot noise model and related it to a random text contains more than 250 exercises and 350 problems. Exercises are elemen-tary drills intended to promote active learning and to develop familiarity with conceptsthrough use. They often simply involve the substitution of numbers into given formu-las or reasoning one or two steps away from a definition. They are the kinds of simplequestions that we, as instructors, hope that students would pose and answer for them-selves as they read a text. Answers to the exercises are given at the end of the book sothat students may gauge their understanding as they go are more difficult. Some involve extensive algebraic or calculus ma-nipulation. Many are word problems wherein the student is asked, in effect, to modelsome described scenario.

10 As in formulating a model, the first step in the solution of aword problem is often a sentence of the form Letx=.. A manual containing thesolutions to the problems is available from the reasonable strategy on the part of the teacher might be to hold students respon-sible for all of the exercises, but to require submitted solutions only to selected prob-lems. Every student should attempt a representative selection of the problems in orderto develop his or her ability to carry out Stochastic Modeling in his or her area to the Third EditionA small number of problems are labeled Computer Challenges. These call formore than pencil and paper for their analyses, and either simulation, numerical explo-ration, or symbol manipulation may prove helpful. Computer Challenges are meantto be open-ended, intended to explore what constitutes an answer in today s world ofcomputing power. They might be appropriate as part of an honors our focus is on Stochastic Modeling , in some instances, we have omitted aproof and contented ourselves with a precise statement of a result and examples of itsapplication.