An Introduction To Stochastic Modeling

1 An IntroductionTo StochasticModelingHoward KarlinAn Introduction toStochastic ModelingThird EditionAn Introduction toStochastic ModelingThird EditionHoward M. TaylorStatistical ConsultantOnancock, Vi giniaSamuel KarlinDepartment of MathematicsStanford UniversityStanford, CaliforniaOAcademic PressSan DiegoLondonBostonNew YorkSydneyTokyoTorontoThis book is printed on acid-free 1998, 1994, 1984 by Academic PressAll rights part of this publication may be reproduced ortransmitted in any form or by any means, electronicor mechanical, including photocopy, recording, orany information storage and retrieval system.

2 Withoutpermission in writing from the may be sought directly from Elsevier's Science and Technology Rights Department inOxford, UK. Phone: (44) 1865 843830, Fax: (44) 1865 853333, c-mail: may also complete your request on-line via the Elsevier homepage: byselecting 'Customer Support' and then 'Obtaining Permissions'.ACADEMIC PRESSAn Imprint of Elsevier525 B St., Suite 1900, San Diego, California 92101-4495, USA1300 Boylston Street, Chestnut Hill, MA 02167, Press Limited24-28 Oval Road, London NW 1 7DX, of Congress Cataloging-in-Publication DataTaylor, Howard Introduction to Stochastic Modeling / Howard M.

3 Taylor, SamuelKarlin. - 3rd bibliographical references ( ) and : 978-0-12-684887-8 ISBN-10: 0-12-684887-41. Stochastic Karlin, '.76--dc2lISBN-13: 978-0-12-684887-8 ISBN-10: 0-12-684887-4 PRINTED IN THE UNITED STATES OF AMERICA05060708 IP 987654 ContentsPrefaceixIIntroduction11. Stochastic Modeling12. Probability Review63. The Major Discrete Distributions244. Important Continuous Distributions335. Some Elementary Exercises436. Useful Functions, Integrals, and Sums53 IIConditional Probability and ConditionalExpectation571. The Discrete Case572. The Dice Game Craps643.

4 Random Sums704. Conditioning on a Continuous Random Variable795. Martingales*87 IIIM arkov Chains: Introduction951. Definitions952. Transition Probability Matrices of a Markov Chain1003. Some Markov Chain Models1054. First Step Analysis1165. Some Special Markov Chains1356. Functionals of Random Walks and Success Runs151*Stars indicate topics of a more advanced or specialized Another Look at First Step Analysis*1698. Branching Processes*1779. Branching Processes and Generating Functions*184IV The Long Run Behavior of Markov Chains1991. Regular Transition Probability Matrices1992.

5 Examples2153. The Classification of States2344. The Basic Limit Theorem of Markov Chains2455. Reducible Markov Chains*258V Poisson Processes2671. The Poisson Distribution and the Poisson Process2672. The Law of Rare Events2793. Distributions Associated with the Poisson Process2904. The Uniform Distribution and Poisson Processes2975. Spatial Poisson Processes3116. Compound and Marked Poisson Processes318 VIContinuous Time Markov Chains3331. Pure Birth Processes3332. Pure Death Processes3453. Birth and Death Processes3554. The Limiting Behavior of Birth and DeathProcesses3665.

6 Birth and Death Processes with Absorbing States3796. Finite State Continuous Time Markov Chains3947. A Poisson Process with a Markov Intensity*408 VIIR enewal Phenomena4191. Definition of a Renewal Process andRelated Concepts4192. Some Examples of Renewal Processes4263. The Poisson Process Viewed as a RenewalProcess432*Stars indicate topics of a more advanced or specialized The Asymptotic Behavior of Renewal Processes4375. Generalizations and Variations on RenewalProcesses4476. Discrete Renewal Theory*457 VIIIB rownian Motion and Related Processes4731. Brownian Motion and Gaussian Processes4732.

7 The Maximum Variable and the Reflection Principle4913. Variations and Extensions4984. Brownian Motion with Drift5085. The Ornstein-Uhlenbeck Process*524 IXQueueing Systems5411. Queueing Processes5412. Poisson Arrivals, Exponential Service Times5473. General Service Time Distributions5584. Variations and Extensions5675. Open Acyclic Queueing Networks5816. General Open Networks592 Further Reading601 Answers to Exercises603 Index625*Stars indicate topics of a more advanced or specialized to the First EditionStochastic processes are ways of quantifying the dynamic relationships ofsequences of random events.

8 Stochastic models play an important role inelucidating many areas of the natural and engineering sciences. They canbe used to analyze the variability inherent in biological and medicalprocesses, to deal with uncertainties affecting managerial decisions andwith the complexities of psychological and social interactions, and to pro-vide new perspectives, methodology, models, and intuition to aid in othermathematical and statistical book is intended as a beginning text in Stochastic processes for stu-dents familiar with elementary probability calculus. Its aim is to bridgethe gap between basic probability know-how and an intermediate-levelcourse in Stochastic processes-for example, A First Course in StochasticProcesses, by the present objectives of this book are three: (1) to introduce students to thestandard concepts and methods of Stochastic Modeling ; (2) to illustrate therich diversity of applications of Stochastic processes in the sciences.

9 And(3) to provide exercises in the application of simple Stochastic analysis toappropriate chapters are organized around several prototype classes of sto-chastic processes featuring Markov chains in discrete and continuoustime, Poisson processes and renewal theory, the evolution of branchingevents, and queueing models. After the concluding Chapter IX, we pro-vide a list of books that incorporate more advanced discussions of severalof the models set forth in this to the Third EditionThe purposes, level, and style of this new edition conform to the tenets setforth in the original preface.

10 We continue with our objective of introduc-ing some theory and applications of Stochastic processes to students hav-ing a solid foundation in calculus and in calculus-level probability, butwho are not conversant with the "epsilon-delta" definitions of mathemat-ical analysis. We hope to entice students towards the deeper study ofmathematics that is prerequisite to further work in Stochastic processes byshowing the myriad and interesting ways in which Stochastic models canhelp us understand the real have removed some topics and added others. We added a small sec-tion on martingales that includes an example suggesting the martingaleconcept as appropriate for Modeling the prices of assets traded in a perfectmarket.