Introduction to Stochastic Processes - College of …

1 1 Introduction to Stochastic Processes IE 7710: Fall 2013 Instructor Dr. Kenneth Chelst Offices: 2069 and 2017 Manufacturing Engineering Building Office Hours 2 hours before each class. E-mail Textbook: Applied probability and Stochastic Processes , Feldman and Flores, Springer 2010 References: Introduction to probability Models: Sheldon Ross Academic Press Chapter 3 - Conditional probability and Conditional Expectation Modeling and Analysis of Stochastic Systems - V. Kulkarni, Chapman Hall Chapter 5 - Poisson Process and Exponential Distribution Modeling random Processes for Engineers and Managers, James Solberg, John Wiley Chapter 2 Application of Markov Chains Overview The course will develop skills in building and analyzing both discrete time (Markov Chains) and continuous time (Poisson Process and Queueing) Stochastic models. It will also introduce a wide range of applications and diverse research topics in the broad area of Stochastic models.

2 Course Goals 1. Develop better skills with regard to basic probability concepts that are directly relevant to Stochastic Processes . These include a) conditional probability and conditional expectation b) binomial, geometric, Poisson and exponential distributions c) order statistics. 2. Markov Chains: a) Develop ability to translate real-world contexts into a Markov Chain model. b) Develop ability to calculate appropriate measures of performance for both ergodic and non-ergodic chains: steady state probabilities, absorption probabilities, mean time between states c) Develop skills in using Excel to calculate these measures. d) Develop a basic understanding of Markov Decision Processes (MDP) and their application. Students will NOT learn the algorithms for solving an MDP. 3. Queuing Models a) Apply appropriate formula to determine L, Lq, W, and Wq for infinite capacity Queues (Poisson Arrivals and Exponential Service time b) Apply appropriate approximation formula for infinite capacity queues without Poisson arrivals or exponential service c) Develop and apply appropriate formula for finite capacity and finite source queues d) Analyze pre-emptive priority queues.

3 4. Stochastic Model: Application a) Learn about a wide range of Stochastic models that were used to address an actual problem. b) Recognize the difference between a Stochastic model that was actually used and a paper describes a potential application 5. Stochastic Models Research Topics a) Develop an understanding of how a Stochastic modeling research topic develops over time b) Exposure to a wide range of research threads in Stochastic Processes 2 Class # Date Topic 1 Wed 28-Aug Basic probability Review Ch1 & Notes 2 Wed 4-Sep Conditional probability & expectation Ross Ch. 3 3 Mon 9-Sep Conditional probability & expectation Ch1 & notes 4 Wed 11-Sep Binomial and Geometric Notes 5 Mon 16-Sep Binomial and Geometric Ch 4 (exclude ) 6 Wed 18-Sep Order Statistics 7 Mon 23-Sep Poisson Process/ Exponential Notes 8 Wed 25-Sep Markov Chain Introduction assumptions 9 Mon 30-Sep Markov Chain Transitions - Ch 5 10 Wed 2-Oct Markov Chains Steady State P(n) No Class Mon 7-Oct INFORMS Conference No Class Wed 9-Oct INFORMS Conference 11 Mon 14-Oct Steady state probabilities and absorbing probabilities Ch 5 12 Wed 16-Oct Mean number of visits 13 Mon 21-Oct Steady State Probabilities and Excel Solberg Ch 3 14 Wed 23-Oct Absorbing Probabilities and EXCEL Solberg Ch 3 15 Mon 28-Oct Markov Decision Process (not on exam)

4 Ch 12 16 Wed 30-Oct Markov Process Ch 6 Mon 4-Nov Take home exam due 17 Mon 4-Nov Queueing basics Single server Ch 7 18 Wed 6-Nov Finite capacity 19 Mon 11-Nov Multiple servers 20 Wed 13-Nov Finite source Solberg 21 Mon 18-Nov Non exponential service Solberg 22 Wed 20-Nov Priority queues Notes 23 Mon 25-Nov Queue - Advanced Concepts: Reneging, Batch Arrival, Batch Service, Optimal Control Holiday Wed 27-Nov No Class 24 Mon 2-Dec Quiz 2 Markov Process and Queueing 25 Wed 4-Dec Real World Applications Students 26 Mon 9-Dec 27 Wed 11-Dec Literature Thread Presentation Students 28 Mon 16-Dec 3 Homework: five sets at 5 points each Points 1. Basic probability through Order Statistics Wed Sept 18 5 2. Poisson and Exponential Chapter 4 Mon Sept 30 5 3. Markov Chains Chapter 5 Wed Oct 28 8 4. Queuing Chapter 7 Mon Nov 11 7 Quiz: two for 20 points each 1. Take home due Nov 3rd Poisson and Discrete Markov Chain 20 2.

5 December 2nd Queueing 20 Application Review Presentation only 10 Literature Thread (4+ papers on related topic) Presentation and Paper 30 Total 105 Point Range Grade 93-105 A 88-92 A- 84-87 B+ 80-83 B 75-79 B- 70-74 C+ 65-69 C 0-64 F Independent of your total points, to obtain at least an A- grade in the course, you must score at least 33 out of 40 on the Exams. Homework Policy Assistance Allowed (No penalty) You are ALLOWED and encouraged to seek help from a classmate or provide help in the form of discussing any of the problems and how to solve them You are NOT allowed to copy homework. Once you have discussed the various problems, you must sit down separately and write and submit homework. HOWEVER, if you just copy the solution and do not write it by yourself you will o 1st time receive a 0 for the homework assignment even if it was JUST 1 of the problems on the homework assignment o 2nd time You fail the course.

6 If you received help, you must record on the last page the name on who helped you and with regard to which problems. There is NO penalty for getting help The individual who helped will be eligible for bonus points and special recognition from me. Individual Real-World Application Article review - 6 to 8 slides: 10-12 minutes Submit article for approval by Nov. 11th Slides must be submitted on Nov. 25th, (You will receive feedback on slide design and be expected to modify before presentation) Description of problem Describe technique - model that was used Unique aspects of Modeling effort Results and Impact 4 Literature Thread Submit most recent paper for approval by Nov. 4th 4 (or more) papers on related topic of one common thread of articles Most recent paper published in 2010 or later. This is primary focus Seminal article earliest article that initiated this stream of literature Review 2 more papers referenced by the most recent Presentation 8 to 10 slides: 15 minutes Slides must be submitted on Dec.

7 4th, a week in advance of presentation (You will receive feedback on slide design and be expected to modify before presentation) 1. Describe broad problem area 2. Key variables, parameters and modeling assumptions 3. Review Seminal paper important earliest paper (3 slides) a. Frame the broad topic area b. Main modeling tool c. Key assumptions 4. Review just 2 papers in presentation. (1-2 slides per paper) a. Key assumptions b. Main contribution 5. Review last paper What was its major contribution (3 slides) a. What issues addressed b. Basics of Model c. Key results Paper 1,500 to 2000 words Due Monday December 16th 1. Describe broad problem area and the seminal paper: key variables, parameters and assumptions 400-600 2. Primary contribution of each of two subsequent papers: 200-400 words per paper 3. Detailed Review of last paper 400 800 words a. What was its major contribution? b. What techniques did it use to make the contribution? c.

8 Future Research suggested? Schedule Task Subject Wed Sept 18 HW 1 Basic probability through Order Statistics Mon Sept 30 HW 2 Poisson and Exponential Wed Oct 16 HW 3 Markov Chains Wed Oct 23rd Quiz 1 Poisson and Exponential and Markov Chains Wed Nov 6th Prof Approval? Recent article (2010+) end point for literature thread Mon Nov 11th HW 4 Queuing basics Wed Nov 13th Prof Approval? Application article Mon Nov 18 HW5 Queueing advanced Mon. Nov 25th Quiz 2 Queueing Basic and Advanced Mon. Nov 25th Prof Review Slides for application article Dec 4th Prof Review Slides for literature thread Dec 4th or 9th Presentation Application article Dec 11th or 16th Presentation Literature thread Dec 16th Paper Literature thread