Procesos estocasticos

1

STAT2003: Introduction to Applied Probability ( 1 unit) 2 STAT3102: Stochastic Processes ( 1 unit) 2
Prior to 2007/08 these courses had codes STATB087 and STATC082 respectively. Lecturer Alexandros Beskos, room 136. Aims of course To provide an introduction to the study of systems which change state stochastically with time and to facilitate the development of skills in the application ofprobabilistic ideas. Objectives of course On completion of this course students should have a clear understanding of the Markov property in discrete and continuous time. For discrete-time Markov chains they should be able to: find and classify the irreducible classes of intercommunicating states; calculate absorption or first passage times and probabilities; assess the equilibrium behaviour. Forsimple examples of continuous-time Markov chains, they should be able to: write down the forward equations; find and interpret the equilibrium distribution. Prerequisites STAT2001 (STATB094) or STAT3101 (STATC080) or their equivalent. Course contents Revision of conditional probability. Markov chains (discrete time and states): transient and equilibrium behaviour, first passage times, classification ofstates, applications. Markov processes (continuous time, discrete states): general theory, forward and backward equations, equilibrium distributions; Poisson process, interval and counting properties; birth and death processes and other simple examples. Main text S.M. Ross Introduction to Probability Models. Academic Press, 6th ed. (1996). Ch 3, 4, 5.

2 Other books G. Grimmett & D. WelshProbability: An introduction. OUP (1986). Ch 4, 10, 11. D. Stirzaker Elementary Probability. CUP (1994). Ch 6, 9. G.R. Grimmett & D.R. Stirzaker Probability and Random Processes. OUP, 2nd ed.(1992). Ch 3,4,6. D.R. Cox & H.D. Miller The Theory of Stochastic Processes. Chapman and Hall (1965). Ch 3, 4. H.M. Taylor & S. Karlin An introduction to stochastic modelling. Academic Press, 3rd ed. (1998). A.B.Clarke & R.L. Disney Probability and random processes. Wiley (1985). K.L. Chung Elementary probability theory with stochastic processes. Springer-Verlag (1974). Ch 8. Assessment for examination grading A 2 1 hour (STAT2003) or 2 hour (STAT3102) written examination in term 3, plus in-course 2 assessment consisting of one ‘closed book’ test. The final mark will be a 9 to 1 weighted average of thewritten examination and in-course assessment marks. Only College approved calculators are to be used in the exam. Please see your handbook for a list of College approved calculators. There will be no choice of questions in the written examination. You must not miss the in-course assesment. If you do you will fail the course. Other set work 8 sets of exercises to provide informal monitoring of students’progress. Timetabled workload Lectures and problems classes: Three hours per week in total. Tutorials: One hour per week.

3

Contents
1 Introduction and revision 1.1 Conditional probability and conditional expectation . . . . . . . . . . . . . . 1.1.1 1.1.2 1.1.3 1.2 Random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discrete random variables . . . . . . . . . . . .. . . . . . . . . . . . Continuous random variables . . . . . . . . . . . . . . . . . . . . . . 5 8 8 9 10 11 11 13 15 17 18 18 19 20 21 22 26 26 27 28 32 33 34 36 37 40 40 43 46

Conditional probability and conditional expectation . . . . . . . . . . . . . . 1.2.1 1.2.2 Discrete case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Continuous case . . . . . . . . . . . . . . . . .. . . . . . . . . . . . .

1.3 1.4

Generating functions are useful . . . . . . . . . . . . . . . . . . . . . . . . . Example problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 Discrete-time stochastic processes 2.1 2.2 2.3 Definitions and basic properties . . . . . . . . . . . . . . . . . . . . . . . . . The Markov property . . . . . . . . . . . . . . . . . . . ....