Course Catalogue

View Specification Document | Reading List

To provide a good understanding of the key concepts of stochastic processes in various settings: discrete time and finite state space; discrete time and countable state space; continuous time and countable state space.

Two lectures per week for 10 weeks and fortnightly tutorials.

Some optional courses may be constrained by space and entry to these is not guaranteed unless you are in a programme for which this is a compulsory course.

STATS4024 Stochastic Processes

120-minute, end-of-course examination (100%)

To provide a good understanding of the key concepts of stochastic processes in various settings: discrete time and finite state space; discrete time and countable state space; continuous time and countable state space.

By the end of this course students should be able to:

■ Explain the concept of a homogeneous Markov chain;

■ Define what it means for a matrix to be stochastic and discuss the consequences this has for the eigenvalues;

■ Explain what it means for a state to be absorbing, periodic, persistent, transient, or ergodic;

■ Explain the difference between a stationary and a limiting distribution;

■ Decide whether a Markov chain has a unique limiting distribution;

■ Describe the gambler's ruin problem in terms of a discrete-time Markov chain;

■ Calculate the probability of ruin and the expected duration of a game in the gambler's ruin problem;

■ Explain the difference between a homogeneous and an inhomogeneous linear difference equation and the standard procedure to solve them;

■ Derive the probability distribution of a random walk;

■ Calculate the first return of a symmetric random walk;

■ Define the concept of a homogeneous Poisson process, and derive the form of the distribution of the inter-arrival times;

■ Calculate the expected length and waiting time for a queue in which arrivals form a Poisson process;

■ Define the concepts of a reliability function and a hazard function;

■ Calculate the expected number of renewals in a renewal process;

■ Explain the difference between a discrete-time and a continuous-time Markov chain and explain the concept of a rate matrix;

■ Decide whether a birth-death process has a stationary distribution;

■ Read further into one topic related to the course and answer questions on this topic.

None