Uncertainty Assessment and Bayesian Computation

Course information

This course aims to develop the foundations of modern Bayesian statistic and demonstrate how prior distributions are updated to posterior distributions in simple statistical models. Learners are introduced to advanced stochastic simulation methods such as Markov-Chain Monte Carlo and model comparisons in a Bayesian setting.

Prerequisite Knowledge

Learners should understand probability theory including Bayes' Theorem, random variables, probability functions, joint probability distributions and transformations of random variables. In addition, Learners should be familiar with statistical inference and linear models.

This course is typically taken in year 2 of the MSc in Data Analytics/Data Analytics for Government programme and learners typically have the knowledge and skills covered in our year 1 courses.

This course assumes that you have comparative knowledge and skills covered in the following courses, alternatively, you may wish to consider taking some of the courses listed before attempting this course.

Intended Learning Outcomes

By the end of this course learners will be able to:

  • Understand prior distributions in the presence of data, and calculate posterior predictive distributions;
  • Compute various summaries of the posterior distribution, including posterior mean, MAP estimate, posterior standard deviation and credible regions and the predictive distribution;
  • Explain the operation and theory of Markov-Chain Monte-Carlo methods and the Metropolis-Hastings algorithm;
  • Derive the full conditional distributions for parameters in simple low-dimensional problems;
  • Perform inference in Bayesian models using high-level software such BUGS, STAN and R;
  • Explain the role of hyperparameters the empirical Bayes approach for their determination;
  • Apply diagnostic procedures to check convergence and mixing of MCMC methods;
  • Implement Bayesian approaches to model selection;
  • Describe alternative approaches to Bayesian computation.

Syllabus

Week 1

  • Understand the scope of Bayesian statistics
  • Revise standard discrete and continuous distributions
  • Joint distribution functions and transformations

Week 2 (sample material)

  • Statistical inference under the Bayesian framework
  • Conjugate priors and the Binomial model

Week 3

  • Posterior distributions
  • Conjugate priors and the Normal model
  • Uninformative priors

Week 4

  • Sampling from a posterior distribution
  • Rejection and Importance sampling
  • Multivariate posterior distributions

Week 5

  • Markov Chain Monte Carlo
  • Metropolis-Hastings sampler
  • Implementation in R
  • Assessing convergence

Mid-term week break

Week 6

  • Linear models under the Bayesian framework
  • Conjugate priors
  • Jeffrey's prior
  • Marginal likelihood
  • Variable selection under the Bayesian framework
  • Bayesian LASSO
  • Case study: housing prices

Week 7

  • Modelling multiple experiments
  • Empirical Bayes
  • Bayesian hierarchical models
  • Shrinkage

Week 8

  • Gibbs sampling
  • Gibbs sampling algorithm
  • Implementing Gibbs sampling in R

Week 9

  • Hamiltonian Monte Carlo
  • Introduction to Stan
  • Implementation in Stan

Week 10

  • Hypothesis testing under the Bayesian framework
  • Estimating marginal likelihoods
  • Information Criteria

Week 11

  • Linear mixed effects models under the Bayesian framework
  • Implementation in Stan
  • Case study: Growth

“The scope and depth of the course is excellent.”

Online Learning

  • Weekly live sessions with tutor(s)
  • Weekly learning material (reading material, videos, exercises with model answers)
  • Bookable one-to-one sessions with tutor(s)

Textbooks

Gelman, A et al (2013) Bayesian Data Analysis, 3rd ed, Chapman & Hall/CRC Texts in Statistical Science.

Kruschke J (2014) Doing Bayesian Analysis: A Tutorial with R, Jags and Stan, 2nd ed, Academic Press.

Bernardo, J M and Smith, A F M (2000) Bayesian Theory, John Wiley & Sons.

Assessment (for credit only)

This will typically be made up of 4 pieces of assessment, including online quizzes and an individual project.

Please note that the deadline for some assessments may fall outside the teaching weeks of the course.

Software

To take our courses please use an up-to-date version of a standard browser (such as Google Chrome, Firefox, Safari, Internet Explorer or Microsoft Edge) and a PDF reader (such as Acrobat Reader). Learning material will be distributed through Moodle. We encourage all learners to install R and RStudio and we provide detailed installation instructions, but learners can also use free cloud-based services (RStudio Cloud). Learners need to install Zoom for participating in video conferencing sessions. We recommend the use of a head set for video conferencing sessions.