Development of the new framework was led by the MRC/CSO Social and Public Health Sciences Unit and supported by the Medical Research Council (MRC) and the National Institute for Health and Care Research (NIHR).

Natural experiments provide valuable opportunities for the evaluation of new and existing policies, programmes and services that may improve population health, including those where a randomised trial would be difficult to conduct.

The framework defines natural experiments as events or processes outside the control of a researcher that divide a population into groups with differing degrees of exposure.

When government policies, services or infrastructure are introduced or changed, this gives useful opportunities for evaluation of these natural experiments.

Examples include the introduction of a minimum price for the sale of a unit of alcohol in Scotland, the UK-wide soft drinks industry levy, or the database errors that led to the failure of the system for testing for COVID infections in some parts of the country.

Researchers are making increasing use of these opportunities, but definitions and methods are still the subject of debate. The framework is aimed at a broad audience including health researchers, practitioners, policy and decision makers, funders and journal editors. It intends to help:

  • researchers be aware of the range of approaches available for evaluating interventions or other exposures as natural experiments
  • decision makers in local or central government decide whether a natural experimental evaluation would be useful, and if so, of what kind
  • funders, journal editors and peer reviewers understand the strengths and weaknesses of funding proposals and articles reporting natural experimental evaluations

The framework updates and extends previous guidance by taking account of recent developments in approaches to evaluating complex interventions and showing how they can be applied in natural experimental studies. The framework explains:

  • definitions and key concepts
  • recent advances in designing and planning evaluations of natural experiments, including the relevance of a systems perspective, mixed methods and stakeholder involvement throughout the process
  • the strengths, weaknesses, applicability and limitations of the range of methods now available, including qualitative methods, economic evaluations and synthesis methods, as well as quantitative methods
  • issues of infrastructure and data governance
  • good practice considerations for different users of the framework

Developing the framework

Development of the framework was led by the MRC/CSO Social and Public Health Sciences Unit with a team of experts, an advisory group and an oversight group. The team gathered expert opinions in workshops and an online consultation with researchers from the UK, Europe, Australasia and North and South America. As well as researchers, members of research funding boards, policymakers, journal editors and representatives of national and local governments contributed views and advice.

Professor Brian Ferguson, Director, Public Health Research Programme, NIHR, said:

‘Public health research requires a range of different approaches and it is good to see this latest framework for natural experiments.  Studies using this methodology need to be robust and of high quality so that decision makers can have confidence in local and national policy evaluation’.

Professor Graham Hart, Professor of Sexual Health & HIV Research at University College London, Co-Director of UCL Health of the Public, and Chair of the MRC Applied Global Health Research Board, said:

‘Having chaired the Oversight Group that supported the development of the Framework, I was impressed at the scientific rigour and hard work that went into getting this right. This is a critical addition to the armamentarium of evaluation methods and will strengthen public health science in the policy arena.’

Professor Nick Wareham, Chair of the MRC Population Health Sciences Group (PHSG), said:

‘This new framework is very welcome. I hope it encourages the population health research community – researchers, policy-makers and practitioners – to make full use of the opportunities natural experiments provide to produce and apply the best possible evidence to support decision-making’.

Professor Peter Craig, MRC/CSO Social and Public Health Sciences Unit and lead author of the framework, said:

‘On behalf of the writing team, I’d like to thank all the people who helped us to develop the framework – our funders and advisory group, the workshop participants and everyone who responded to our consultation. I hope they’ll use the framework in their work as researchers, journal editors, research funders and public health decision-makers, and that they’ll engage with it critically so that we can continue to improve the way we all use the whole evaluation toolkit’.

First published: 28 March 2025

<< News