Evaluation
It is important to evaluate all public or community engagement activities to monitor their success, demonstrate impact and discover ways to improve. Evaluation methods should always be tailored to the audience engaged.
Why should you evaluate your public engagement?
Evaluation is about proving and improving. Evidence from evaluation can be used to support your personal and professional development, from inclusion in P&DR to promotion criteria and award applications. It can help you better understand your audience and ensure that you are making a difference.
What should you evaluate?
When assessing impact, you should look to evaluate:
1) Reach - how widely has the impact been felt, and by whom? e.g. geographical spread, demographics, number of attendees, number of events
2) Significance - what's the depth of your impact? What has changed as a result of your engagement? e.g. attitudes, knowledge, understanding
What impacts might public engagement have?
Public engagement can have short and long term impacts on both the researchers and public. Examples of impacts include changes to:
- Knowledge or understanding
- Attitudes, enjoyment
- Enhanced or new interest
- Inspiration or creativity
- Improved skills or confidence
- Improved performance
- Access or participation
- Improved wellbeing
- Behaviour change - will people behave differently after the engagement? Will they attend future activities?
- Policy or practice
Annex A in the REF 2020 Panel Criteria & Working Methods contains a list of potential impacts arising from public engagement.
Many of these impacts will be short term. It's also useful to consider whether there is potential for long-term impacts from your project, and how you can capture evidence of this e.g. can you follow up in 6 months?
Planning your evaluation
Evaluation should be embedded as part of your project planning, not an after thought! There are a range of tools available to help plan your evaluation. Logic models, for example can be a useful way of informing your evaluation plan.
- Evaluating impact planning template (login required)
- Evaluation training slides (login required)
- Questionnaire design - getting started (login required)
- Questionnaire design - example questions (login required)
- STFC Evaluation Framework
- NCCPE guidelines
- Evaluation Support Scotland
Evaluation methods
There's a wide range of evaluation methods. Make sure that the methods you select are appropriate to your audience and the environment. A two-page questionnaire may not be appropriate at a music festival, for example. The language used in a questionnaire for adults may be inaccessible to children. Below are a range of methods you may wish to consider:
- Questionnaires - paper, or online (e.g. Online Surveys, or embedded in booking pages like Eventbrite)
- Voting - with tokens/buttons, stickers
- Online voting - using Smartphones for live audiences e.g. Slido, Kahoot
- Graffiti wall - chalk board, post-it notes, for qualitative feedback
- Comment cards, guestbooks
- Interviews/focus groups, voice recordings
Remember: "No stats without stories, no stories without stats." You should aim to collect both quantitative and qualitative data.
Remember: don't over-evaluate!
Evaluation shouldn't tarnish or take longer than the engagement experience. Every question should be there for a reason, and map onto your objectives.
More evaluation does not mean better evaluation! Consider your audience and the user experience. Is there a way of weaving evaluation into your activity, or making it enjoyable for the participant?
Need some help?
I'm happy to offer evaluation training for your group, or one-to-one advice on your evaluation plans. Feel free to get in touch: zara.gladman@glasgow.ac.uk
Resources to support your evaluation, including equipment for loan is available. Find out more.