The impact of What Works Scotland on evaluation planning, in particular the increasing use of evaluability assessments.
What Works Scotland has had a significant impact on the approach to evaluation planning across Scotland, for example working closely with partner organisations to develop and apply evaluability assessments, such as:
- NHS Health Scotland
- Glasgow Community Planning Partnership
- Scottish Collaboration for Public Health Research and Policy
Evaluability assessments (EAs) are a novel and innovative approach to planning the evaluation of new policies and programmes. We have developed guidance, delivered training to academic researchers, practitioners and policymakers, and conducted more than 10 EAs for a wide range of national and local stakeholders, covering a diverse array of interventions.
What Works Scotland and Glasgow community planning partners worked together to develop an evaluation approach to assess the impact of participatory budgeting activities, enabling them to make evidence-based decisions on how to evaluate Thriving Places, a ten-year, area-based, anti-deprivation initiative.
The partners involved included:
- Glasgow Life
- Glasgow City Council
- NG Homes
- NHS Greater Glasgow and Clyde
- West of Scotland Housing Association
- Scottish Fire and Rescue
- Glasgow Council for the Voluntary Sector
- Police Scotland
- Glasgow Centre for Population Health
- New Gorbals Housing Association
- Jobs and Business Glasgow
Based on interviews with participants from a range of public and third sector organisations, our evaluation of this EA initiative showed that the process enabled them to discuss issues that were rarely addressed otherwise. Participants reported that the Theory of Change identified through the EA had clarified their own understanding of what Thriving Places was aiming to achieve, and provided a resource they could use in their own practice.
Evidence of impact/changes
Evaluability assessments have informed the design and commissioning of evaluations of the Family Nurse Partnership, the introduction of free school meals for all children in P1-3, and the Enhanced Health Visiting Pathway.
Evaluability assessment is becoming an established part of the planning of evaluation by the Scottish Government. EAs have been commissioned of the Community Empowerment Act (Parts 3 & 5) and subsequently influenced approach to developing the evaluation of Part 2 (ongoing); the Fair Start Programme and Scotland’s Baby Box; and EAs are being included in the specification of requirements for the evaluation of major policy initiatives such as the Basic Income pilots and Self-Directed Support.
Interest in EA is now growing elsewhere in the UK. We have contributed guidance on EA to an online evaluation resource for practitioners, published by Public Health England. With Health Scotland we are developing proposals for a UK-wide EA collaboration, including Public Health England and Public Health Wales.
The Participatory Budgeting Evaluation Toolkit was highlighted in the Audit Scotland report Local government in Scotland: Challenges and performance 2018 (page 14) published in April 2018.
See all of our resources on evaluation approaches
