Evaluation Methodology

The methodology for the GOAL evaluation is shaped by the complexities of the project design, namely the facts that:

  • GOAL is multi-site (12 ‘sites’ or locations, that is, two in each of six countries) and multi-organisational.
  • GOAL has multiple objectives.
  • GOAL is predicated on cross-organisational collaboration.
  • Each partner country has its own unique context and target groups (and target numbers to achieve).
  • Programme resources are finite, and should be primarily focused on the interventions rather than the evaluation.

For these reasons it was neither feasible nor advisable to conduct an experimental or quasi-experimental evaluation involving comparison groups. Instead the evaluation draws on two theoretical frameworks: the ‘Theory of Change’ and Realist Evaluation approaches.

 

Theory of Change approach specifies what programme implementers believe will happen as a result of the intervention, and what processes are required. The articulation of the underlying programme theory serves as the foundation for the initiative’s operation as well as its evaluation. Realist evaluation approaches emphasise the central importance of programme contexts and programme mechanisms, and focuses on the interplay between these two factors, and their combined impact on outcomes.

The evaluation is integrated within the project throughout its lifespan, not only at programme level, but also at policy level: an important element of the evaluation’s multilevel approach is the description and assessment of the policy processes that play a role in influencing programme success. It is hoped that this dual focus on programme-level and policy-level processes, and their interaction, will provide useful evidence for a range of policymakers working in complex fields.

Evaluation data are being gathered via:

  • client monitoring data (to establish baseline, ongoing and exit data)
  • client satisfaction and outcome data (user survey and qualitative interviews)
  • programme and policy data (literature review; needs and strengths analysis)
  • case studies of programme sites (qualitative interviews, document analysis, analysis of quantitative data)
  • qualitative interviews with policy actors.

The evaluation includes: a) ongoing data collection (throughout the life of the project) and b) wave-specific data collection (wave 1 and wave 2).