Exitosa

[PLEASE NOTE: The following SOURCEBOOK text is designed for presentation as part of the Internet-based collection of materials for Evaluating Socio Economic Development, and should be viewed in this context. Introductory remarks are on the site www.evalsed.info]

FORMATIVE EVALUATION

Description of the technique

Formative evaluation seeks to strengthen or improve a programme orintervention by examining, amongst other things, the delivery of the programme, the quality of its implementation and the organisational context, personnel, structures and procedures. As a change oriented evaluation approach, it is especially attuned to assessing in an ongoing way, any discrepancies between the expected direction and outputs of the programme and what is happening in reality, to analysingstrengths and weaknesses, to uncovering obstacles, barriers or unexpected opportunities, and to generating understandings about how the programme could be implemented better. Formative evaluation is responsive to the dynamic context of a programme, and attempts to ameliorate the messiness that is an inevitable part of complex, multi-faceted programmes in a fluid policy environment.

Formativeevaluation pays special attention to the delivery and intervention system, but not exclusively. In formative evaluation, the evaluator also has to analyse the intervention logic, the outcomes, the results and impacts.

Formative evaluation activities include the collection and analysis of data over the lifecycle of the programme and timely feedback of evaluation findings to programme actors toinform ongoing decision-making and action (i.e. it is a form of operational intelligence). It requires an effective data collection strategy, often incorporating routinised monitoring data alongside more tailored evaluation activities. Feedback is primarily designed to fine-tune the implementation of the programme although it may also contribute to policy-making at the margins through piecemealadaptation.

Evaluators conducting a formative evaluation ask many different kinds of questions and use a variety of methods to address them. Questions are commonly open-ended and exploratory, aimed at uncovering the processes by which the programme takes shape, establishing what has changed from the original design and why, or assessing soft organisational factors such as the extent of ‘buy in’ bypractitioner staff to the programme’s goals and intended outcomes. . Formative evaluation questions also investigate the relationship between inputs and outcomes, which can involve the formulation and measurement of early or short-term outcome measures. These often have a process flavour and serve as interim markers of more tangible longer term outcomes.

Formative evaluation lends itself mostreadily to a case study approach, using a qualitative mode of inquiry. There is a preference for methods that are capable of picking up the subtleties of reforms and the complexities of the organisational context and wider policy environment. Methods which might be used include stakeholder analysis, concept mapping, focus groups, nominal group techniques, observational techniques and input-outputanalysis. Formative evaluation’s concern with the efficiency and effectiveness of project management can be addressed through management-oriented methods like flow charting, PERT/CRM (Programme Evaluation and Review Technique and Critical Path Method) and project scheduling. The measurement of interim or short-term outcome measures, which capture steps in the theory of how change will be achievedover the long term, may involve construction of qualitative or process indicators and use of basic forms of quantitative measurement.

Formative evaluation may be planned and managed in a variety of ways. The prevailing practice has been to prioritise the information needs of central staff (policy makers, programme managers) as those primarily responsible for programme steerage, leaving...