June 2019Vol. 20, No. 5Toolkit Offers Guidance for Evaluating Programs and Early Outcomes
A toolkit prepared for the Children's Bureau offers step-by-step instruction for evaluating the initial implementation of social programs and their early outcomes. It is primarily intended for child welfare agencies and professionals to use in partnership with their evaluators.
The toolkit introduces the concept of formative evaluation, a method to evaluate programs during the earliest phases of implementation to inform and guide program improvement and determine readiness for more rigorous summative evaluation. The authors explain in the introduction that very few evaluations of social programs have shown positive results, perhaps due to program teams conducting summative evaluations of programs before they are ready for such assessments. They also note that in child welfare, program implementation is a process that takes place over time. The toolkit outlines four overlapping phases of implementation: exploration, installation, initial implementation, and full implementation.
The toolkit consists of three sections:
- Understanding Formative Evaluation explains the difference between formative and summative evaluation and how formative evaluation is important in the evidence-building process. For example, formative evaluation occurs during a program's initial implementation, while it is still being developed, whereas a summative evaluation occurs during full implementation when a program is fully functioning and no additional adjustments are anticipated. A formative evaluation seeks to answer whether the key points of the program are functioning as planned and whether early outcomes are moving in the right direction, whereas a summative evaluation seeks to answer whether the program had the intended effects on its target population and whether the outcomes can be attributed to the program.
- Meeting Preconditions of Formative Evaluation asks questions of program evaluators and provides resources to help them approach formative evaluation.
- Conducting Formative Evaluation offers program evaluators guidance on how to design and conduct such an evaluation and how they can use their findings to adjust and improve the programs and determine when it is appropriate to conduct a summative evaluation.
The toolkit makes use of a fictional agency to provide relevant examples. It also provides a glossary of key terms and additional resources.
Formative Evaluation Toolkit: A Step-by-Step Guide and Resources for Evaluating Program Implementation and Early Outcomes is available at https://www.acf.hhs.gov/cb/resource/formative-evaluation-toolkit.
