April 2018Vol. 19, No. 3Using a 'Road Test' to Improve Human Services Programs
The Office of Planning, Research and Evaluation, which is within the Administration for Children and Families of the U.S. Department of Health and Human Services, created a brief to explain how to use a "road test" for improving human services programs. While pilot testing is a common practice, a road test uses rapid prototyping by implementing a new strategy in multiple, short learning cycles in contained practice settings and using feedback to improve the strategy's design for the next implementation cycle. This allows programs to make necessary adjustments to fit their particular needs. The brief also provides practical guidance and concrete examples of road tests.
A successful road test is partially dependent on a three-phase process known as LI2, or Learn, Innovate, Improve. While the road test occurs during the Improve phase, it is important to note that LI2 can be nonlinear, depending on the goals of the program. Guidance for a successful road test include the following:
- Commit to active leadership and efficient coordination
- Anchor the road test in a detailed plan (i.e., a "road map")
- Create clear learning questions to focus each cycle of a road test
- Articulate a timeline to foster staff buy-in and ensure everyone is on the same page
- Design a process to collect feedback in a way that aligns with the learning questions and can feasibly be implemented
- Analyze the feedback for common threads and themes after each learning cycle
- Share and discuss the insights from each road test learning cycle
It is important to keep in mind that a road test focuses more on the quality of strategy implementation as well as the strategy's usability and feasibility and less on the impact. Additionally, it is not a replacement for an indepth research study. Programs that do not have a strong internal design and analysis capability could partner with external researchers, which could help build the programs' capacity to use LI2.
Using a "Road Test" to Improve Human Services Programs is available at https://www.acf.hhs.gov/sites/default/files/opre/road_test_brief_final_b508.pdf (179 KB).