September 2019Vol. 20, No. 7Looking Forward to Round 4 of the Child and Family Services Reviews
Written by Jennifer Haight, supervisory children & family program specialist, Children's Bureau
As we near the end of round 3 of the Child and Family Services Review (CFSR) cycle, a process that was established through a regulation nearly 20 years ago, we have already begun planning for round 4. The purpose of these reviews is to assess the extent to which federally funded child welfare programs function effectively to promote the safety, permanency, and well-being of children and families with whom they have contact. Child welfare systems that underperform on one or more of seven specific outcomes and/or one or more of seven specified systemic factors are subject to financial penalties, which are held in abeyance as the state develops and implements a Program Improvement Plan (PIP). Approved plans are monitored during the PIP implementation and measurement period. States that have successfully executed their strategies and achieved improvement goals have their penalties rescinded.
In between each of the prior CFSR rounds, the Children's Bureau adjusted—at times significantly—the review process to improve the methods, measures, and tools used to evaluate state performance. What has remained consistent and durable is the CFSR structure used to assess state performance and to develop improvement plans. That structure was presciently embedded in a continuous quality improvement framework. Working with child welfare system partners, each state is expected to use both quantitative and qualitative evidence gathered from administrative data–—from case records and from interviews with case members and key stakeholders—to assess its performance on the outcomes and systemic factors. Buttressing their observations with further analyses that identify key challenges and point to potential solutions, states are expected to develop substantive improvement plans outlining the goals and strategies they will undertake to address the areas of concern. Joint federal and state performance monitoring during the PIP period tracks progress and allows for midcourse adjustment, when appropriate.
However thoughtful the structure of the CFSR is, some notable concerns have emerged in practice. Even after three rounds, all states have continued to incur penalties because of poor performance on most of the key outcomes and many of the systemic factors. Thus, each state continues to need to develop a PIP. A second significant concern is the often lengthy PIP development and approval process. Aggravating to everyone, this process has engendered frustration, protracted the back and forth between the parties, and has meant that approved PIPs are increasingly removed from the performance data and analysis on which they are based.
Moving forward, we anticipate adapting the CFSR and PIP processes to address both concerns. We want to see a positive system change, and we believe it is more likely to occur if we implement improved and expedient procedures. That is, we want to be better at identifying key challenges and connecting them with core causes and reasonable solutions that result in stronger outcomes and want to do that more quickly—not only to avoid the frustration that the protracted process produces but also to decrease the distance between the observation and diagnosis of the challenges and the implementation of the remedies.
While we expect to a make a number of adaptations to the full review process in round 4, we have already introduced and tested a significant change. Three states (Louisiana, Maryland, and Michigan) participated in a PIP pilot project that we introduced in round 3 to jumpstart the critical changes we want to see take place. In each of the pilot states, child welfare stakeholders came together with federal partners to work in a compressed time frame. They produced substantive PIPs that were approved within the 90-day deadline prescribed in regulation.
The pilots occurred in three stages. The central event was a multiday, onsite working session that was facilitated by professionals from the Capacity Building Centers and included a wide range of system partners (federal staff, state agency staff, legal/court representatives, community-based partners, service agencies, youth, caregivers, and parents). During the onsite session, participants worked with data that had been gathered and reviewed in the first stage (preonsite work) to highlight the key challenge areas that surfaced during the CFSR review and to begin to identify root causes associated with those challenges. Over the course of the onsite sessions, participants narrowed down root causes, focused on cross-cutting themes related to root causes, and began to identify promising strategies that would address the challenges. The third and final stage followed the onsite meeting and involved the continued elaboration of results from the onsite work and the development of the PIPs.
These PIP pilots were scheduled successively. Each pilot built on prior experiences to identify and strengthen the essential components of this approach to PIP development and to adjust those that were less successful. In the end, we arrived at a promising prototype for future PIP development and also identified three core principles that we expect to guide our approach to monitoring and oversight in general and to the CFSR round 4, specifically.
First, we note the importance of tethering program innovation to a state's central vision for its child welfare system. Each state is required to articulate its vision in their Child and Family Services Plan (CFSP), which is submitted every 5 years. The core tenets of that vision are operationalized in the programs, policies, procedures, and practices that constitute a state's approach to working with children, families, and their communities. In practice, the vision undergirds each system's theory of change—or the specific organization of a child welfare system's structures and functions to produce desired outcomes. Our oversight will consider the alignment between that vision, the processes in place to execute the vision, the quality of that execution, and the outcomes that are its result.
Second, we should seek broad involvement from child welfare system partners throughout the review process. This principle is grounded in the knowledge that child welfare systems can only be effective when all partners who have a role in it contribute to its design and operation. One of the most notable and successful features of the PIP pilots was the involvement of a wide range of system partners in the conversations that dug deeply into root causes and reached broadly for solutions. Including not only system partners (such as judicial and legal partners, private agencies, community stakeholders, and workers) but also those with lived experience in child welfare added depth and detail to the evidence used to shape decisions that will guide system change.
Third, we should gather and apply the appropriate evidence for each phase of the CFSR cycle. This was modeled during the PIP pilots but must be the defining feature of the full review cycle. In order to make accurate observations about performance as well as to accurately uncover causes and land on defensible remedies, the proper evidence must be considered. Reliance on evidence that is produced in accordance with best practices in measurement should be the cornerstone of both the CFSR and PIP processes. This expectation applies to the collection and use of both quantitative and qualitative data and will govern our adaptations to the CFSR and PIP procedures for round 4.
In fact, each of these principles is already featured in the regulations that outline the CFSR and PIP development processes. In the coming months, as we wind down round 3 and look to shape the next review cycle, we will return to these principles and develop procedures that reflect them when we launch CFSR round 4.
