September 2021Vol. 22, No. 8Lessons Learned From the Child and Family Services Review Rounds 1 Through 3

Written by Jennifer Haight, director, Division of Performance Measurement and Improvement, and Linda Mitchell, senior child welfare specialist, Child and Family Services Review Unit, Children's Bureau

Over the past 20 years, the Children's Bureau (CB) has moved federal monitoring of titles IV-B and IV-E from a checklist approach of state compliance with federal requirements to a more comprehensive examination of the fundamental practices and systems processes that lead to better outcomes for children and families being served by state child welfare agencies. These monitoring refinements are embedded in the Child and Family Services Review (CFSR) processes and are featured in our collaboration with states on the development of their Program Improvement Plans (PIPs). Additionally, our efforts to frame federal monitoring as an aspect of our own continuous improvement has meant that we have been able to use the knowledge collected through these monitoring activities to support CB's broader work.

As we head into CFSR round 4, we want to first highlight some lessons from the preceding rounds that may be useful to consider when contemplating the next CFSR/PIP cycle. Secondly, we briefly describe the ways in which the knowledge gained from prior CFSR/PIP rounds has informed a wide range of CB programs and initiatives.

The following are six key lessons learned from prior rounds of the CFSR. As states enter the planning phase for round 4, we recommend keeping these observations in mind:

Continuous quality improvement (CQI) is the underlying foundation of the CFSR. Establishing a comprehensive CQI framework and cultivating the development of state CQI capabilities and capacities has helped states move forward to more meaningful system and practice changes over time.
Carefully developed data evidence should be used more consistently and effectively throughout performance improvement activities. This includes assessing the state's functioning when completing the statewide assessment; identifying underlying practice and systems issues that contribute to challenges in meeting PIP measures; and maintaining post-PIP monitoring to insure continued improvement in practice, systems change, and outcomes.
Innovation and change can only occur if there is effective leadership at all levels of the agency. Leadership should maintain focus on improving organizational culture and climate, communicating expectations to field staff and stakeholders, and advocating with governors and legislators for the resources needed to fully implement and sustain program and system improvements. It is critical for leadership to be engaged in all phases of the CFRS/PIP cycle.
Interviews with case participants provide valuable information on how the agency engages with them to assess their needs, provide services they believe are helpful to them, and ensure connections important to them are maintained. Data from these interviews can usefully augment the knowledge that informs each phase of the CFSR/PIP cycle and help agencies understand the impact of practice change on the families they serve.
Engaging stakeholders is necessary to provide insight on how the system is functioning in key areas that may impact outcomes. Stakeholder experiences with the system can help illuminate strengths and barriers in meeting the needs of families, and their thoughts on ways to address the gaps and barriers can be integrated into both the PIP and the title IV-B Child and Family Services Plan.
Training strategies in and of themselves do not routinely result in observable and sustained systems improvements. However, developing and delivering training designed to support the implementation of specific PIP strategies, new practices, and improvements to system functioning is critical to effective innovation and system improvement.

Incorporating these lessons into the work of CFSR round 4 will support ongoing integration of the federal monitoring responsibilities with our collective interest in building the evidence base that can inform CB's programs and priorities. The CFSR has become more than a federal monitoring process. It builds knowledge across the field of child welfare. As federal and state partners continue to identify best practices in the design and delivery of child welfare systems, that knowledge fuels federal innovation in the development of programs, policies, and opportunities that continue to promote improved outcomes for children, families, and communities. Many of the lessons learned above are routinely integrated across CB's work. The following are some examples:

CB uses CFSR findings to develop discretionary grants that support a wide range of initiatives intended to spur system and practice improvements that will result in measurable improvements for children and families.
The training and technical assistance/capacity-building network is structured to be more responsive to states' identified needs and to assist states in generating and acquiring evidence necessary to support decision-making at all phases of the performance improvement process.
CB regularly engages with youth, families, foster/adoptive parents, and key community stakeholders to solicit input and feedback in the development and implementation of CB programs and policies.
Workforce issues are continually identified through CFSR and PIP efforts and highlighted and strengthened through National Child Welfare Workforce Institute projects and resources.
CB emphasizes the importance of agency and legal/judicial collaboration around common goals to improve services to families and facilitates opportunities for joint agency and court efforts in all areas of our work.
The CB data team regularly develops and disseminates resources to inform partners and stakeholders about child welfare system dynamics and incorporates their use across CB divisions and operations.

Over time, we expect the CFSR/PIP cycle to continue to strengthen monitoring and program improvement within the larger framework of shared accountability and CQI. Establishing a comprehensive foundation for CQI has helped states move forward to implementing more meaningful systems and practice changes over time. On the federal end, CB has built its capacity to provide more data and information to states through both the national data profiles, including individual state context data, as well as additional technical assistance resources for data analysis. State, community, and other federal partners can expect to see more attention on how to use the information to assess states' functioning when completing the statewide assessment and in developing targeted PIP goals and strategies, identifying underlying practice and systems issues contributing to challenges in meeting PIP measures, and ongoing monitoring to ensure continued improvement in practice and systems changes. In addition, CB will continue to emphasize the importance of stakeholder involvement and engagement of state leadership so that improvements made through the CFSR/PIP process are successful and integrated into practice for lasting change.