September 2012Vol. 13, No. 8Utah's Qualitative Case Review

In 1999, Utah's Division of Child and Family Services (DCFS) was embroiled in a lawsuit, resulting in a settlement agreement that required a new evaluation of child welfare practice. That new evaluation became the State's Qualitative Case Review model that evaluates outcomes for children and families and practice model fidelity. For the past 12 years, Utah's rigorous evaluation of child welfare practice has created a culture change benefiting children, families, and child welfare staff statewide.

An independent evaluator, the State's Office of Services Review (OSR), conducts the Qualitative Case Reviews (QCRs). Each year, OSR administers a total of six reviews, ranging from 20 cases for the smallest region to 50 cases for the largest metropolitan area, totaling 150 cases. "One hiccup with the evaluation tool prior to the QCR," said Kristin Lambert, Services Review Manager at OSR, "was that it was compliance based. In the first years of the settlement agreement, the Division was not doing better on the compliance review. Improvement was happening, but the measurement tool wasn't showing it." OSR then shifted to an interview-style evaluation involving service recipients, moving from a compliance-driven to an outcome-based evaluation. "It mattered less if the case plan was signed on time. What mattered was whether the services provided resulted in better outcomes for children and families."

QCRs are conducted similarly to the Federal Child and Family Services Reviews (CFSRs). A team of two review the case and interview key stakeholders—including birth and fosters parents, teachers, therapists, caseworkers, lawyers, mentors, and more. Reviewers are drawn from OSR, DCFS, and community partners. Child and family status is assessed across eight indicators, including safety, stability, permanency, behavioral and emotional well-being, family connections, and satisfaction. Child welfare practice is evaluated across seven indicators, including engagement, teaming, assessment, intervention adequacy, and tracking and adaption. If QCR indicators decline below standards, DCFS must create an action plan to improve practice. All annual QCR reports are posted on the OSR website.

The first full QCR was conducted in 2000. Between 2000 and 2010, DCFS scores on child and family status increased from 78 percent to 89 percent, with a decline from 96 percent in 2007. System performance scores increased from 42 percent in 2000 to 89 percent in 2010, with a slight decline from 93 percent in 2009. Linda Wininger, Director of Programs and Practice Improvement, said the QCR can be used for more than performance evaluation. "It is a way to measure the broader child welfare system. When we look at outcomes for children and families, and we see a track record of improvement for 10 years and then 2 years of decreased scores, we can't ignore that. We were able to use QCR results as a part of a presentation to ask for more caseworkers so that we could once again provide the level of service that we had in the past. That's a compelling way to use this information."

Utah's QCR and its Practice Model are closely aligned. In fact, the QCR was used as a foundation for building the practice model. Jeff Harrop, Practice Improvement Coordinator at DCFS, said, "Our practice model was designed based on what we thought a great child welfare system should look like. We built a system that supports great practice, and the QCR informed that process."

The QCR has become part of the DCFS culture. While there have been several new directors over the past 12 years, most have been certified reviewers and all have been dedicated to the QCR process. DCFS has taken foolproof measures to ensure sustainability. The QCR is now required under State law. "With budget cuts, something like this could be axed. We see the value in this type of review, and we know we need to keep sharpening our tools," said Aude Bermond Hamlet, Practice Improvement Coordinator.

When asked for advice to States or counties interested in implementing a QCR, Lambert noted the importance of the balance between costs and benefits. "You can have the most helpful and beneficial review system, but if it's too demanding on resources, if it's too burdensome or complex, or if it tries to measure everything, it won't survive." DCFS welcomes administrators from other States to take part in the QCR process as shadow reviewers.

The QCR process was the subject of a December 2011 webinar for the National Resource Center for Organizational Improvement's (NRCOI's) Practice Model Peer Network. A recording of that webinar and multiple materials pertaining to the State's QCR, OSR, and more, are available:

Special thanks to Linda Wininger, Director of Programs and Practice Improvement, Aude Bermond Hamlet, Practice Improvement Coordinator, Jeff Harrop, Practice Improvement Coordinator, and Brad McGarry, Director of Office of Services Review, and Kristin Lambert, Services Review Manager with the Offices of Service Review, for providing information for this article.

Related Items

Children's Bureau Express featured the Practice Model Peer Network in the article "NRCOI Practice Model Peer Network" (August 2012).

Children's Bureau Express also highlighted Utah child welfare practice in the articles "Practice Models in Child Welfare" (March 2008) and "Child and Family Team Meetings Improve Utah Outcomes" (September 2010).