Children's Bureau ExpressJune 2019 | Vol. 20, No. 5

Table of Contents
 

Spotlight on Child Welfare Workforce Development
This issue of CBX features workforce development and ways the child welfare workforce can increase its capacity while maintaining self-care. Read a message from Jerry Milner, associate commissioner of the Children's Bureau, that emphasizes the need for child welfare professionals to empower, support, and respect the families they serve. The issue also includes a variety of resources and publications aimed at strengthening and supporting the child welfare workforce as it performs its often challenging job.

  • Becoming a Community for Strengthening Families: The Words We Use
  • Why Workforce Development Matters: Child Welfare Workers Making a Difference Day by Day
  • Georgia’s Multi-Disciplinary Child Abuse and Neglect Institute
  • Conducting Comprehensive Workforce Needs Assessments in Child Welfare

News From the Children's Bureau
Listen to a Child Welfare Information Gateway podcast that features three former caseworkers and their perspectives on child welfare work, read an article about effective professional development and training interventions for home-based child care providers, and find the latest updates to the CB website.

  • Podcast Highlights Key Casework Skills for Successful Child Welfare Practice
  • Professional Development for Home-Based Providers
  • CB Website Updates

Child Welfare Research
We highlight a study on field training experiences for newly hired child welfare workers as well as an evaluation of a trauma-informed self-care measure for child welfare professionals.

  • Study Shows Importance of Practical Field Training, Mentorship for Newly Hired Workers
  • Trauma-Informed Self-Care Measure for Child Welfare Workers

Strategies and Tools for Practice
This section of CBX offers publications, articles, reports, toolkits, and other resources that provide either evidence-based strategies or other concrete help to child welfare and related professionals.

  • Steps to Creating, Maintaining a Stronger Child Welfare Workforce
  • Toolkit Offers Guidance for Evaluating Programs and Early Outcomes
  • Testing and Piloting an Intervention for Implementation Success

Resources
This section of CBX provides a quick list of interesting resources, such as websites, videos, journals, funding or scholarship opportunities, or other materials that can be used in the field or with families.

  • Real Stories From the Field
  • New Toolkit Helps Child Welfare Leaders Promote Family First Law

Training and Conferences

  • Supervision Matters Training
  • Conferences

Spotlight on Child Welfare Workforce Development

Becoming a Community for Strengthening Families: The Words We Use

Written by Jerry Milner

At the Children's Bureau, we've been thinking quite a bit about the words we use in child welfare and what they convey. We think the words we use are important and carry connotations about how we perceive our work, the families we serve, and the manner in which we practice our crafts. We think words reflect beliefs and can affect the way we behave. We also think language is a powerful vehicle for creating and sustaining the mind shift necessary to reshape child welfare in the United States.

At our National Conference on Child Abuse and Neglect, I recently challenged attendees and the field—including the Children's Bureau—to consider stopping the use of certain words and to start to use other words. The words I requested that we stop using included foster child, birth parent, foster parent, client, and consumer. I challenged all of us to replace those words with words of empowerment, words that communicate respect, and words that remind us of our common humanity. I would like to add case to that list and to extend the challenge again.

Rather than foster child, consider using a child living in foster care.
Rather than birth parent, consider using parent(s).
Rather than foster parent, consider using resource parent or family.
Rather than case, client, or consumer, consider using the name of the human being.

The reason for the charge is simple. Words matter. Words reveal how we view our work and those with whom and for whom we work. Words describe the culture of where and how we work. Words can convey empowerment and respect or disempowerment and difference. Words can desensitize us to the often harsh realities of those we serve and stand in the way of treating people with dignity and respect. In order to stay relationally connected with the children, youth, and families we serve, we should carefully consider the words we use to describe them and their circumstances.

When I worked in the Alabama child welfare system many years ago, we began changing the words we used in our day-to-day work. As I recall, it did not happen as part of a planned effort. Rather, it happened spontaneously as we became clearer on the important values we held and wanted our work to reflect those values. As part of implementing a statewide overhaul of the child welfare system due to a class action lawsuit and moving toward a family-centered approach to child welfare, we found that the words we had traditionally and habitually used did not match the underlying values of our new way of work. In turn, we made conscious efforts to discontinue using terms that were not strengths based and family supportive, such as dysfunctional families, problematic families, clients, and cases.

Partly because of that simple shift in terminology, coupled with broad retraining of the workforce to view families differently, we changed our perception of what our roles were in relation to the families we served. When we spoke of them and to them in more humanistic terms, we were better able to form the helping relationships that underlie the field of child welfare. Gradually, the expectations of our reformed system required clear demonstrations of the value of dignity, respect, and respectful language within our child welfare workforce. 

The results? Family team meetings became more inclusive and effective, family time for parents and their children in foster care became more sensible and supportive of their relationships, and time spent between social workers and family members became more supportive and encouraging. Oh, and children were safer, if our data were any indication.

Getting to that place was not solely attributable to changing the words we used. It was necessary to change the culture of the system to do that, but language is a major part of any culture. We could not logically practice one way and talk in contradictory terms.

Ours is a difficult field of service. It is emotionally demanding and can be physically dangerous. While it can be greatly rewarding, it can also be incredibly stressful. It is not a materially rewarding field. We see others' trauma and experience our own. And we must sometimes deal with tragedy. 

I believe that we are driven to do the work we do because of the value we place on human life and human beings. In the midst of sometimes witnessing the worst that humans can inflict on each other, we also have the privilege of seeing families and children challenge their difficult situations and win. Our commitment to our values defines and sustains us as a workforce, and the words we use should reflect those values accurately. 

As the child welfare workforce for the United States, we can drive the changes that are needed in our system to reach and support families before they encounter insurmountable difficulty and relive the trauma that often travels with them over generations. We can protect children and respect the integrity of critical relationships that, if nourished and supported, can lead to healthier adults.

Choosing words that reflect these inherent values that we all share can go a long way in identifying ourselves as a community whose mission is to strengthen families and nurture healthy children. It can better position us to stay connected to our mission and treat families and each other with the respect and dignity we all deserve. It can help define the culture of the work and the work places we inhabit. Choosing words that identify our partners—and us—as a community that is focused on a common vision will position us to do what all healthy communities do—care for one another.
 

Issue Date: June 2019
Section: Spotlight on Child Welfare Workforce Development
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5340


Why Workforce Development Matters: Child Welfare Workers Making a Difference Day by Day

Written by Katharine Briar-Lawson, professor, National Child Welfare Workforce Institute, and Mary McCarthy, coprincipal investigator, National Child Welfare Workforce Institute

Across this country, thousands of child welfare employees go to work each day thinking about the day ahead and how they can best support families who may be struggling with violence, depression, trauma, substance use, homelessness, and poverty. These women and men find the work of engaging families and children through child welfare systems deeply compelling and meaningful. Few jobs and careers bring as much opportunity for using finely honed skills, community networks, and teamwork to support healing for families and children. In fact, practitioners may find such work constitutes the pinnacle of their careers.

However, there is a paradox here. Although child welfare work is important and highly impactful, the fact is that some child welfare agencies struggle to recruit caseworkers and lack sufficient systems to ensure their safety, retention, and well-being. Agencies and workers are confronted with demanding caseloads and are often tasked with making life-and-death decisions about safety and risk, traumatic stress, and public oversight and scrutiny. Staff turnover, which is sometimes as high as 40 percent a year, is more than a fiscal issue for an agency; it has a direct impact on case outcomes and to the safety, permanence, and well-being of children and families. Caseworkers and supervisors who experience secondary trauma and insufficient support may carry scars, like the children and families they serve.

Consequently, workforce development is much more than a recruitment and hiring issue. Reducing turnover increases the chances of stability for families and permanency for children, improves the timeliness of investigations, ensures greater contact between children in care and their families, and supports effective service delivery (National Child Welfare Workforce Institute [NCWWI], 2018). It facilitates a child welfare system that can focus on supporting families and providing resources that help restore family functioning and promote child and family well-being.

Workforce development is a top priority for the Children's Bureau. In fact, since its inception in 1912, workforce development efforts that recruit and support a strong and skilled cadre of caseworkers and supervisors to serve the most vulnerable have been a consistent focus of the Children's Bureau. So how do we begin to understand and address the factors that underlie the workforce crisis currently impacting our child welfare systems? One way is by clarifying for the general public what child welfare workers do. When asked, the general public may say that child welfare workers take children away from their parents or accuse parents of being unfit to care for their children. While it is true that there are times when children may need to be brought into care to protect their immediate safety, the vast majority of families are engaged with the child welfare system due to child neglect, not abuse. Since 2017, NCWWI has been working with the Children's Bureau to educate the public and celebrate child welfare work being done across the country. September has been designated as National Child Welfare Workforce Month, and efforts to publicize this event nationwide will be a focus of NCWWI in the coming years. 

Other focal areas within tribal, voluntary, and public child welfare agencies are the importance of organizational culture and climate, including workload and work conditions, workforce competencies and skill sets, university-community partnerships that support workforce preparation, career ladders, ongoing professional development, high-quality supervision, and inclusive management. Many states have been using title IV-E funds for years as an important workforce development strategy to support social work students and child welfare staff returning to school. At least 34 states have nurtured these robust partnerships between schools of social work and agencies. NCWWI has been working intensively with jurisdictions and tribes across the country on leadership training, university-agency partnerships, education and transitions to work for new employees, and organizational health.

Currently, NCWWI is working to foster partnerships with eight jurisdictions, tribes, and schools of social work in a systemwide, outcome-focused change effort. These include a systematic assessment of the health of an organization through a comprehensive organizational health assessment. The findings of the assessment will be used to develop and implement an organizational intervention, embed leadership training into agency training systems, work in partnership with a school of social work to prepare graduates for child welfare practice using tailored educational programs and long-term supports for the transition to work, and inform an embedded continuous quality improvement process as well as evaluation data to measure progress and assess outcomes.

The Quality Improvement Center for Workforce Development (QIC-WD), another initiative funded by the Children's Bureau, is engaged in research on effective strategies to attract and retain child welfare workers. Its partnerships with eight jurisdictions and tribes will provide important knowledge to guide the development of evidence-based and evidence-informed recruitment and retention strategies. Its team is also examining research from multiple fields and collecting information about workforce trends and what works in areas including staff recruitment and retention as well as agency culture and climate (QIC-WD, 2019).

The national conversation about the child welfare workforce and the urgency to "create the conditions for strong and thriving families and communities where children are free from harm" (Milner, 2019, para. 10) reinforces the message that meaningful, effective work with vulnerable children and families is coveted as a career option by many. Developing more preventive services, including initiatives through the 2018 Family First Prevention Services Act, and utilizing practitioners' family-centered and culturally competent skills earlier on can help families before their stress and impoverishment take an often irreversible toll. Moreover, the child welfare system's expanded focus on prevention will also bolster services, programs, and collaborations in the health care, legal, education, child care, early childhood education, and social services fields.

Attention to primary prevention and early intervention is a community issue and requires concerted, collective efforts among all stakeholders to build systems of family support, capacity building, and community-based child protection. These intentional efforts, fostered and supported by the Children's Bureau, offer a strong foundation to support our current and future workforce, the essential component for achieving excellence in our child welfare system.

References

Milner, J. (2019). "Strong and thriving families and communities are our best prevention strategy." Children's Bureau Express, 20(3). Retrieved from
https://go.usa.gov/xmE2U.

National Child Welfare Workforce Institute. (2017). Why the workforce matters [infographic]. Retrieved from http://ncwwi.org/files/Why_the_Workforce_Matters.pdf (130 KB).

Quality Improvement Center for Workforce Development. (2019). An overview of QIC-WD literature review [blog post]. Retrieved from
https://www.qic-wd.org/blog/overview-qic-wd-literature-review.
 

Issue Date: June 2019
Section: Spotlight on Child Welfare Workforce Development
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5342


Georgia’s Multi-Disciplinary Child Abuse and Neglect Institute

Written by Judge Michael Key and Jerry Bruce, colead faculty for the Multi-Disciplinary Child Abuse and Neglect Institute. Judge Key is the juvenile court judge for Troup County, Georgia, and a past president of the National Council of Juvenile and Family Court Judges. Jerry Bruce is a former juvenile court judge and the director of Georgia's court improvement program.

We all talk a lot about system change. In fact, our child welfare legal system can be described as a system that is about system change in families. And anyone with experience in this system understands that it is very difficult to create the conditions for change in families if everyone in the family has not bought in. We've all experienced the frustration of trying to work on change in a family dynamic when one or more of the necessary parties are not present. We're frustrated because we know by experience what's needed for successful change in families: an honest assessment of strengths and weaknesses, a plan (developed with participation by all parties) that sets forth realistic and attainable goals, a data-driven measure of progress, and mutual accountability. Silos are for storage, not for growth.

Yet, we accept siloed approaches as the norm in our work on our own child welfare legal system. We have conversations and conduct training in a sort of isolation. There are trainings for judges, trainings for attorneys, trainings for agency staff, and trainings for court-appointed special advocates (CASAs) and guardians ad litem (GALs). Each constituency has its own concerns, its own goals, and its own way of measuring success (usually more by intuition than by evidence).

As the major organizer and funder of statewide trainings in Georgia, the Georgia Supreme Court Committee on Justice for Children (our state court improvement program), in consultation with the judges who make up our court improvement initiative, made a decision to move away from supporting siloed training toward a multidisciplinary approach with a univocal best-practices message. One of the major components of this new approach is our Multi-Disciplinary Child Abuse and Neglect Institute (MDCANI).

Modeled on the Child Abuse and Neglect Institute of the National Council of Juvenile and Family Court Judges, MDCANI is an intensive, 2-day training on every aspect of a dependency case and is wholly informed by nationally accepted best practices. The first phase of MDCANI deals with the first 75 days of a dependency matter, from the decision to remove a child through the initial review hearing. The second phase (now in the planning stage) will address the issues that arise from initial review through the final order closing the case.

Because many people throughout the state find it difficult to come to Atlanta or other centralized trainings, we bring MDCANI to the local jurisdictions to ensure we have involvement from all classes of stakeholders. Ideally, we train two jurisdictions together so even the local practices of a particular jurisdiction are open to discussion. We have judges, attorneys, agency staff, CASAs, GALs, and other professional participants together in the same room for 2 days of intensive training and conversation.  

We begin our discussion with an overview of federal legislation and the foundational constitutional liberty interest in family integrity. Reference is made to this material throughout the training. We also present detailed data that take us beyond what stakeholders think about how they're performing by looking objectively at local and statewide outcome measures. Because we can tell participants what their data are, but not why their data measures look like they do, this module forms the basis for frank conversations about local practice. Because we have everyone present in the same room, there is a high level of mutual accountability. At the close of the training, we ask participants to identify steps to improve practice in their jurisdictions.

Since the fall of 2017, we have provided MDCANI to jurisdictions representing just over 50 percent of our statewide population. And although we have not had sufficient time to do a thorough study of outcomes in these jurisdictions, we hear from participants that these trainings are changing practice in their courts. 
 

Issue Date: June 2019
Section: Spotlight on Child Welfare Workforce Development
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5343


Conducting Comprehensive Workforce Needs Assessments in Child Welfare

Written by Megan Paul, University of Nebraska-Lincoln, Center on Children, Families and the Law, and Courtney Harrison, CLH Strategies and Solutions

Staff recruitment, selection, and retention are persistent challenges for many child welfare agencies. Agency directors recognize these challenges but often lack the capacity or strong relationships with human resources (HR) to hire and keep the right child welfare staff. The lack of evidence-based practices to address child welfare workforce issues compounds the situation. The Quality Improvement Center for Workforce Development (QIC-WD) was created through a 5-year cooperative agreement with the Children's Bureau to advance knowledge about evidence-informed workforce interventions, test a variety of interventions in real-world settings, and examine how workforce stability is related to outcomes for children.

The QIC-WD partnered with eight child welfare agencies (including state, county, and tribal systems) to test the effectiveness of promising workforce interventions. Each site identified an implementation team made up of agency leadership, caseworkers, supervisors, HR professionals, quality assurance/continuous quality improvement staff, and training administrators. The QIC-WD provides workforce, implementation, and evaluation specialists, known as a WIE team, to work with each site.

The QIC-WD created a continuous workforce development process (adapted from the implementation and evaluation approach developed by the Permanency Innovations Initiative) to guide its work selecting, implementing, and evaluating workforce interventions. The first phase of this process is exploration, which includes a comprehensive workforce needs assessment to identify a specific workforce problem, who it affects, why it exists, and barriers or facilitators to addressing the problem; a theory of change for how to improve it; and an intervention to address it. Exploration is followed by the installation, initial implementation, and full implementation phases. Each phase includes significant teaming and evaluation components.

Many types of data are used to inform the assessment, such as HR recruitment, hiring, and retention metrics; agency records, such as exit surveys or past evaluation studies; organizational culture and climate survey results; and data on agency readiness for change. The WIE teams work with each site to provide context and additional qualitative information not available through existing data sources.

The QIC-WD learned a variety of valuable lessons about this process:

The assessment is only the first step in a continuous workforce development process, yet it is crucial because it is the foundation for the implementation and evaluation activities that come next. For more information visit the QIC-WD website (https://www.qic-wd.org/) or attend the Children's Bureau Evaluation Summit in August 2019.

For more information, contact Michelle Graef, project director for the QIC-WD, University of Nebraska-Lincoln, Center on Children, Families and the Law at mgraef1@unl.edu.

 

 

 

 

 

Issue Date: June 2019
Section: Spotlight on Child Welfare Workforce Development
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5344


News From the Children's Bureau

Podcast Highlights Key Casework Skills for Successful Child Welfare Practice

A Child Welfare Information Gateway podcast explores the skills needed for successful child welfare casework by featuring three former caseworkers who discuss the realities of real-world practice and how they managed their time, energy, and emotions while working with children and families.

The podcast, "Casework: What It Really Takes," seeks to answer the following questions for new and prospective caseworkers:

The caseworkers, who are now Information Gateway staff members, discuss the challenges of working with individuals who may have very different perspectives and backgrounds and how to maximize the partnerships needed to serve children and families more effectively. They share tips from their own experience on how to approach families, manage caseloads, maintain professionalism, and develop useful cross-agency partnerships. They also share tips on how supervisors can support new workers. One of the participants emphasizes, "It's okay not to be perfect," while another talks about the importance of taking it "a day at a time."

The podcast is available on the Children's Bureau website at https://www.acf.hhs.gov/cb/resource/child-welfare-podcast-casework.
 

Issue Date: June 2019
Section: News From the Children's Bureau
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5345


Professional Development for Home-Based Providers

Home-based child care providers serve almost a quarter of the 1.4 million children in subsidized care and millions more in the general population. It is important to help these professionals access the training and skills they need to help the children in their care with their social, emotional, physical, and cognitive development.

The 2014 reauthorization of the Child Care and Development Block Grant requires state child care agencies to provide preservice and ongoing training for all providers who receive assistance through the Child Care Development Fund. This includes an opportunity for training and professional development for home-based child care providers. The Office of Planning, Research and Evaluation, an office of the U.S. Department of Health and Human Services, Administration for Children and Families, published a report to inform state agencies about effective professional development and training interventions. This is useful for the child welfare field as this report can be used to develop more effective training for home-based providers, especially if they work with younger children in out-of-home care or otherwise involved with child welfare.

The report reviewed 19 recent studies that examined ways to improve the skills and professional development of home-based child care providers. The following are tips that emerged from the review:

The report also examined recruitment challenges and data collection methods and called attention to how study populations are important, how samples affect data, and what to consider when reading past studies.

The full report, Evaluation Training and Professional Development for Home-Based Providers, is available at https://www.acf.hhs.gov/opre/resource/evaluating-training-and-professional-development-for-home-based-providers.

 

Issue Date: June 2019
Section: News From the Children's Bureau
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5346


CB Website Updates

The Children's Bureau website hosts information on child welfare programs, funding, monitoring, training and technical assistance, laws, statistics, research, federal reporting, and much more.

Recent additions to the site include the following:

Visit the Children's Bureau website often to see what's new.
 

Issue Date: June 2019
Section: News From the Children's Bureau
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5347


Child Welfare Research

Study Shows Importance of Practical Field Training, Mentorship for Newly Hired Workers

A new study of the field training experiences for newly hired child welfare workers points to the need to integrate classroom knowledge and skills with actual field practice and provide supervisory support from certified child welfare workers.

A random sample of newly hired workers in Florida was selected to participate in telephone interviews regarding field training experiences. The interviews revealed that almost half of all new workers had positive or meaningful field experience, about 40 percent did not, and the rest had mixed experiences. Those reporting generally positive experiences found that field experience offered them important and realistic job exposure, while those who did not believed they had received incomplete training and lacked the professional guidance they needed during the training to move forward.

The study found that successful field experience depends on the levels at which classroom learning is integrated into field practice, structure is provided, practice takes place in disrupted environments, and certified child welfare workers mentor newly hired workers.

The authors note that, due to the diverse educational backgrounds of workers, preservice training can help to ensure that workers can effectively deliver child welfare services. Field practice gives new workers their first opportunity at receiving feedback on their child welfare practice skills and on using constructive criticism to improve their practice. The study recommends that certified and newly hired workers be afforded the time needed to work together on training content in a supportive environment.

"Field training experiences of child welfare workers: Implications for supervision and field education," by Melissa Radey, Lisa Schelbe, & Erin A. King (Clinical Social Work Journal, 47), is available at https://bit.ly/2DZJMla.
 

Issue Date: June 2019
Section: Child Welfare Research
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5350


Trauma-Informed Self-Care Measure for Child Welfare Workers

The emotionally stressful situations that child welfare professionals can find themselves in, often coupled with large workloads, can lead to secondary trauma and burnout. Carrying that stress and trauma can have a negative effect on their well-being, both physically and emotionally.

An article from Children and Youth Services Review focuses on a study that aimed to evaluate and revise the Trauma-Informed Self-Care (TISC) measure for child welfare professionals to better assess their level of self-care and to address possible areas where their needs are not being met. It also sought to clarify items in the measure, test additional items related to prevailing trauma-informed and self-care literature, confirm preliminary factors, and further test the reliability and validity of the revised measure. 

In this study, 177 child welfare workers were surveyed on measures such as burnout, secondary traumatic stress, psychological well-being, and more. The results from the survey, which were divided into subscales, indicated overall reliability for the TISC measure. It was negatively associated with burnout and secondary traumatic stress and positively associated with psychological well-being, among others.  

The study also discusses the following implications for practice:

"Development of a trauma-informed self-care measure with child welfare workers," by Alison Salloum, Mi Jin Choi, & Carla Smith Stover (Children and Youth Services Review, 93) is available at https://www.sciencedirect.com/science/article/pii/S0190740918302937.

 

Issue Date: June 2019
Section: Child Welfare Research
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5351


Strategies and Tools for Practice

Steps to Creating, Maintaining a Stronger Child Welfare Workforce

The Annie E. Casey Foundation released a brief that aims to help strengthen the child welfare workforce. Five Steps to a Stronger Child Welfare Workforce, which was created as part of its On the Frontline workforce development initiative, describes the following five steps, which can be used in state systems and state civil service agencies to hire, train, and maintain a healthy child welfare workforce: 

The brief also contains advice for staff charged with strengthening their workforce, including links to tools and resources that can be adjusted for each agency.

Five Steps to a Stronger Child Welfare Workforce is available at https://www.aecf.org/m/resourcedoc/aecf-fivestepstoastrongerchildwelfare-2018.pdf (2,674 KB).
 

Issue Date: June 2019
Section: Strategies and Tools for Practice
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5352


Toolkit Offers Guidance for Evaluating Programs and Early Outcomes

A toolkit prepared for the Children's Bureau offers step-by-step instruction for evaluating the initial implementation of social programs and their early outcomes. It is primarily intended for child welfare agencies and professionals to use in partnership with their evaluators.

The toolkit introduces the concept of formative evaluation, a method to evaluate programs during the earliest phases of implementation to inform and guide program improvement and determine readiness for more rigorous summative evaluation. The authors explain in the introduction that very few evaluations of social programs have shown positive results, perhaps due to program teams conducting summative evaluations of programs before they are ready for such assessments. They also note that in child welfare, program implementation is a process that takes place over time. The toolkit outlines four overlapping phases of implementation: exploration, installation, initial implementation, and full implementation.

The toolkit consists of three sections:

The toolkit makes use of a fictional agency to provide relevant examples. It also provides a glossary of key terms and additional resources.

Formative Evaluation Toolkit: A Step-by-Step Guide and Resources for Evaluating Program Implementation and Early Outcomes is available at https://www.acf.hhs.gov/cb/resource/formative-evaluation-toolkit.
 

Issue Date: June 2019
Section: Strategies and Tools for Practice
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5353


Testing and Piloting an Intervention for Implementation Success

Written by the Capacity Building Center for States

Imagine that a state child welfare agency is implementing a new kinship navigator program. After creating a theory of change, the agency selects a program model, adapts it to fit the needs of grandparents and other relative caregivers in the communities it serves, and creates tools for staff to use when providing information to caregivers and referring them to needed services. Imagine the state implements the program statewide without first testing the tools or piloting the program. Soon, staff discover that things are not working out as they had expected. Staff and relative caregivers are not completing referral forms correctly, and intake processes are overwhelmed. The agency is forced to backtrack.

Now, imagine that before statewide implementation, the state tests the forms with potential users and pilots the program in one or two divisions for a few months. The results are then used to "work out the kinks" before the program is adopted statewide.

Testing and piloting a program or other intervention can help agencies identify possible roadblocks, assess the target population's reactions to it, and make adjustments before implementing the program on a larger scale. These processes provide feedback loops to inform improvements and lay a foundation for successful and sustainable implementation.

The Capacity Building Center for States brief, Change and Implementation in Practice: Intervention Testing, Piloting, and Staging, can help child welfare agency leaders, managers, and implementation teams test, pilot, and stage an intervention to address an identified challenge. Some ideas from that brief are highlighted below.

Usability Testing

Usability testing is a process that helps teams quickly try out key program elements, assess their functionality, and refine or clarify, as needed. Usability testing provides a valuable opportunity to analyze and improve procedures and tools and sets a foundation for continuous quality improvement (Akin et al., 2013).

Usability testing follows a "plan, do, study, act" cycle, in which teams quickly prepare, implement, monitor, and gather feedback about a process or tool and then make corrections (see https://deming.org/explore/p-d-s-a for more information). The improved version of the process or tool may then be retested by repeating the cycle.

In a "plan, do, study, act" cycle, an implementation team performs the following (Permanency Innovations Initiative Training and Technical Assistance Project, 2016):

Piloting

Pilot testing, or piloting, is a trial run of the implementation of a whole intervention (or significant parts of it) on a small scale (e.g., a single county or agency division) to identify problems and work out issues before launching it on a larger scale (Durlak, 2013). Piloting helps agencies decide whether they should continue to implement the intervention and, if so, how implementation should occur.

Piloting helps teams perform the following (Office of Adolescent Health, 2011):

Although they are similar concepts, usability testing and piloting differ in several ways. First, while usability testing focuses on one aspect of an intervention or implementation process (e.g., a tool or procedure), piloting involves implementation of the entire intervention or large portions of it, sometimes under carefully chosen conditions. Second, usability testing answers questions such as "Does the tool or process work?" and "Do the instructions make sense to staff?" On the other hand, piloting addresses questions such as "Was implementation successful?" and "Was the short-term outcome achieved?" Finally, while usability testing usually is a quick and targeted process, piloting is more comprehensive and may take longer to complete.

Taking a deliberate approach and using data to test and refine an intervention both help agencies establish a firm foundation for sustainable change. For more information on testing and piloting an intervention and other change and implementation topics, visit the Change and Implementation in Practice web page on the Center for States website.

References

Akin, B. A., Bryson, S. A., Testa, M. F., Blase, K. A., McDonald, T., & Melz, H. (2013). Usability testing, initial implementation, and formative evaluation of an evidence-based intervention: Lessons from a demonstration project to reduce long-term foster care. Evaluation & Program Planning, 41, 19–30. Retrieved from https://doi.org/10.1016/j.evalprogplan.2013.06.003

Durlak, J. (2013). The importance of quality implementation for research, practice, and policy. Retrieved from https://aspe.hhs.gov/report/importance-quality-implementation-research-practice-and-policy

Permanency Innovations Initiative Training and Technical Assistance Project. (2016). Guide to developing, implementing, and assessing an innovation: Volume 4: Initial implementation. Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/guide_vol4_initial_implementation.pdf (1,978 KB)

U.S. Department of Health and Human Services, Office of Adolescent Health. (2011). Tips and recommendations for successfully pilot testing your program. Retrieved from https://www.hhs.gov/ash/oah/sites/default/files/pilot-testing-508.pdf (49 KB)
 

Issue Date: June 2019
Section: Strategies and Tools for Practice
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5354


Resources

Real Stories From the Field

The National Child Welfare Workforce Institute (NCWWI), an organization dedicated to increasing the equity and effectiveness of child welfare practice, has created a multimedia collection of workforce development stories from students, faculty, and child welfare professionals at all levels. These stories offer insight into NCWWI initiatives and partnerships, and both professionals and students can leverage the information in this collection to improve their own practice.

Stories range in topics from university partnerships, such as a traineeship program at the University of New Hampshire that helped a worker earn her degree, to trainings and resources for supervisors. These stories offer real-life examples, lessons learned, and best practices. The stories are available as podcasts, articles, and videos. Additionally, a toolkit is available with additional discussion resources for every story.

Real Stories From the Field is available at https://www.ncwwi.org/index.php/real-stories-from-the-field-we.
 

Issue Date: June 2019
Section: Resources
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5355


New Toolkit Helps Child Welfare Leaders Promote Family First Law

A toolkit from the Annie E. Casey Foundation is designed to help child welfare leaders and advocates talk about how the Family First Prevention Services Act (Family First) can improve outcomes for children and families. Family First provides more support for prevention services designed to keep families together, such as in-home treatment and family therapy.

The toolkit offers talking points on the benefits of the Family First law and tips for media interviews. It also includes a helpful summary on the basics of the new law as well as a guide that details the new funding available for child welfare prevention services and information on the opioid epidemic.

The toolkit is available at https://www.aecf.org/blog/new-tool-kit-helps-leaders-communicate-the-benefits-of-family-first-law/.
 

Issue Date: June 2019
Section: Resources
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5356


Training and Conferences

Supervision Matters Training

The University of Maryland School of Social Work has a training intended for child welfare supervisors (any division) in Maryland's local departments of social services who have been in their current role for 5 years or less. Each supervisor's administrator is required to participate in a monthly 1-day course concurrently with his or her staff.

The course is designed to help new supervisors gain the knowledge, core skills, and self-awareness necessary to be effective in their jobs, including the following:

The course, which is usually held in a training room in a local Department of Social Services, consists of 1 full-day orientation and 2 full-day trainings per month for a total of 11 full days of training over the course of 5 months. Additionally, the course requires approximately 3 hours of phone consultation related to transfer of learning activities during the 5 months of training. There is an optional one-on-one coaching component at the conclusion of the classroom training portion that is approximately 2 hours per month for 6 months.

To learn more about the Supervision Matters training, visit http://www.family.umaryland.edu/cwa-supervisionmatters.

 

Issue Date: June 2019
Section: Training and Conferences
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5357


Conferences

Upcoming conferences and events on child welfare and adoption include the following:

June

July

August

Issue Date: June 2019
Section: Training and Conferences
URL: https://cbexpress.acf.hhs.gov/index.cfm?event=website.viewArticles&issueid=207&articleid=5358



Articles in Children's Bureau Express are presented for informational purposes only; their inclusion does not represent an endorsement by the Children's Bureau or Child Welfare Information Gateway.

Children's Bureau Express does not disclose, give, sell, or transfer any personal information, including email addresses, unless required for law enforcement by statute.


Contact us at cb_express@childwelfare.gov.

Return to the Children's Bureau Website.