July/August 2018Vol. 19, No. 6Look Before You Leap: How a Data Plan Can Help You Dig Deeper Into Your Agency's Needs

Written by the Children's Bureau's Capacity Building Center for States.

Much like in their work with families, agencies must understand their own underlying needs to develop solutions that will effectively address them. Six basic tasks can guide agencies as they dig deeper into their organizational needs and identify reasons for their performance on key outcomes. One task is creating a comprehensive data plan developed in collaboration with program leadership, stakeholders, and data/information technology staff to guide data collection and analysis. Once a problem or need is identified, a data plan helps organize existing data and identify new sources to understand the problem and get to the root cause(s).

A comprehensive data plan includes the following:

• Research questions aimed at the problem's scope and characteristics—A good data plan starts with asking key questions to better understand the problem the agency is tackling. For example, an agency looking into why children are not achieving timely permanency might ask questions like: Does time to permanency vary by population group (by age, race, reason for entry)? Are there differences across regions within our state? How do placement history (e.g., number of placements) and placement type affect time to permanency?

• Data sources that are reliable, timely, and valid—The agency should use multiple sources of both quantitative (numerical) and qualitative (narrative) data to answer its research questions. Quantitative data often provide a big picture of a broad population. Examples of quantitative data sources include administrative data, state or federal data, and agency partner data. Qualitative data usually explore a smaller sample, offering valuable insight into quality of services. Examples of qualitative data sources include case reviews, focus groups, interviews, and child welfare case study results. For timely permanency, quantitative data would include statistics on children's length of time in care and patterns by county. Qualitative data might focus on children's relationships with their caregivers or quality of visits with their caseworkers. In addition to using multiple data sources, agencies should consider data quality and appropriate analysis methods. Data quality includes data reliability (data entry is consistent, and analysis yields the same results), timeliness (data are current), and validity (data measure intended target and represent the population being served by the agency).

• Data analysis that helps tell the story—Analyzing data involves looking at patterns, trends, and relationships to verify and clarify the problem. Analysis helps shed light on the whole story. There are many types of analyses, including those that help summarize data in meaningful ways and those that test relationships between variables for a sample of the population. When exploring factors that prevent timely permanency, the agency may analyze the differences in timely permanency across age groups and then further analyze the relationship between age at removal and length of time in care.

A good data plan will guide the problem-exploration process and help ensure that an agency has a solid understanding of the problem it wants to address and what kind of solution is needed. Investing time and energy into this phase of the improvement process will pay off later, as the agency is able to pinpoint the best intervention and clearly communicate with its stakeholders why change is needed.

Look for the Change and Implementation in Practice series, which offers user-friendly resources to walk agencies through a research-based process (including developing a data plan) for effectively making changes to improve outcomes. In addition, the following resources provide more information on data analysis and quality to support development of a strong data plan:  

• Focused CQI Services Indepth Skill Building, "Module 5: Data Analysis for CQI: Identifying and Understanding the Problem," "Handout 2: Types of Data Analyses" (Capacity Building Center for States, 2017)

• Focused CQI Services, "Module 4: Overview Curriculum" (Capacity Building Center for States, 2017)