November 2022Vol. 23, No. 9Use of Artificial Intelligence-Based Decision Tools in Child Welfare

A recent research paper, Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support, explores child welfare workers’ experiences with using artificial intelligence-based decision support (ADS) tools at a public child welfare agency. These tools are increasingly being used in public sector agencies to augment human decision-making in high-stakes social contexts.

 

The paper presents findings from a series of interviews and inquiries at the Office of Children, Youth and Families in Allegheny County, PA, regarding their use of ADS tools. Specifically, investigators looked into how the workers’ reliance upon ADS tools was guided by their knowledge of contextual information beyond what the artificial intelligence model captures, their beliefs about the tool's capabilities, organizational pressures, and awareness of misalignments in decision-making objectives.  

 

The ADS tool used at this specific agency augmented the human decision as to whether to investigate a call alleging abuse or neglect. However, most of the workers interviewed expressed that the ADS tool plays a relatively minor role in their overall decision-making processes. In addition, most workers knew very little about how the tool works, what data it relies on, or how to work with the tool effectively.

 

Based on the findings, researchers provided the following design implications for agencies using or implementing ADS tools:

 

• Support workers in using their expertise to improve an ADS tool's performance.
• Design training tools that support workers in understand the boundaries of an ADS tool's capabilities.
• Support open, critical discussion around the tools.
• Provide workers with balanced and contextualized feedback on their decisions.
• Codesign measures of decision quality with the workers.
• Communicate how decision-making power should be distributed among workers and the ADS tool.
• Support diverse stakeholder involvement in shaping ADS tool design.

 

The findings also highlight opportunities for further research into how ADS tools are used in child welfare and other decision-making contexts.

 

More information is available in the research paper, Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support.