Prediction is Local: The Benefits of Risk Assessment Optimization
Melissa Kowalski (Assistant Professor in the Department of Criminal Justice) published a peer-reviewed article regarding risk/needs assessment for youth in Justice Quarterly.
As part of a research team, Melissa Kowalski co-authored an article, “Prediction is local: The benefits of risk assessment optimization” with Zachary Hamilton and Alex Kigerl.
In most states and jurisdictions, risk assessments are incorporated into justice system practice. Despite decades of use, the methods of tool development are rarely translated to the field. Many agencies implement ‘off-the-shelf’ versions, where a tool developed with a unique set of methods and subjects demonstrates prediction shrinkage when applied to a new jurisdiction. Using a large, 10-state sample of assessed youth (N=494,050), we isolate, test, and evaluate the relative impact of notable risk assessment variations, including:
- item selection
- response weighting
- outcome definition/duration
We further combined approaches to evaluate an ‘optimized’ development approach. Findings revealed substantial gains with each variation tested, where optimized models provided a full effect size predictive improvement. We discuss best practices for the future of risk assessment, noting the predictive accuracy lost when implementing tools off-the-shelf, and describe how optimization techniques substantially improve risk prediction, specifying a given tool to an agency’s needs.