Predicting progress ratings on disruptive behavior targets with practices derived from the evidence-base

Abstract

Carefully controlled experimental research indicates that certain treatment approaches are more efficacious than others in addressing disruptive behavior problems in youth. However, community practitioners infrequently employ these treatments and their effectiveness in usual care settings is less well known. One way to increase implementation of science-based findings into treatment as usual might be to encourage the use of therapeutic practices commonly found in the descriptions and manuals of evidence-based services. However, the therapeutic impact of such practices in treatment as usual is mostly unknown. The current study investigated whether the extent to which community therapists applied practice elements derived from the evidence base (PDE) predicted rate of improvement on average disruptive behavior progress ratings. The first five months of clinical data for youth (N=720) receiving non-manualized, intensive in-home services, delivered by therapists (N=225) in the state of Hawaiʻi, Child and Adolescent Mental Health Division were included in analyses. These youth had two or more, of five possible, disruptive behavior-specific targets endorsed as a focus of treatment on the Monthly Treatment and Progress Summary (MTPS). PDE use was assessed using three overlapping sets of variables: practice elements identified for youth independent of age criteria, practices identified specifically for youth ages 13 years and older, and practices identified specifically for youth ages 12 years and under. These variables were based on the proportion of practice elements endorsed on the MTPS that were in 30% or more of treatments attaining Level One (Best Support) for disruptive behavior in the literature. Utilizing a three-level multilevel model approach, monthly rate of change in average disruptive behavior progress ratings was predicted by each measure of PDE use separately. Additional exploratory analyses examined whether the presence or absence of specific practice elements within each set predicted treatment change. A greater proportion of practices from Level One (Best Support) protocols for youth ages 13 years and older significantly predicted greater rates of change in average disruptive behavior progress ratings per MTPS month. Higher proportions of PDE based on the other two criteria were in the same positive direction, but were not statistically significant predictors. Consistent with these findings, several specific practice elements aimed at building youth skills and decreasing family stress were significantly associated with greater change in average progress ratings. While further research is needed, findings suggest that increasing the use of PDEs is a promising strategy for bringing evidence-based research into usual care. Furthermore, the current study offers an innovative method of evaluating outcomes in community mental health, by integrating targets, progress ratings, and practice elements. Additional implications and limitations are discussed.