Reliability and validity evidence for the dual-disorder treatment fidelity scale

Abstract

Psychometric properties of the Dual-Disorder Treatment Fidelity Scale (Mueser, Noordsy, Drake & Fox, 2003) were evaluated. Reliability was assessed through measures of inter-rater agreement, and validity was assessed through concurrent, predictive, and construct validation procedures. Two raters assessed 3 community mental health center (CMHC) and 3 dual diagnosis enhanced (DDE) programs for fidelity. Fifty-two consumers and 58 staff completed an outcomes survey and a measure of knowledge about the model, respectively. Inter-rater agreement on fidelity ratings across programs, within each program, and for most scale items was good. Total fidelity scores did not distinguish between CMHC and DDE program types; however, a cluster analysis revealed concurrent validity based on the item level ratings. Fidelity scores did not predict consumer outcomes, and fidelity was unrelated to staff knowledge of the integrated model. Conclusions drawn from item level ratings appear valid; the validity of conclusions based on total fidelity scores remains unknown.