Evaluation of the reliability of an instrument to assess the content of treatment planning documents within the Child and Adolescent Mental Health Division

Abstract

This study evaluated the reliability of the Service Guidance Review Form (SGRF). The SGRF was an instrument designed to quantify specific aspects of recommendations for treatment planning documents within the context of the Child and Adolescent Mental Health Division (CAMHD) in Honolulu, Hawaii. Documents utilized in this study included Mental Health Assessments (MHA), Coordinated Service Plans (CSP), and Mental Health Treatment Plans (MHTPs). The SGRF is a method of distilling recommendations from each of these documents into component parts, both in terms of specific treatment practice elements and targets of treatments. Several CAMHD personnel and University graduate students were trained in the use of the measure, and used it to rate documents from a randomly selected set of 200 new cases from CAMHD Fiscal Year 2003 (July 1, 2003 - June 30, 2004). Reliability of the instrument was examined by comparing raters' codes using Intraclass Correlations (ICCs; Shrout and Fleiss, 1979).