| Session Title: Stakeholder Involvement in the Development of a Multi-Purpose Evaluation System for a Public Mental Health System |
| Multipaper Session 658 to be held in Mineral Hall Section G on Friday, Nov 7, 3:25 PM to 4:10 PM |
| Sponsored by the Alcohol, Drug Abuse, and Mental Health TIG |
| Chair(s): |
| Sandra Sundeen, University of Maryland Baltimore, ssundeen@psych.umaryland.edu |
| Abstract: Evaluators who work with public mental health authorities can provide policy makers with valuable information about mental health system effectiveness at multiple levels. In Maryland, the State mental health agency established a partnership with the University of Maryland, Baltimore, to develop an outcomes measurement system that would provide state officials with information about the outcomes of state-funded services in order to make informed decisions about service effectiveness and, ultimately, priorities for reimbursement. Outcomes Measurement System results can also be used to inform local quality monitoring activities, service provider program development, and clinical decision-making This session will describe the participatory process that was undertaken to develop the Outcomes Measurement System, including the involvement of multiple stakeholders. The challenges encountered in analyzing the data and presenting it in a way that addresses the primary objective of usefulness for policy decision-making will also be discussed. |
| Using Evaluation to Support State Mental Health Policy: Development and Implementation of a System to Measure Outcomes of Services |
| Diana Seybolt, University of Maryland Baltimore County, dseybolt@psych.umaryland.edu |
| Sandra Sundeen, University of Maryland Baltimore, ssundeen@psych.umaryland.edu |
| Through a collaborative partnership with the Maryland mental health authority the Systems Evaluation Center of the University of Maryland, Baltimore developed, pilot-tested and participated in the implementation of an Outcomes Measurement System. This paper will focus on the policy implications of this project, including data requirements imposed by the Federal and State governments, along with the expectation to address the frequently competing concerns of other stakeholders. The participatory process that was implemented throughout all phases of the project will be discussed. Examples will be given of the policy challenges that were encountered and the ways in which they were addressed. Current issues related to maintenance, evaluation and potential revision of the system will be presented. |
| Evaluating Individual Changes Over Time in the Substance Abuse and Mental Health Services Administration's National Outcome Measures: Methodological Challenges and Complexities |
| Timothy Santoni, University of Maryland Baltimore County, tsantoni@comcast.net |
| Qiang Qian, University of Maryland Baltimore, qqian@psych.umaryland.edu |
| Even before Maryland Outcome Measurement System challenges shifted from data collection to data analysis and interpretation, several recording issues in the implementation of the collection system were identified. Data validity checks were missing for many fields; the collection system was changing certain entries. Special attention was needed to understand the meaning of blanks and zeros. Data inconsistencies were identified that had to be rectified. A method for the selection of t(0) and (t(n)) forms was required. Determination of what difference was required to determine 'real change', and the significance of maintenance related to positive, negative, and neutral maintenance, were explored. The effect of the calendar on certain items had to be considered. All of these considerations then had to be presented in formats and with explanations appropriate to audiences at several levels of sophistication. Sample analyses adopted to address these issues will be presented. |