| Session Title: Context Matters: Conducting Evaluations in Federal, State, and Local Government Funding Source Environments |
| Multipaper Session 755 to be held in Sebastian Section I1 on Saturday, Nov 14, 10:55 AM to 11:40 AM |
| Sponsored by the Government Evaluation TIG |
| Chair(s): |
| Katrina Bledsoe, Walter R McDonald and Associates Inc, kbledsoe@wrma.com |
| Discussant(s): |
| Kimberly Wells, United States Office of Personnel Management, kimberly.wells@opm.gov |
| Abstract: The variance in contexts of evaluation projects demands the ability to understand the environment in which the evaluation is performed. For instance, projects that are performed within the context of federal funding vary greatly from the contexts of those that are funded by state and local governments. Each has a general culture that must be considered including goals of the evaluation, political agendas, key players, contracts, and methodological barriers such as participant recruitment. This presentation highlights the evaluation contexts of two common types of projects: one at the federal government level, and one at the state government level. Anticipated audience discussion will focus on navigating the contexts to address the needs of clients and of the evaluation. |
| Evaluation in State and Local Government Contexts: Considerations, Challenges and Successes |
| Cindy Crusto, Yale University, cindy.crusto@yale.edu |
| Meghan Finley, The Consultation Center, |
| This paper presents considerations, challenges, and successes of evaluation in a state/local government agency. Information is based on the evaluation of a large children's mental health cooperative agreement between the funder (SAMHSA) and a state partner. The project integrates three state systems to meet the needs of children with social, emotional, and/or behavioral health care challenges and their families. Considerations of context and evaluation include the utility of a champion at a high level in the project/organization who believes in accountability and CQI processes, need for developing relationships at multiple levels of state agencies and across multiple state agencies, maintaining integrity of the evaluation through multiple state-level transitions (i.e., governance bodies, economic difficulties). Challenges experienced include history of poor coordination and data sharing among agencies and significant system level changes. Successes include identification of key staff in each collaborating state agency that is committed to the evaluation/accountability process. |
| Evaluation in Federal Government Contexts |
| Carolyn Lichtenstein, Walter R McDonald and Associates Inc, clichtenstein@wrma.com |
| The presentation focuses on the facilitators and challenges for conducting evaluation work within the context of federal government contracting. The presenter discusses issues such as accountability, logistics of working within Federal contract guidelines, scope of the evaluation and evaluation data collection, data use, and the numerous dissemination requirements, both technical and scholarly. Facilitators of conducting Federal evaluation work include the emphasis on data-driven performance monitoring, having one major stakeholder (generally), and relatively little Federal staff turnover. In contrast, there are many challenges that must be overcome or incorporated into the evaluation; for example, specific performance monitoring needs sometimes restrict the evaluation design and type of data requested by the client, and approval must be obtained for the burden any proposed data collection efforts place on the public. Examples will be provided from several Federal-level evaluations conducted by the author. |