|
Issues and Opportunities in Consortium Evaluations: Cross-Sites, Levels, and Groups
|
| Presenter(s):
|
| Jeffry White, Ashland University, jwsrc1997@aol.com
|
| James Altschuld, The Ohio State University, altschuld.1@osu.edu
|
| Abstract:
More and more evaluators are providing assessments for consortium types of projects. In such situations numerous challenges arise and by the same token there are unique opportunities to conduct research. The presenters are providing evaluation services to a two site project with each site having three levels of institutions involved (universities, 2 year colleges, and high schools). One obvious problem is that IRB clearance is more difficult in this circumstance. Another is that project activities are varied across the two sites necessitating minor variations in the items used in surveys. An example of an opportunity is the creation of a database that will allow tracking of students from high school through college or other post secondary activities. The project will be briefly described followed by an analysis of the issues and opportunities inherent in it and how they have been handled by the evaluators.
|
|
Process Elements in Program Fidelity Measurement: Inferential Gains in a Multi-site Experimental Study
|
| Presenter(s):
|
| Gregory Teague, University of South Florida, teague@fmhi.usf.edu
|
| Abstract:
Critics of recent practice in program fidelity measurement have faulted emphasis on structural features; although process elements can be more difficult to measure, they may be more essential to a program's intended outcomes. A SAMHSA/CMHS multi-site study illustrates the value of emphasizing program processes. The study investigated outcomes of consumer-operated services offered adjunctively to traditional mental health services for persons with serious mental illnesses. It used an experimental design and a common assessment protocol, following over 1,600 participants for 12 months in eight sites offering different program models. Both consumer-operated and traditional service programs were assessed using a fidelity measure informed by consumer-defined concepts of recovery-oriented service characteristics. Inclusion of fidelity data in analysis supplemented the primary hypothesized experimental finding of increased well-being by showing that specific recovery-oriented characteristics of the service environments accounted for substantial variation in well-being and other outcomes across program types and conditions.
|
|
A Balance Between Standardization and Flexibility: Using a Compendium of Indicators to Measure Child Well-Being in Multi-site Evaluations
|
| Presenter(s):
|
| Isabelle Carboni, World Vision International, isabelle_carboni@wvi.org
|
| Nathan Morrow, World Vision International, nathan_morrow@wvi.org
|
| Abstract:
A multi-site evaluation of a global NGO's contribution to child well-being is a complex challenge. Individual project sites, National and Regional Offices, Global Head Quarters, as well as donors, all require particular information for advocacy, accountability and learning. How can one set of indicators meet all these needs? Using a compendium of indicators and a 'roll-up' evaluative process, each entity can collect, analyse and learn from the information that matters most.
Child well-being is a multi-dimensional concept, understood differently in different places. Through a global consultation process, a common organizational understanding of child well-being was developed, in health, education, relationships/spirituality and rights. Building on this, a compendium of outcome indicators was developed. The compendium allows projects staff to choose indicators relevant to their specific context, based on community perceptions of child well-being, adding in new indicators as relevant, whilst still measuring progress towards child well-being outcomes at the global level.
|
| | |