|
A Process for Determining the Acceptability Of Measurement Tools: How to Decide?
|
| Presenter(s):
|
| Lydia Marek, Virginia Tech, lydiam8992@yahoo.com
|
| Donna-Jean Brock, Evaluation Consulting Services Inc, djbrock.ecs@cox.net
|
| Abstract:
Many organizations fund multiple programs which are diverse in audience and focus. This poses certain challenges including inconsistent quality of evaluation across sites and an inability to determine overall impact of the initiative. To address these challenges, the CYFAR Initiative and the 4-H National Council through Kraft Foods, funded research to develop methodology for determining the adoption of measures for program evaluation. A brief rating form was developed based upon criteria identified in the literature as critical in selecting quality evaluation instruments. This rating form was reviewed by four experts in the field. It was then implemented with 44 reviewers across the country to review over 400 potential common measures for acceptability and applicability to these two projects. This methodology of choosing acceptable measures streamlined a massive review process, increased buy in by key stakeholders for the use of the measures, and ensured relevancy of measures to these initiatives.
|
|
Creating and Sustaining Systemic Change: A Rubric for Measuring Organizational Capacity in Higher Education Alliances
|
| Presenter(s):
|
| Sarah Hug, University of Colorado, Boulder, hug@colorado.edu
|
| Heather Thiry, University of Colorado, Boulder, heather.thiry@colorado.edu
|
| Abstract:
Evaluating organizational capacity and sustainability in a collaborative alliance is a challenge. Through a National Science Foundation-funded project, we developed an evaluation rubric that measures four constructs vital to understanding capacity in multi-site initiatives in higher education settings: healthy educational pipeline development, academic resource development and training, faculty/staff engagement, and alliance-wide collaborative engagement. The Computing Alliance of Hispanics (CAHSI) is a ten-institution consortium funded by the National Science Foundation to recruit, retain, and advance Hispanics in computing fields. We propose the CAHSI evaluation rubric as a viable model for evaluating organizational capacity and sustainability. In this talk, we highlight the components that led to the development of this evaluative measurement tool. In addition, we show how it can be used in collaborative higher education alliances focused on educational reform and innovation.
|
| |