| Session Title: Building Evaluation Capacity at the Society for Advancement of Chicanos and Native Americans in Science (SACNAS) |
| Multipaper Session 580 to be held in Hopkins Room on Friday, November 9, 11:15 AM to 12:00 PM |
| Sponsored by the Multiethnic Issues in Evaluation TIG |
| Chair(s): |
| Jack Mills, Independent Consultant, jackmillsphd@aol.com |
| Abstract: This session will describe the challenges, opportunities and lessons learned in using evaluation results to evolve services provided to underrepresented minority (URM) scholars in higher education. The Society for Advancement of Chicanos and Native Americans in Science (SACNAS) is nationally recognized for initiating and executing effective programs that provide underrepresented minority (URM) students and young scientists with the tools they need to successfully advance in the sciences and related technical fields. This session features a dialogue and interplay between the immediate past president of this nationally prominent minority serving organization and its external evaluator. Members of the audience will be able to hear from the organization's perspective what steps were taken to prepare for a major evaluation initiative as well as challenges, opportunities and lessons learned from the perspective of the external evaluator. |
| Preparing the Way for Evaluation: The Experience of the Society for Advancement of Chicanos and Native Americans in Science (SACNAS) |
| Marigold Linton, University of Kansas, mlinton@ku.edu |
| Jack Mills, Independent Consultant, jackmillsphd@aol.com |
| Over the past five years, program evaluation research has become a critical strategic initiative for the Society. Moving an organization toward increasingly more sophisticated evaluation approaches requires commitment on a number of levels. At the board of directors level there was a need to establish evaluation as a priority, allocate program resources to evaluation that otherwise might provide direct service to clients and be open to findings that might challenge traditions that evolved over a number of years. At the program staff level there was a need to establish a degree of comfort in working with a paid skeptic-someone who would both bear good news about the program's successes while pointing out areas in which the program operations could be strengthened. We will discuss ways in which evaluation has affected many aspects of program operations and future evaluation directions as we progress up the organizational learning curve. |
| A Theory-based Approach to Measuring Minority Career Advancement in the Sciences: A Case Study of the Society for Advancement of Chicanos and Native Americans in Science (SACNAS) |
| Jack Mills, Independent Consultant, jackmillsphd@aol.com |
| Marigold Linton, University of Kansas, mlinton@ku.edu |
| This presentation will describe the development of program evaluation research at SACNAS. We have distilled a model of program theory for factors we believe are essential in assisting URM students navigate scientific and technical careers. With this model in hand, we began to develop multiple methods to measure SACNAS' impact, including: a survey of students' developmental assets prior to SACNAS involvement, focus groups, interviews and participative observations. The Society is evolving a web-based process to track the outcomes of career progress. The presenters will describe how the program evaluation methodology has evolved at SACNAS and future directions we are taking to strengthen the organization's evaluation practice. The two SACNAS presentations in this session will highlight the dynamic interplay and the beneficial impact on strategy that can emerge when the leadership of an organization and an external evaluator develop a strong collaboration. |