| Session Title: Using Evaluation Perspectives to Develop Organizational Capacity to Conduct Useful Student Learning Outcomes Assessment in Higher Education |
| Multipaper Session 235 to be held in Monterey on Thursday, Nov 3, 8:00 AM to 9:30 AM |
| Sponsored by the Assessment in Higher Education TIG |
| Chair(s): |
| Hallie Preskill, FSG Social Impact Consultants, hallie.preskill@fsg.org |
| Discussant(s): |
| William Rickards, Alverno College, william.rickards@alverno.edu |
| Abstract: In the evaluation literature, an emphasis on utilization of evaluation has led to a great deal of theoretical and practical development, including the importation of a "learning organization" perspective. Higher education accreditation bodies call for universities to develop student learning outcomes assessment systems that are self-reflective and improvement-oriented, nurturing true learning organizations. However, many educators still experience the outcomes assessment process as a management-owned mandate, and genuine use of assessment findings is relatively rare. This multi-paper session examines how organizational capacity to engage in useful and meaningful student learning outcomes assessment can be tracked and developed, impacting assessment design, implementation, and utilization. Three presentations showcase empirical studies and capacity-building efforts across various program contexts: humanities departments, teacher education programs, and undergraduate general education. The session will conclude with suggestions for future research as well as recommendations for ways to facilitate assessment capacity-building in university settings. |
| Fostering Evaluative Thinking, Values, and Identity Through Outcomes Assessment in College Humanities Programs |
| Yukiko Watanabe, University of Hawaii, Manoa, yukikow@hawaii.edu |
| This paper presents findings from a longitudinal multiple case study that explores human, contextual, and assessment factors that enhance or hinder (a) planning and implementing utilization-focused outcomes assessment, and (b) organizational assessment capacity development. Eight college humanities programs across two universities participated in the study. Faculty surveys, chair interviews, and meeting observations showed variability in leadership style, power and decision-making structures, perceived value of assessment, and assessment and evaluation expertise. Update reports and meeting recordings provided insights into value conflicts and assessment issues programs faced. Noteworthy factors influencing assessment capacity development were collaborative culture, disciplinary values and norms, the roles of assessment facilitators, near-peer role models, and program-external demands and opportunities. Readiness for outcomes assessment and capacity development capability had a reciprocal relationship with the way outcomes assessment was pursued and implemented. The presenter will also highlight emerging issues and questions for future research on outcomes assessment in higher education. |
| The Role of Evaluation Use in Defining Organizational Readiness for the Purposes of Improvement and Accountability in Teacher Education Accreditation |
| Georgetta Myhlhousen-Leak, University of Iowa, gleakiowa@msn.com |
| This study reports types and factors of use that impact teacher education programs' readiness for and use of accreditation. An examination of the purposes of teacher education program review (i.e., state accreditation) as accountability and improvement sets the stage for recognizing "what" and "how" use occurs. Respondents from four programs completed structured interviews and six response scales detailing how use occurred in their program. Descriptions were coded and interpreted based on the known types and factors of use in the evaluation utilization literature. Findings of types of use revealed that the distinction of process and findings use provided a more comprehensive picture of readiness and use. Descriptions and scale results based on Alkin's (1985) factors of use provided evidence that each factor emerges with a unique impact. This study answers the call by Banta (2002) for research to identify "characteristicsGǪthat inhibit or facilitate use of assessment information" (p.65). |
| Nurturing Readiness for a "Culture of Learning" for General Education |
| John Stevenson, University of Rhode Island, jsteve@uri.edu |
| Here's a despairing question from a university administrator: "Other than pouring money we don't have into assessment, how can we create a culture that values the process?" This paper addresses that plea, applying empirical and theoretical ideas drawn from previous evaluation work on learning organizations. The specific focus is on general education, an orphan program with no necessary faculty community. Two case studies of public universities provide data on academic culture, stages in readiness for genuine assessment, and actions that foster utilization. These can be applied to analysis of practical progress and recommendations for enhancing the movement from an initial group of committed faculty who devise outcome objectives for general education to a learning organization that will be able to benefit from assessment findings on a sustainable basis. How can an evaluation perspective aid this process? What values and value conflicts must be understood to nurture such a learning community? |