|
Another Look at Methods of Evaluating Course Placement Systems
|
| Presenter(s):
|
| Howard R Mzumara, Indiana University Purdue University Indianapolis, hmzumara@iupui.edu
|
| Wen Linn, Indiana University Purdue University Indianapolis, lin39@iupui.edu
|
| Abstract:
The purpose of this presentation is to describe methods for evaluating the effectiveness of course placement systems based on a systematic approach that involves asking specific evaluation questions and using multiple methods or data sources to answer the questions posed in a validation study. Presenters will address an approach to system evaluation that promotes use of comprehensive validation studies (that involve use of decision theory and logistic regression approaches to provide validity evidence for assessing probabilities of success in particular courses and assess the appropriateness of placement cutoff scores), placement-enrollment comparisons, placement testing exit surveys (or pre-enrollment questionnaires), end-of-course evaluations, and feedback from academic advisors and instructors. The session will include an interactive discussion based on a review of a selected list of evaluation-related questions and methods used to facilitate validation studies involving the ACT COMPASS Mathematics Placement System for placing students in mathematics courses at a large Midwestern university.
|
|
Assessing Readiness for a 'Culture of Learning'
|
| Presenter(s):
|
| John Stevenson, University of Rhode Island, jsteve@uri.edu
|
| Melinda Treml, Northern Arizona University, melinda.treml@nau.edu
|
| Thomas Paradis, Northern Arizona University, thomas.paradis@nau.edu
|
| Abstract:
In colleges and universities, organizational context is critical for the use of assessment data in program improvement. This paper examines ways of assessing organizational readiness for the kind of learning-organization environment that features an institution-wide, ongoing, highly valued system for using student learning outcome data in curriculum design. We propose a five-stage developmental model, and illustrate its utility with two empirical strategies. A chairperson survey and an assessment report analysis are provided to demonstrate how this developmental perspective can aid internal evaluators in directing attention to the organizational context and to steps needed to promote systemic change from an externally driven, top-down, mandated assessment process to an internally valued, learning-centered process.
|
|
Evaluating Change in Interdisciplinary Collaboration
|
| Presenter(s):
|
| Jill Lohmeier, University of Massachusetts Lowell, jill_lohmeier@uml.edu
|
| Steven Lee, University of Kansas, swlee@ku.edu
|
| Abstract:
Evaluators are often asked to assess changes in the behaviors of members of an organization. One goal for many organizations is to increase the willingness of its members to work together. In academic settings, there is often a desire to increase interdisciplinary work. While evaluators are frequently asked to assess changes in collaboration, or interdisciplinary work, few, if any, measures of interdisciplinary work are available. This presentation will describe the creation and implementation of an evaluative Tool for Interdisciplinary Assessment (TIA). Twenty-two graduate students and faculty associated with an NSF grant at a Midwestern university took the TIA online, as well as a modification of the Levels of Collaboration Scale (LCS; Frey, Lohmeier, Lee & Tollefson, 2006). The effectiveness of using the TIA and LCS for evaluation, as well as the relationship between the two will be discussed. Challenges associated with evaluating interdisciplinary collaboration will also be discussed.
|
|
Engaging Constituents: Using Assessment Data to Inform Practice
|
| Presenter(s):
|
| Candace Lacey, Nova Southeastern University, lacey@nova.edu
|
| Barbara Packer-Muti, Nova Southeastern University, bpacker@nova.edu
|
| Jennifer Reeves, Nova Southeastern University, jennreev@nova.edu
|
| Abstract:
The Southern Association of Colleges and Schools (SACS) requires that all universities seeking accreditation implement a Quality Enhancement Plan (QEP) based on the core facet of how the university focuses on enhancing student learning. Based on the theme of enhancing student academic engagement, central administration at Nova Southeastern University (NSU) determined that a 360 degree assessment of all constituents would provide important insights into perceptions and drivers of engagement. To this end, NSU entered into a multi-year assessment of engagement with questionnaires distributed via a web-based modality to all students, alumni, faculty, staff, and administration. Findings from these assessments were distributed to the 14 colleges and nonacademic units within the university to assist in developing student, employee, and alumni engagement plans. This session will provide information about the methodology utilized for evaluation, assessment, instrumentation, and dissemination of findings.
|
| | | |