|
Session Title: Issues in Multisite, Multilevel Evaluations of Science, Technology, Engineering, and Mathematics (STEM) Education Programs
|
|
Panel Session 381 to be held in Room 107 in the Convention Center on Thursday, Nov 6, 3:35 PM to 4:20 PM
|
|
Sponsored by the Cluster, Multi-site and Multi-level Evaluation TIG
|
| Chair(s): |
| Samuel Held,
Oak Ridge Institute for Science and Education,
sam.held@orau.org
|
| Discussant(s):
|
| Samuel Held,
Oak Ridge Institute for Science and Education,
sam.held@orau.org
|
| Abstract:
A lack of funding in many government agencies has forced program managers to figure out ways to leverage resources through multilevel and multisite programs. Evaluation of these multilevel and/or multisite programs comes with a host of challenges involving coordination of multiple resources. Evaluators faced with the task of carrying out complicated evaluations must uphold professional standards when resolving these challenges. This panel discussion will cover issues in evaluating multilevel, multisite STEM education programs while adhering to the Program Evaluation Standards set for by the Joint Committee on Standards for Educational Evaluation. The presenters will outline some of the challenges they have faced in implementing their evaluation plans, and discuss how the standards of utility, feasibility, propriety, and accuracy have shaped their dealings with these issues.
|
|
Issues in Instrumentation and Data Collection for the Evaluation of a Multisite, Multilevel National Workforce Development Endeavor
|
| Pamela Bishop,
Oak Ridge Institute for Science and Education,
pbaird@utk.edu
|
|
The Department of Energy’s Office of Workforce Development for Teachers and Scientists funds five national experiential learning programs, which the presenter has been charged with evaluating. The scope of the evaluation includes both the five federally-funded, multisite science education programs, and a metaevaluation of common enterprise outcomes and impacts of all programs. The presenter will discuss her experience with developing and implementing a multilevel, multisite evaluation plan for this large-scale program. Included in the presentation will be her unique experiences with development of data collection tools that can be used across programs, but are still sensitive enough to capture information at the local level, as well as methods used to align resources and goals across the levels and sites to ensure accurate and reliable data collection (and subsequent analyses) take place in a timely and efficient manner throughout the evaluation.
|
|
|
Multilevel Evaluation of the Newly Established Tennessee Governor's Academy for Mathematics and Science
|
| Amy Sullins,
Tennessee Governor's Academy for Mathematics and Science,
acsullins@gmail.com
|
|
The Tennessee Governor's Academy for Mathematics and Science (TGA), a residential specialty experience for talented juniors and seniors, is in its second year of operation and the implementation phase of its multilevel evaluation plan. TGA is a component of Governor Phil Bredesen's P-16 education network; numerous resources are being utilized to educate and nurture this population of talented students with the goals of graduates choosing STEM majors and, later, contributing to the STEM community. The multilevel plan includes evaluation of individual students, the school, and TGA partners, including Oak Ridge National Laboratory (where TGA students are engaged in scientific investigation with scientist mentors) and the University of Tennessee (where the senior class pursues math and science coursework) for program effects. The TGA multilevel evaluation model will be presented and discussed. Challenges of implementation will also be discussed, including formation of a new organization, studying a minor population, coordination of data collection, and the evolvement of the evaluation plan.
| |