| In a 90 minute Roundtable session, the first
rotation uses the first 45 minutes and the second rotation uses the last 45 minutes.
|
| Roundtable Rotation I:
Adaptations to Evaluation Design: Two Examples of Ensuring Quality in Practice |
|
Roundtable Presentation 882 to be held in BONHAM A on Saturday, Nov 13, 2:50 PM to 4:20 PM
|
|
Sponsored by the Pre-K - 12 Educational Evaluation TIG
|
| Presenter(s):
|
| Stephanie Feger, Brown University, stephanie_feger@brown.edu
|
| Elise Laorenza, Brown University, elise_laorenza@brown.edu
|
| Abstract:
Often in external evaluation, quality is initially established through development of an appropriate design, selection of instruments, and planning for data collection and analysis. However, evaluation plans often require adaptations based on program modifications. Evaluation adaptations are frequently needed to clarify key program components discovered over the course of implementation and also to determine benchmarks of program impact. Evaluation adaptations can improve evaluation relevance, an indicator of overall evaluation quality, and provide new tools and/or data to identify program activities that effectively contribute to program goals and outcomes. Through discussion of two evaluation studies of statewide science programs this roundtable will explore; (1) the development of a student reflection instrument as an evaluation adaptation in the context of program benchmarks, (2) the process for aligning evaluation adaptations with original methods and the integration of results, and (3) the utilization of evaluation adaptations to support program goals and improve program impact.
|
| Roundtable Rotation II:
Assessing Program Implementation in Multi-site Educational Evaluations: The Development, Alignment, And Incorporation of Evidence-based Rubrics in Rigorous Evaluation Design |
|
Roundtable Presentation 882 to be held in BONHAM A on Saturday, Nov 13, 2:50 PM to 4:20 PM
|
|
Sponsored by the Pre-K - 12 Educational Evaluation TIG
|
| Presenter(s):
|
| Amy Burns, Brown University, amy_burns@brown.edu
|
| Tara Smith, Brown University, tara_smith@brown.edu
|
| Abstract:
This roundtable will share a multi-step approach to program implementation assessment developed by the Education Alliance at Brown University. Presenters will provide examples from rigorous evaluations of four districts which received federal Magnet School Assistance Program funding to describe: the development of implementation rubrics that align with districts’ logic models; data sources for evidence-based measures that are used to identify compliance with logic models; rubric scoring processes; and the incorporation of these rubric data into multivariate statistical models. The presenters will promote discussion with the roundtable group on ways to address challenges in “quantifying” implementation data.
|