|
What Teachers Need in Terms of Results Versus What is Commonly Reported: Some Standard Deviations?
|
| Presenter(s):
|
| Guido Gatti, Gatti Evaluation Inc, gggatti@gattieval.com
|
| Katya Petrochenkov, Gatti Evaluation Inc, katyap@gattieval.com
|
| Abstract:
The authors surveyed 109 teachers about what answers they require from educational research and how they need those answers reported to them.
Teachers as practitioners are basically interested in how they can best implement a program, how many of their students will benefit, and how much students will benefit.
In digesting information as to 'how much' and 'how many', teachers have little knowledge of, or use for, traditional effect size statistics such as Cohen's d.
When described in teacher friendly terminology, only 34% were familiar with Cohen's d, only 24% thought it useful, and only 41% found it understandable.
In contrast, teachers found simple comparisons in metrics familiar to them (e.g., number questions correct, grade equivalence, percent meeting standard or above average) most understandable (i.e., 90%) and useful (i.e., 73%).
In the authors’ view, teacher friendly measures should be reported along with traditional statistics when reporting educational research results.
|
|
The Impact of Releasing Rubrics on Performance Assessment Scores
|
| Presenter(s):
|
| Ashlee Lewis, University of South Carolina, ashleealewis@hotmail.com
|
| Min Zhu, University of South Carolina, helen970114@gmail.com
|
| Xiaofang Zhang, University of South Carolina, jae2008@gmail.com
|
| Abstract:
Rubrics are commonly used as scoring tools in educational assessments. Although assessment specialists argue that releasing scoring rubrics to educators can improve student performance and teachers’ instructional practices, most studies have focused on scoring procedures and have not attempted to provide empirical evidence for the value of releasing rubrics to teachers. The study addresses that gap by examining the relationship between releasing rubrics to teachers and student performance task scores on a statewide arts assessment. Evaluators will conduct a longitudinal study across four years to investigate trends in scores before and after rubrics were released. To further contextualize score changes and to examine the impact of releasing rubrics on instructional practices, evaluators will collect qualitative information from teachers who administered the task. The study contributes substantially to literature on performance assessment and rubric use by investigating the assertion that releasing scoring rubrics can offer instructional benefits and improve student performance.
|
|
The Instructional Impact of North Dakota State Accountability System: A Consequential Validity Study
|
| Presenter(s):
|
| Xin Wang, Mid-continent Research for Education and Learning, xwang@mcrel.org
|
| Abstract:
The purpose of this investigation is to examine both intended and unintended consequences of North Dakota state accountability system on curriculum change and local instruction, including the use of proper accommodation practices. A mixed-methods approach is used to gather both qualitative and quantitative data from an online survey and focus groups. Teachers and principals from 70 schools will complete an online survey about the implementation of the state assessment and accountability systems and related impact on their instructional practices. Focus groups of teachers and school administrators will verify and expand upon school personnel’s survey data regarding the extent of support for the new direction of education in North Dakota, whether the organization of school district curriculum has changed, and whether the classroom teachers are aligning their instruction with the state standards. Initial findings from survey data and focus group protocols will be reported.
|
|
Defining Domains of Coaching Knowledge Using a Modified Delphi Process
|
| Presenter(s):
|
| John Sutton, RMC Research Corporation, sutton@rmcdenver.com
|
| Beth Burroughs, Montana State University, bburroughs@math.montana.edu
|
| David Yopp, Montana State University, dyopp@math.montana.edu
|
| Abstract:
With a growing interest in the use of instructional coaches to improve mathematics instruction in K-12 schools, this paper will share how a modified Delphi process was used to engage national coaching experts and practitioners to define coaching knowledge. The eight knowledge domains identified and defined through this process represent the first agreed upon definitions of coaching knowledge and are being used in a national research study funded by the National Science Foundation. The paper provides a detailed description of the three step process, along with respondents ratings of agreement and the full definitions.
|
| | | |