| Session Title: Summative Confidence: How Accurate are Your Evaluative Conclusions? |
| Expert Lecture Session 731 to be held in International Room on Saturday, November 10, 9:35 AM to 10:20 AM |
| Sponsored by the Quantitative Methods: Theory and Design TIG |
| Chair(s): |
| Brooks Applegate, Western Michigan University, brooks.applegate@wmich.edu |
| Presenter(s): |
| Cristian Gugiu, Western Michigan University, crisgugiu@yahoo.com |
| Abstract: One of the cornerstones of methodology is that "a weak design yields unreliable conclusions." While this principle is certainly true, the constraints of conducting evaluations in real-world settings often necessitate the implementation of less than ideal designs. To date, no quantitative or qualitative method exists for estimating the impact of sampling, measurement error, and design on the precision of an evaluative conclusion. Consequently, evaluators formulate recommendations and decision makers implement program and policy changes without full knowledge of the robustness of an evaluative conclusion. In light of the billions of dollars spent annually on evaluations and the countless millions of lives that are affected, the impact of decision error can be disastrous. This paper will introduce an analytical method that can be used to estimate the degree of confidence that can be placed on an evaluative conclusion and discuss the factors that impact the precision of a summative conclusion. |