|
Session Title: Findings Patterns in Evaluation Data: Searching for Clues That May Help Better Understand Programs and Policies
|
|
Panel Session 559 to be held in Centennial Section B on Friday, Nov 7, 10:55 AM to 11:40 AM
|
|
Sponsored by the Quantitative Methods: Theory and Design TIG
|
| Chair(s): |
| Patrick McKnight,
George Mason University,
pmcknigh@gmu.edu
|
| Abstract:
There are many patterns in quantitative data. Unfortunately, quantitative analyses of program and policy evaluation data tends to be mechanistic and structured to find only one pattern in our data. The pattern we seek is one that fits a linear relationship with normally distributed residuals. In short, we seek to confirm the general linear model at some level. There are other patterns in our data that may be indicative of program effectiveness and without an focused effort to search for these patterns we will not discover them. The purpose of this talk is to introduce the concept of pattern recognition as a data analytic routine to stimulate interest in this budding area of quantitative methodology. Two presentations cover both the theoretical underpinnings of pattern recognition as well as specific examples from real program evaluations in education.
|
|
Discovering Patterns in Longitudinal Data
|
| Patrick McKnight,
George Mason University,
pmcknigh@gmu.edu
|
|
Discovering patterns in longitudinal data may help us better understand who changes, to what extent, and under which circumstances. Some individuals may show no change while others change in odd ways. Only through a deliberate effort to find these different patterns may we come to this realization. what's more, finding the patterns allows us to then seek predictors for the different patterns. These patterns may also lead to insights into the nature of how programs or policies may be effective. The purpose of this talk is to demonstrate pattern discovery methods. Data from education and mental health provide a set of examples where longitudinal data may be better characterized by pattern recognition methods. These methods are contrasted with more traditional longitudinal analyses that are typical in contemporary social science and program evaluation.
|
|
|
Assessing Patterns of Readiness for Program Engagement
|
| Katherine McKnight,
Pearson Achievement Solutions,
kathy.mcknight@pearson.com
|
|
Composite variables are combinations of variables that are individually meaningful and are thought to be indicators of the same construct. For example, socioeconomic status is often measured as a combination of income, household size, occupation, educational level and so on. Problems arise in determining how to combine these variables to produce a useful measure of the given construct. We typically sum the scores for each variable to create an index, which assumes that each variable contributes equally. In this paper, we discuss the use of pattern assessment applied to an index measuring readiness for effective engagement in teacher workgroups. Assessing patterns of scores for the different variables of the index--e.g., administrative support, identified 'point person,' etc.-- allows us to assess the contribution of each variable to 'readiness' and weight it accordingly. Applying pattern assessment for multidimensional composites helps to better understand the phenomenon and to create more thoughtful measurement.
| |