| Session Title: Cluster, Multi-site, and Multi-level TIG Business Meeting and Demonstration: A Mixed Methods Approach to Measurement for Multi-site Evaluation |
| Business Meeting Session 367 to be held in Lone Star F on Thursday, Nov 11, 4:30 PM to 6:00 PM |
| Sponsored by the Cluster, Multi-site and Multi-level Evaluation TIG |
| TIG Leader(s): |
| Rene Lavinghouze, Centers for Disease Control and Prevention, shl3@cdc.gov |
| Martha Ann Carey, Maverick Solutions, marthaann123@sbcglobal.net |
| Presenter(s): |
| Fred Springer, Evaluation, Management & Training Associates Inc, fred@emt.org |
| Wendi Siebold, Evaluation, Management & Training Associates Inc, wendi@emt.org |
| Carrie Petrucci, Evaluation, Management & Training Associates Inc, cpetrucci@emt.org |
| Abstract: This mixed-method approach to measurement for use in multi-site evaluations (MSE’s) treats natural diversity in program context and implementation as a learning opportunity, rather than a challenge to internal validity. Capitalizing on the differences within and across multiple sites, knowledge for evidence-based practice is built inductively by measuring naturally occurring variability across sites, and using it to identify robust relationships with program effects. Multi-level analysis examines relations and interactions at and across individual, group, process, and context levels, and provides strong tests of external validity. Measurement is developed inductively, and combines the use of available and primary data collection, including observational data, semi-structured interviews, standardized instruments, and administrative data. This multi-site, multi-level approach enhances the quality of evaluation by using a “site-level protocol” with measures that are pertinent to practice. The epistemological foundation will be discussed, followed by explicit examples of how this approach is implemented from design to analysis. |