| Session Title: Using Concepts From Complex Adaptive Systems and Agent-based Modeling to Understanding the Role of Federal Programs in Improving College Access and Completion |
| Multipaper Session 616 to be held in Balboa C on Friday, Nov 4, 10:45 AM to 11:30 AM |
| Sponsored by the Government Evaluation TIG and the Systems in Evaluation TIG |
| Chair(s): |
| Sharon Stout, United States Department of Education, sharon.stout@ed.gov |
| Discussant(s): |
| Margaret Cahalan, Pell Institute, margaret.cahalan@pellinstitute.org |
| Abstract: Evaluation as a field has been tending away from simple causality models with a focus on single static counterfactual comparisons. It's recognized that programs evolve and adapt over time to changing conditions and that there are multiple systems and interacting lines of causality (Morrell et.al 2010). The first paper examines theories of causality and explores alternative ways of viewing the Academic Competitiveness Grants and National SMART grants programs to test how and whether the contributions of these programs can be distinguished from other federal, state, and institutional changes. The second paper using data aggregated at the national, state and school levels treats these entities as nested complex adaptive systems, investigating how agent-based modeling may inform our understanding of past and future estimates of college attainment. The paper observes what patterns and college access service levels would need to change and how altering assumptions changes estimates towards the President's 2020 goal. |
| How do we Frame and Model Federally-Funded Education Programs to Improve Programs Addressing College Persistence and Success? |
| Sharon Stout, United States Department of Education, sharon.stout@ed.gov |
| The current administration appears to be relying more on broad goals and performance measures, using the term 'contribution' in considering accountability (John Mayne, 2008). However, as federal programs remain relatively narrowly focused, even these relaxed accountability requirements suggest a mismatch of evaluation methods and evaluands. Particularly problematic are weak links between actual interventions, evidence, and broadly defined outcomes. This paper reviews the philosophical literature on causation (Pearl, 2000; Pearl, 2010) and on recent methodological advances (e. g., structural equation models, graphical causal models, targeted learning). Multiple frameworks and methods are then applied to the challenge of discerning what the Academic Competitiveness Grant (ACG) program and National Smart Grant (NSG) Programs contributed to the persistence of low-income youth in college and their choice of college majors. National survey databases and financial aid administrative data (and longitudinal data from Florida, Texas, and Indiana) are used - and shortcomings in these data assessed. |
| Treating the Us Educational System as a Complex Adaptive System, and Investigating Agent-based Models of Student Behavior and Computational Simulation of Federally-Funded Access Programs |
| Margaret Cahalan, Pell Institute, margaret.cahalan@pellinstitute.org |
| Stephanie Miller, Pell Institute, stephanie.miller@pellinstitute.org |
| Kelly Middleton, Pell Institute, kelly.middleton@coenet.org |
| Reaching the President's educational goals of regaining a global lead in percent of citizens with postsecondary degrees will require raising the percentage of Americans, age 25 to 64, with a postsecondary degree from 41.2% to nearly 60.0%. At the current pace, projections suggest educational attainment is expected to be roughly 46% in 2020, leaving us nearly 24 million degrees shy of the 60% target rate. This paper treats the US educational system as a complex adaptive system, and investigates how agent-based models of student behavior and computational simulation of the federally-funded programs at the school, state, and district level may inform national estimates of college attainment. We examine how altering assumptions changes estimates towards the President's 2020 goal. The paper uses data from the American Community Survey and NCES surveys as well as evaluation and performance data from federal college access programs such as Talent Search, and Upward Bound. |