Date: Wednesday, December 3, 2025
I’m Anthony Clairmont, an evaluator with a background in social sciences who primarily evaluates human services programs (mental health, housing access, and jobs), as well as public spaces (libraries, museums, and city halls). I want to share some thoughts on how AI is changing the landscape of quantitative evaluation research and why I embrace this shift.
This has been the reality for most evaluators doing quantitative work since the beginning of the field has gone something like this: you work within the boundaries of your training, limited by whatever software your organization licenses, with never enough time to try alternative analytical approaches that might yield better insights. We’ve made do with SPSS, Stata, or a few R packages, sticking to the analyses we learned in school because venturing beyond meant hours of manual coding or expensive software purchases. The tradeoffs were costly in both time and money.
AI has blown these constraints to bits. I am far from the first to argue that Generative AI has the potential to improve survey research, online experiments, automated content analyses, agent-based models, and other techniques commonly used to study human behavior. Some routine data analysis tasks that used to take me hours now take minutes. The time I save gets reinvested into trying multiple analytical strategies or hammering away at my assumptions to determine which of them might make a difference. This kind of pluralistic multiverse approach was a luxury before – now I think it should be standard practice.
Speed is nice, but it has limits as long as there is a human in the loop. What are truly unlimited are the gains we will see from improved accessibility. Building custom software for specific evaluation needs is no longer the province of dedicated programmers. Need a tool to automate your data quality checks? Build it with Claude Code or Cursor. Want to implement that brand new statistical test from a recent journal article?
With all this excitement, we need to be careful not to let ourselves get carried away. It is now easier than ever to implement analyses you don’t fully understand. Just because AI can help you spin up a spatial regression doesn’t mean you grasp the underlying assumptions. More conceptually, having infinite analytical flexibility can lead us deeper into the garden of forking paths. When you can try ten different approaches in an afternoon, the temptation to cherry-pick results that support your preferred narrative intensifies. We need to remember that our ethical obligations as evaluators do not change just because our tools have.
AI-augmented evaluation is about augmenting our expertise rather than replacing it. The Stanford AI Index 2025 highlights major gains in model performance, record levels of private investment, and growing real-world adoption. If you are a leader in quantitative methods on your team, there’s no longer any excuse for not pressure-testing your findings or building a custom tool that could improve your workflow.
The evaluators who will thrive in this new landscape are those who pick up these tools while maintaining rigorous standards and embracing the philosophical depth of the discipline. In this new paradigm, the rules have shifted from ‘work within your constraints’ to ‘expand your capabilities’, but the fundamental rule remains unchanged: let your expertise guide the technology, not the other way around.
The American Evaluation Association is hosting Quantitative Methods: Theory & Design TIG Week. The contributions all this week to AEA365 come from evaluators who do quantitative methods in evaluation. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.