Date: Sunday, October 18, 2026
Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Hello! I’m Arthur Hernández, an evaluator and educator interested in evaluation practice, community engagement, and the forms of professional judgment required when technology changes how knowledge is produced. I want to suggest that AI-assisted evaluation requires a new layer of evaluator reflexivity.
Evaluators have long been expected to ask reflexive questions: How do my assumptions, positionality, relationships, culture, institutional location, and power shape the evaluation? What do I notice? What do I miss? Whose interests am I advancing? Whose knowledge do I treat as credible? Those questions remain essential.
But AI-assisted evaluation adds another layer. Evaluators must now also ask: How does the tool shape the work? What assumptions are embedded in the model? What forms of language, evidence, culture, and reasoning are privileged by the training data? What constraints are imposed by vendor policies, interface defaults, system settings, or hidden design choices? How does my prompt language steer the output? What does the model make easier to say, harder to see, or too quickly resolved?
This is not a technical side issue. It is part of professional competence in an algorithmic era.
AI may help draft, summarize, classify, translate, code, compare, or generate options. But it does not relieve evaluators of responsibility. The danger is not only that AI may be inaccurate. The deeper danger is that AI may sound plausible while narrowing the evaluator’s imagination, flattening community meaning, normalizing dominant language, or hiding uncertainty behind fluent prose.
Evaluator reflexivity must therefore include tool reflexivity. I need to ask not only, “What am I bringing to this evaluation?” but also, “What is the tool bringing into this evaluation through me?” That includes examining whether AI-generated language changes the tone of findings, whether coding suggestions reproduce stereotypes, whether summaries erase disagreement, whether translation alters meaning, and whether recommendations become more generic, managerial, or detached from lived context.
The language we use with AI is not neutral. Prompts frame the evaluand, define the audience, name the problem, assign importance, and invite particular kinds of evidence. A prompt can center compliance, efficiency, accountability, equity, learning, community voice, institutional risk, or funder preference. Each choice matters.
For that reason, evaluators should document consequential AI use. What tool was used? For what purpose? With what data? Under what privacy conditions? What prompts or instructions shaped the output? What human review occurred? What was accepted, rejected, revised, or questioned? Transparency about AI use should become part of methodological transparency, not an optional appendix.
A practical AI reflexivity memo might ask: What work did AI perform? What assumptions did it introduce? What perspectives were amplified or minimized? What risks were created for confidentiality, fairness, representation, or interpretation? How did I verify, contextualize, and take responsibility for the final product?
AI-assisted evaluation does not eliminate evaluator judgment. It raises the standard for it. In this moment, professional competence requires more than learning how to use AI efficiently. It requires learning how to interrogate AI critically, ethically, and reflexively before its outputs become our conclusions.
Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please do so using the form on our website. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.