Health Evaluation TIG Week: Unlocking AI’s Power in Evaluation: Essential Tips & Real-World Lessons Learned on Trust, Ethics, and Innovation by Will Minor and Jenica Reed

Date: Sunday, July 13, 2025

Join us—Jenica Reed and Will Minor, from Deloitte’s Evaluation and Research for Action (ERA) Center of Excellence (CoE), writing on behalf of the Health Evaluation TIG—as we explore the role of AI in evaluation. This week’s blog series will showcase how AI tools can augment evaluation practices and offer new insights for your work across the evaluation lifecycle including developing logic models, identifying evaluation questions, creating evaluation plans, conducting data analysis, and synthesizing findings.

Throughout the week, we’ll see how integrating AI can help uncover patterns that humans might miss by crafting clear prompts and having evaluators validate AI outputs for accuracy.

Whether you’re experienced or new to evaluation or AI, you’ll find value this week. Understanding these topics is key to using AI responsibly and effectively! We’ll be kicking off the week with today’s focus on trust, ethics, and innovation in AI.

The AI Advantage: Transforming How We Evaluate

As we’ll demonstrate throughout the week, AI offers the potential to strengthen evaluations, as demonstrated by the following examples of impact:

1. Literature Reviews: AI reviews efficiently analyze academic papers, highlighting key themes and gaps, and suggesting relevant sources, saving evaluators time. Join us here on AEA365 on Monday to learn more about AI-assisted vs. traditional literature reviews!
2. Logic Model Development: AI augments constructing logic models by analyzing data to identify causal relationships, aiding evaluators in visualizing program components more effectively. Join us here on AEA365 on Tuesday for more insights on logic model development!
3. Conclusions and Recommendations: AI aids formulating conclusions and recommendations by analyzing data, identifying patterns, and suggesting evidence-based strategies. Join us here on AEA365 on Friday and Saturday for more on creating evaluation results with AI assistance!

Trust by Design: Ethics in the Age of AI

Incorporating AI in evaluations can bring ethical challenges. Here are some key lessons we’ve learned to help you adopt these advanced tools effectively:

• Lesson Learned #1: Bias and Fairness: AI may perpetuate existing biases in data. Identifying and mitigating these biases, using broad datasets, and human validation of AI models is essential for fair evaluations.
• Lesson Learned #2: Privacy and Confidentiality: It is important to protect the privacy of individuals whose data is used. This involves following data protection regulations, anonymizing data when possible, and maintaining secure data storage and handling practices.
• Lesson Learned #3: Accountability: There should be clear mechanisms to address issues related to AI-driven evaluations. Evaluators should be involved throughout to provide ongoing validation and collaboration, maintaining process integrity and effectiveness.

Code of Conduct: Building Trustworthy AI

Trust is crucial for effective AI implementation, especially in evaluations. Interest holders need confidence that AI is reliable, unbiased, and transparent. Here are some hot tips to build trust while integrating AI into your work:

• Hot Tip #1: Use High Data Quality: AI needs high-quality, representative data for accurate evaluations. Evaluators are essential to removing errors and biases as part of a rigorous data cleaning and validation process.
• Hot Tip #2: Maintain Transparency: Interest holders need to comprehend AI and decision-making processes. Evaluators should use proper process documentation, open communication, and replicable techniques to interpretable AI outputs.
• Hot Tip #3: Validate AI Models: Evaluators need to regularly test models to confirm performance as expected. Use real-world data and various prompts to validate consistent outputs. Join us here on AEA365 on Thursday to learn more about developing and validating prompts.

As we delve into AI-assisted evaluations this week, AI’s potential to strengthen the effectiveness and impact of evaluations becomes clear. What are your thoughts on AI-assisted evaluations? Please drop your thoughts in the comments section. We’d love to hear from you!

This posting contains general information only, does not constitute professional advice or services, and shall not be used as a basis for any decision or action that may affect your business. Deloitte shall not be responsible for any loss sustained by any person who relies on this posting. As used in this posting, “Deloitte” means Deloitte Consulting LLP, a subsidiary of Deloitte LLP. Please see www.deloitte.com/us/about for a detailed description of our legal structure. Certain services may not be available to attest clients under the rules and regulations of public accounting. Copyright © 2025 Deloitte Development LLC. All rights reserved.

The American Evaluation Association is hosting Health Evaluation TIG Week with our colleagues in the Health Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our Health Evaluation TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.