Date: Saturday, September 12, 2026
Hello! I am Carmen Carrillo, a DrPH student at the Johns Hopkins Bloomberg School of Public Health, and an evaluator with the County of San Diego’s Office of Evaluation, Performance, and Analytics.
During my DrPH practicum, I explored a question that may feel familiar to many evaluators: In times of change, how can we tell whether an organization is truly ready to use evidence, rather than simply being equipped to produce it?
Many organizations can point to trained staff, dashboards, plans, tools, data systems, reports, and analytic teams. These supports matter, but they are not enough. An organization may have strong tools and strategies while lacking a culture that encourages people to question assumptions, discuss what is not working, learn from findings, and change course.
A capacity assessment can show whether staff, tools, budgets, plans, and reporting processes exist. It may not show whether they change everyday behavior. Culture becomes visible when we examine what leaders and teams repeatedly do, protect, reward, and use in decisions. It shapes whether evidence is welcomed or resisted, difficult findings are explored or avoided, and learning leads to action.
During my practicum, I developed an Integrated Evidence Capacity Framework with five domains: culture and leadership; people and tools; evidence building processes; learning and feedback loops; and decision integration. Culture and leadership are the foundation of this framework because technical capacity alone does not create evidence use. Organizations also need trust, curiosity, reflection, dialogue, and leadership that supports learning.
Instead of asking only, “Do we have the tools, and are people trained?” it asks, “Are we behaving like a learning organization?”
Sample questions include:
These questions shift the conversation from infrastructure to practice and distinguish evidence production from evidence use.
Use the questions as a checklist with a leadership team, program team, or evaluation partner. Rate each behavior as consistently, sometimes, or not yet. Then select one behavior to strengthen during the next 60 to 90 days.
For example, add one standing question to a leadership meeting: What are we learning, and what should we change because of it?
The questions can serve as a structured prompt for reflection, discussion, and action. Its value is making expectations visible and helping teams identify one observable next step.
For evaluation capacity building practitioners, the key question is not only, “Can this organization produce evidence?” It is, “Does its culture enable people to question, learn, and act on what the evidence shows?”
The American Evaluation Association is hosting Organizational Learning and Evaluation Capacity Building (OL-ECB) Topical Interest Group Week. The contributions all this week to AEA365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please do so using the form on our website. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.