Date: Tuesday, July 7, 2026
Hello! I’m Dr. Freeman Gerhardt, a program evaluator with the University of South Florida who works primarily in behavioral health and substance use services. Much of my work involves evaluating community-based programs designed to improve access to care and support recovery. Today, I’d like to share a lesson learned that has influenced how I approach evaluation planning and stakeholder engagement.
One of the most common challenges I encounter is that stakeholders want evidence that a program is working, but the data needed to answer that question are nonexistent. In behavioral health, programs are often launched quickly in response to urgent community needs, funding, policy changes, or emerging crises. Meanwhile, data-sharing agreements, outcome tracking systems, and databases can take months or years to develop. By the time those systems are in place, valuable opportunities to collect meaningful data have passed.
I used to view this as a major obstacle. If we can’t measure outcomes such as reduced overdose risk or treatment retention, how can we conduct meaningful evaluations? Over time, I realized this was the wrong question. When outcome measures are limited, I often start with three practical questions.
Many programs routinely collect referral, enrollment, and participation data, which can determine whether services reach the people they were designed to serve. If a program is not reaching its intended population, outcome data alone may not explain why results are falling short.
I increasingly rely on geographic analysis to identify patterns that are difficult to see in tables or standard reports. Simple maps can reveal service gaps, concentrations of need, or inequities in access that are not immediately obvious. You don’t need a full GIS workflow to start using geography in evaluation. Excel includes built-in mapping features that allow you to quickly visualize geographic patterns. For evaluators with access to ArcGIS, an evaluator can transition from basic exploratory maps to more advanced spatial analysis.
Implementation data can be incredibly valuable. Staffing shortages, referral bottlenecks, data-sharing barriers, and inconsistent service delivery can all affect outcomes. Understanding how the program operates in practice helps stakeholders distinguish between implementation challenges and shortcomings in the intervention itself.
Many behavioral health programs produce intermediate outcomes long before other outcomes become visible. For example, a post-overdose outreach program may increase successful treatment referrals. A crisis response initiative may strengthen collaboration among community partners. These changes may not immediately translate into reductions in overdoses, hospitalizations, or other long-term outcomes, but they often represent meaningful progress toward those goals.
While everyone wants to know whether a program “worked,” decision-makers also want actionable information. Findings about service reach, workflow challenges, and referral patterns frequently lead to immediate program improvements. The lesson I wish someone had taught me earlier is that evaluation is rarely about finding the perfect dataset or outcome measure. It is about helping stakeholders learn from the information available and making thoughtful decisions in the face of uncertainty.
Of course, this does not mean we should abandon efforts to collect outcome data. Strong outcome measures remain an essential part of evaluation. However, waiting for perfect data can delay learning opportunities and limit the value evaluation provides to programs and communities.
Sometimes the greatest contribution is to help stakeholders understand what is happening right now, identify opportunities for improvement, and scaffold for stronger outcome measurement in the future.
The American Evaluation Association is hosting Behavioral Health TIG Week with our colleagues in Behavioral Health Topical Interest Group. The contributions all this week to AEA365 come from our Behavioral Health TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.