Date: Saturday, June 27, 2026
I am Mansoor Kazi, PhD, with over 30 years of evaluation experience in the USA and England. I am now retired from universities but still serve as President of Realist Evaluation Inc.
Evaluators and service providers can build partnerships that reach across professional, methodological, and even international boundaries, and in doing so make evaluation far more useful in everyday practice. The approach I want to share is a longitudinal one. Instead of analyzing data once at the end of a year, evaluators and agencies analyze big data together at regular intervals, drawing on entire service populations. That partnership helps agencies target their interventions more precisely, promote equity and social justice, and adjust course where an intervention is not yet working.
Realist evaluation asks a practical question: what works, for whom, and in what circumstances? To answer it, we can use research methods drawn from both epidemiology and effectiveness-research traditions, applied to the live data that agencies already collect in their management information systems, including schools, social services, mental health, and youth justice. Because the data comes naturally from practice, we can use quasi-experimental designs, matching intervention and comparison groups on demographic and contextual variables rather than randomizing people into them.
In practice, realist evaluation means the systematic analysis of three things: the service users’ circumstances; the dosage, duration, and frequency of each intervention for each user; and repeated, reliable outcome measures for every user. When evaluators work alongside agencies to clean and analyze that data throughout the year, findings can inform practice in real time, including questions of diversity and where, and with whom, an intervention is more or less effective.
Because the data covers all service users, for example every child in a school district, you can compare outcomes between matched intervention and non-intervention groups. The same data supports the family of methods used to establish epidemiologic evidence based on association, environmental equivalence, and population equivalence.
Binary logistic regression is a useful method here. Candidate variables are identified through bivariate analysis, entered in a forward-conditional model, and the ones that hold are retained. The significant factors yield an exponential beta, the odds of the intervention achieving the outcome when that factor is present. That is how you locate the conditions under which an intervention is more or less likely to succeed, and how you turn findings into practice changes that keep advancing equity and social justice.
None of this happens at arm’s length. The evaluation crosses professional boundaries between evaluators and frontline providers, epidemiological boundaries where effectiveness research meets population methods, and international ones, since the same approach has been applied across the USA and the United Kingdom. The common thread is a working partnership in which evaluators and agencies establish cause and effect together, and use what they learn while there is still time to act on it.
To learn more about realist evaluation and these partnership-based methods, check out:
The American Evaluation Association is celebrating Human Services Evaluation TIG Week with our colleagues in the Human Services Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our Human Services Evaluation TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to aea365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.