Date: Thursday, May 8, 2025
My name is Marlana Lastres, and I am an instructor in the College of Education & Human Sciences at Tennessee Tech University. I teach graduate courses on qualitative research and at-risk populations, and I also work in evaluation, often collaborating on grant projects, accreditation tasks, and program reviews. Recently, the use of generative Artificial Intelligence (AI) tools has rapidly grown in prominence across many fields. In educational evaluation, these tools offer a thorough and expedited way for evaluators to create meaningful evaluations and assessments for a variety of projects. While this innovation is incredibly useful and efficient, it should be used with caution. One significant concern for evaluators is the risk of developing evaluations and assessments that inadvertently reinforce gender bias and perpetuate inequity, especially in education.
As AI adoption accelerates, it’s essential to ask whether these tools are fostering fairness or reinforcing existing educational biases. As powerful and promising as AI is, we must remember it does not operate in a vacuum. It is a product of the data it is trained on. If the data reflect historical biases, the AI could inadvertently “copy and paste” those biases into its decision-making processes. This often results in perpetuating gender inequities that have plagued educational systems for decades. I encountered similar concerns in an interview with Zinnya del Villar for UN Women, which focused on how AI reinforces gender bias, especially in hiring decisions and healthcare.
So, what does this potential discrimination in evaluation look like in practice?
AI systems are typically built using large datasets and learn patterns based on historical data. If these datasets reflect historical gender imbalances, the AI may reproduce these patterns in its output. For example, if an AI tool was trained using data that tracks student success in STEM fields and male students were overrepresented, the AI might prioritize male students in its recommendations or support systems. Another way AI may perpetuate gender inequity is through the use of gendered language and stereotypes embedded in datasets or past literature. Generative AI tools can fail to account for those who have been excluded, marginalized, or underserved historically. As evaluators, it is our responsibility to put these at-risk groups at the forefront of our work in order to dismantle these power structures and give voice to those who have been previously silenced.
What can we do as evaluators to combat this concern?
1. Audit & Diversify DataEvaluators must regularly review the data used to train AI tools, with a focus on identifying and addressing gender imbalances. We play a crucial role in ensuring that the data are representative of all genders so that AI systems consider the needs of diverse populations. Whenever possible, inclusive gender categories should be encouraged and implemented.
2. Apply Human Oversight & Spread AwarenessHuman oversight and clear detection protocols can help prevent inequity within evaluations. Generative AI tools cannot fully replicate human critical thinking skills. To maintain diverse perspectives in our evaluations, it is essential that we review all AI output with an eye toward gender equity and continuous improvement. Evaluators should also raise awareness among stakeholders about the risks associated with AI use in educational evaluation processes.
3. Advocate for Transparency & CollaborationEvaluators should advocate for transparency in AI development and for the clear communication of how datasets and algorithms are selected and used. By engaging with AI developers and other stakeholders, evaluators can foster collaboration and ensure that efforts are made to mitigate gender bias and promote fairness.
The American Evaluation Association is hosting Educational Evaluation TIG Week with our colleagues in the Educational Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.