Date: Wednesday, November 11, 2026
Hi everyone! We are Melissa K. Demetrikopoulos, Past-Chair of STEM Education and Training TIG and Director of Scientific Communications at the Institute for Biomedical Philosophy, and Andreas C. Thompson, Data Analyst at the Institute for Biomedical Philosophy. The Institute for Biomedical Philosophy is dedicated to university support services and partnership development, with specific interest in evaluation and research processes within an ethical framework that supports the development of a thriving community of practice.
Over time, changes have been made in how demographic questions are asked and how many are given. Changes to factors such as race, ethnicity, gender, and gender identity have often increased the granularity of answer responses in an attempt to make these questions more inclusive. However, many of these same changes increase the possibility of individuals being identifiable within a data set. This is especially problematic with small data sets (a few hundred participants), as are commonly found in educational settings. As few as three questions can often identify individuals in small data sets, and as additional questions are added (year in school, major, etc.), data becomes more and more identifiable. Because of this, if it is critical to include demographic questions within your data set, it is more appropriate to describe data collection as being ‘confidential’ rather than anonymous, since at least some participants will be identifiable within the data set. Furthermore, care must be taken in the reporting of data that is disaggregated in any manner, in order to ensure that individual participant responses cannot be determined; for example, the use of data suppression thresholds can be employed, where cells with fewer than a predetermined number of respondents will be left unreported.
As most surveys are now collected electronically, there are additional factors that can inadvertently make anonymous data identifiable. Here are some strategies that help: Use general links to surveys rather than having surveys linked through email address invitations. If you are using a system that sends out reminders to only non-responders, know that your data collection is confidential at best and is not anonymous. Avoid tools that collect meta data such as IP addresses, and strip any meta data including time and date stamps from data sets since these may be used to help identify participants if data is not all collected at the same time.
Carefully consider why you are asking demographic questions and how this information will inform your work so that only explicitly necessary questions are ever asked, always make demographic questions optional, and include a “prefer not to answer” option. While it may seem desirable from an inclusion perspective to include open ended options for demographic questions such as “Prefer to self-describe (please specify),” these formats are most likely to make an individual participant identifiable. There are some approaches that can help when you need to report the demographics of your participants, even if you do not have enough participants to realistically disaggregate the data by demographic factors for analysis. In this case, collect the demographic data in a separate survey that is given as a link in the primary survey. This ensures demographic data are never tied to survey responses, and cannot be used to identify a participant’s individual survey, while allowing you to fully describe your study population.
Some other considerations concerning the ethical collection of demographic data can be found in Chomsky’s blog https://aea365.org/blog/data-ethics-week-collecting-demographic-data-with-care-by-inti-chomsky/
Nikolaidis offers some great strategies for writing survey questions that are more likely to result in anonymous answers https://www.askattest.com/blog/articles/anonymous-surveys
The American Evaluation Association is hosting Educational Evaluation TIG Week with our colleagues in the Educational Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please do so using the form on our website. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.