Earlier this week, I attended a webinar on data equity. For an hour, statistician Heather Krause talked about some of her work experiences where her internal biases and assumptions meaningfully changed the results of her analyses and gave some tips for spotting these in future work.
At Scioto Analysis, we believe that equity should be considered in every policy analysis. The truth is that while equity is always a part of policy adoption, the only thing that changes from an analytic standpoint is whether or not we choose to acknowledge it.
Consider this example: we have three classrooms, one with three students, one with six students, and one with nine students. What is the average number of students in each class? This is an easy enough calculation, (3+6+9)/3 = 6. As simple as this seems, it actually relies on an important assumption about equity. In this case the variable we are measuring is classroom size.
Instead, let's consider things from the students’ perspectives. What is the average class size that a student experiences? In this case, the variable of interest is classroom size for each student. Here, our calculation becomes much larger. If you add up the experiences of all 18 students within these three classrooms, you get (3+3+3+6+6+6+6+6+6+9+9+9+9+9+9+9+9+9)/18 = 7.
Now we have two different conclusions from the same data. Although in this case the results are quite close, we still need to ask ourselves which of these results is more accurate.
This depends entirely on the question we are trying to answer. If our research is about how smaller classroom sizes affect teachers, then saying the average class size is six best reflects how teachers are experiencing classroom size.
If instead we are trying to measure the effect of class size on students, then the second number better reflects how students are experiencing classroom sizes.
This example is meant to show that all of our assumptions have equity implications, whether we notice them or not. When I first saw the classroom example, I immediately thought that six was the only correct answer. It did not cross my mind to reframe what variable we were trying to take the average of and how that could possibly influence the equity of the results.
In this webinar, we also talked about how equity can fit into every part of the analysis process. Is the data being collected in an equitable way? Is the final report being written to discuss the equity implications of your research? Depending on the situation, as analysts we might not be in charge of some of these steps. However, we need to understand how these assumptions influence our results.
The good news is that being careful about including equity in an analysis is almost exactly the same as simply being a good analyst. Identifying assumptions, understanding their implications, and honestly acknowledging them is the core of good analysis.
In this sense, more equitable analysis is the same as more scientifically rigorous analysis. The difference is that we need to ask more questions about our own internal biases and assumptions as researchers and make sure they are not getting in the way of giving policymakers the answers they need.