We were involved in a project trying to understand the impact of a new school board program. The program was intended to reduce LGBTQ+ bullying and improve the inclusive climate within the schools. The boards wanted to know how much improvement we saw in LGBTQ+ students feeling comfortable at their schools. This is a great outcome to study, but they initially crafted their research question like this: “Do LGTBQ+ students now feel better about school?”.
At first glance, this is a fine research question. It aims to be equitable and inclusive, but it’s actually problematic. It frames the research around changing the students’ feelings rather than around changing the school environment. Feeling negatively about an environment that is hostile towards you is a healthy, appropriate feeling that the students might have. We were trying to directly accomplish a change in our school environment, with an outcome of increasing student comfort. This was always what they meant, it just wasn’t well reflected in the research framing. We needed to craft a question where the responsibility for the ‘success’ or ‘failure’ of the program was in the most equitable place. In this project we went with: “Has the school environment become less hostile to LGBTQ+ students?”
At We All Count, we like to think of the different variables involved in a project as puzzle pieces that can fit together well, poorly, or not at all. When we design research questions we try to decide which piece to rotate, shift or swap out to see improvement or ‘positive change’. Choosing which piece to study is easy when you ask ‘What’s the most equitable piece to adjust?’.
Consider the research question from another project: “What factors and trends are causing the vulnerable Indigenous children in Australia to have poor health outcomes relating to burns?” Is this an equitable research question or world view? The expectation that indigenous populations change to adjust to ‘the system’ rather than the other way around reflects a common colonial worldview and reinforces an underlying assumption that non-indigenous people ‘figured out’ how to use a system rather than the reality in which that system was constructed specifically to work for them. Acknowledging this allowed the question to be reframed to examine how the healthcare system better supported white children with burns and how that effectiveness could be extended and altered to serve a more inclusive group. The researchers in this case changed the research question to be “How can an understanding of the ways in which healthcare systems produce advantage and positive health outcomes for white Australians help improve Indigenous healthcare.”
Lastly, examine the research question: “How can we keep Hispanic boys from being expelled from our schools at a higher rate than non-Hispanic boys?”. This question arose out of another school district project aimed at reducing the rate of expulsion of a specific group of young men. At the outset of the project Hispanic boys and their community of parents were understandably resistant to the study. The question made it seem like there was something wrong with the Hispanic boys.
When the question became “What processes in our school are most strongly related with pushing out Hispanic boys?” and “What school characteristics are most strongly related with creating environments that encourage Hispanic boys to fulfill pre existing desires to remain in school?” the project suddenly saw broad support, was able to change methodologies and get more participatory engagement – so changing research questions isn’t just about the analysis stage, it fundamentally affects your entire project. In this example, it was always the school’s intention to change rather than dismiss the attitudes, habits and goals of any of their students, it just wasn’t coming through in the research question. Make sure that your research questions are embedded with the equity that you are trying to foster!
This is a great article with lots to think about! I’m curious, though: in the second example, the original research question (What factors and trends are causing vulnerable Indigenous children to have poor healthcare outcomes related to burns?) seems more similar to the revised question in the third example (What processes in our school are most strongly related with pushing out Hispanic boys?). To me, the second question’s revision is potentially problematic in the way that it centers the white experience. If we’re focused on exploring how a system advantages white patients, that doesn’t necessarily mean that we are also focusing on the specific factors that disadvantage other groups, though the two may be related.
Thanks so much Callie. Completely agree that there are pros and cons to most ways to revise research questions. I also have questions about the healthcare question centering white experiences. However, the Aboriginal researchers designing the project wanted the question in the revised way. What I’m hoping is that we can embed an equity assessment of research questions as a quantitative best practice and then move the ball forward to meet the specific objectives of each project. The best and most equitable question for one context or project won’t be the one for another.
That’s totally fair, and I appreciate knowing the extra context that the researchers designing the project were Aboriginal. Thanks for responding! I think embedding equity-focused assessments (or meta-evaluation, as the case may be) of research/evaluation questions has the potential to make a huge difference.
Thanks Callie! I agree totally.