Here at We All Count, we spend a lot of time talking about research questions. Step 3 of the Data Equity Framework is Project Design, and that’s where you want to embed equity while crafting your research question(s), but in order to that, you need to know what a research question is.
A data project research question defines what information you want from the data, written in a way that is concrete, measurable and can be answered with data.
Research questions are used in corporate marketing, public policy, academia, UX design, mission-driven organizations and more. Basically, any time you’re trying to get specific answers from data, you have a research question. Some data projects are made up of many research questions, while others are trying to answer just one.
Example: If we’re working for an organization that is trying to improve the school performance of students by making sure they have access to a healthy breakfast, one of our research questions might be:
“Did math test scores increase for the students who participated in our breakfast club at least once a week?” – This question is specific, measurable and can be answered with data.
Things that are not research questions:
“Is our program working?” – This question is too general, it’s more related to the overall goal or motivation of the data project. There are many specific research questions that could come out of this general question.
“Did you eat breakfast this morning?” – This is a survey question and survey questions sometimes get confused with research questions. Survey questions can provide data that is used to answer research questions, but they aren’t what we mean when we say ‘research question’.
Now that you have a good sense of what a research question is, you might encounter one of the following scenarios when trying to figure out your research question:
Scenario 1: You have a good research question, you just call it something else.
Do you have a Logic Model or Theory of Change?
If you have a Logic Model or Theory of Change, your research question is going to be found in the lines or arrows of the model. If you have a line showing that one thing connects or causes another thing, testing that relationship is your research question.
Do you have a Learning Agenda?
If you have a Learning Agenda, there is a research question in there. The research question is going to be found in the sentences that are the most specific and say what you are hoping to understand – what relationship, cause, or outcome you’re trying to estimate.
Do you have a Metrics or KPI Plan?
If you have a Metrics or KPI plan, your research question is most commonly found in the changes in Key Performance Indicators (KPI) or Metrics, how or why this change happened, and what is associated with this change.
Scenario 2: You skipped defining a research question and reverse engineered one from the data you collected.
Sometimes you go from a general question like “is our program working?” right into data collection and analysis without figuring out any specific questions. Maybe you collected data on student test scores and breakfast club attendance and in your report, you say something like:
“80% of students who attended the breakfast program regularly, saw an average improvement in math test scores of 15%”.
This is the answer to a specific research question you just never wrote out. The downside of unconsciously creating research questions is that you let your instincts and assumptions direct what kind of data you collect and how you use it. This can lead to equity problems like reinforcing a specific worldview, ignoring alternative questions, and assuming what kind of data will be the most relevant.
Scenario 3: Your methodology dictated your research question for you.
If you choose what kind of data project you’re going to do first, you will severely limit what kind of research questions you can ask. This happens all the time. If you are required to do a Randomized Control Trial, your research question will have to be about the average effect of your program as a whole.
If we are obligated to do a Randomized Control Trial on our breakfast club kids, we will never be able to answer a question like: “Is the breakfast club effective for the students with the worst GPAs?” or “Whose test scores are improving the most?”.
We highly recommend taking control of your data project by defining your specific questions first then matching them to an appropriate methodology.
Scenario 4: You are worried about the equity of your research question.
There are many ways that your research questions affect the equity of your projects. Whose perspective they reflect, what data they can be answered by, how they are defined, and even how many of them you have, matter. To get started, check out our piece on how to frame your research questions equitably.
There’s a lot more to know about research questions, and on top of that, there are more things to know about their equity implications. Now that you know what a research question is, you can start identifying and improving them in your work!