When it comes to data projects, one of the most powerful tools we use is what we call a Motivation Touchstone. It’s a document that you and your team can return to over and over for guidance when figuring out how to make effective and equitable decisions at every step in the data process.
A Motivation Touchstone is made of 4 parts:
- Whose perspective (or perspectives) you want to center when making decisions.
- A specific core motivation.
- The restrictions you need to adhere to.
- The rewards you hope to gain.
In this piece, we’re going to talk about the third part: Restrictions.
In setting our data project up for success, we need to not only understand our core motivation we also need to consider other factors.
Let’s imagine that instead of a data project, we’re working on a space project! Our space agency has received funding to build a rocket to go to the moon. Our team of engineers excitedly starts drawing doodles of rocket designs and soon enough we got three equally awesome but very different rocket designs. Which one should we build? All of them would likely make it to the moon, but is “build the best rocket to make it to the moon” (our very basic core motivation) the only requirement for this mission to be successful? No. It’s always more complicated than that.
There are 4 categories of restrictions that are key considerations in almost any project:
- Rules & Regulations
Building the “best” rocketship could take decades or even stretch on indefinitely as we come up with more and more design improvements. It’s unlikely that we’ve got unlimited time to get to the moon. Maybe we need to land on the moon when it’s at a certain part of its orbit. Maybe our funder set an arbitrary deadline of 6 months. In this case, we want to beat a rival space agency up there and we think that their rocket could be ready in as little as 2 years! The restriction of time is going to significantly modify our decision-making. Now we’re not just aiming for the “best” rocket, we’re aiming for the best rocket built in the next 2 years.
Much like time, our budget is rarely unlimited. We’ve got 2 years, and 40 million dollars. Ok, so it’s the best rocket built in the next 2 years, within a budget of $40M. Looks like the diamond-titanium frame on this one will take too long to build and cost too much money.
Next, we need to consider capacity. This can mean our resources, facilities, expertise, work hours, etc. We’re looking for the best rocket we can build in the next 2 years, for $40M or less.
Lastly, we need to consider rules and regulations. This means everything from laws to internal ethical guidelines and general agreed to practices. We can’t test our new nuclear engine near this small town. We can’t use human subjects to see how fast our rocket can accelerate without turning them into mush. Etc. Rules and regs often set limits on how you can go about your project.
Ok, so you can see how this information on restrictions provides vital parameters to succeeding in picking a rocket design, but how does this matter for equity? Let’s take a look at a more *ahem* down-to-earth example.
We want to know, are our after-school soccer programs effective at reducing suspensions for young boys in our school district? What kind of data project should we design to get an answer?
Time: We need to present the results to our Board at the Annual General Meeting (AGM) in six months. This really, really limits the possible methodologies we can use to answer this question. No longitudinal studies in that time frame.
Money: Our entire soccer league only has a budget of $80,000, and we’ve only been allocated $6000 to answer this question, crucial to the survival of our program. Outside consultants, fancy digital tools, and expensive data collection companies are way outside our budget.
Capacity: As you can see in the money section, we need to accomplish this task in-house. It’s just two part-time employees and whatever methodologies, software, and project structures they know how to use are what we’ll have to work with.
Rules and Regs: There are all kinds of important privacy considerations to make when dealing with the data of minors. We may also be accessing schoolboard data that can’t be disaggregated. Etc.
So now we can see that we’re not looking for some objective “best” answer to this question, but rather, like all science, we’re trying to get the best possible answer we can within these parameters. If we do a 10-year study, our answers may be more robust and reliable, but they won’t matter at all if the AGM was 9.5 years ago and our program got defunded.
Why does this matter for equity?
It’s important for equity to talk about these restrictions openly and attach them to your reporting. There’s nothing wrong with saying that a project had to be done in time to get refunded or evaluated, but if you don’t tell anyone, your stakeholders may wonder what the rush is and feel like a project is compromised for no reason. Conversely, if you have a long project timeline to address more complicated questions or get more reliable answers, some stakeholders may be confused by your lack of urgency. “What do you mean you’ll have the results in 10 years?! My son got suspended this week!”. Being open about the time, money, capacity, and regulatory restrictions that you’re working under explains to your stakeholders and your team members why you are going about your project in the way you are.
Now, if you really, really don’t want to publicize one of your restrictions, that’s a very valuable equity red flag. I was once working on a project with a multi-million dollar budget evaluating the effectiveness of micro-financing for some very, very poor entrepreneurs. I couldn’t bear the thought of telling these hardworking people making a few dollars a day that we spent many times over their entire town’s collective income studying them. I’m not saying it was absolutely wrong, but it definitely felt problematic to me.
For a more clear-cut example, let’s say that you’ve been asked to complete your project in the next four months, before an election, so that a local councillor can use the results as a feather in his cap. A 4-month restriction may significantly compromise the calibre of results we can produce, is that restriction compatible with our equity goals? Does it prioritize the people that we want it to? If you wouldn’t be happy explaining the why behind an element of your project, it usually points to an equity issue.
A conflict between your core motivation and your goals should not be viewed as a crisis, but rather an opportunity. Sometimes you can adjust your restrictions (get more time, get more money, develop new capacities, get rule exceptions or even change laws), but more often you’ll need to adjust your goals or even core motivation. Maybe this time we just do a manned space flight and leave the moon for the next budget round.
Your team, your funders, and your stakeholders will be confused when they can sense a compromise but they don’t know why you made it. Defining your restrictions beforehand sets them up as reasonable and realistic considerations that you used to make efficient and transparent decisions, rather than flaws in your final data product. By writing them down together, you can also make sure that your equity goals and promises are possible and that the root of your restrictions lie where you want them to (or at least where you have accepted them). Not everyone will be happy about every restriction you have submitted to, but trying to hide them or ignore them guarantees a project that is, at least, fraught with distrust and, at worst, a catastrophic failure.