A well-crafted Motivation Touchstone contains your success criteria and aimed-for rewards, as well as a summary of the major restrictions and parameters that your project faces. These elements give us its outline, the scaffolding required to make decisions throughout the project, and in terms of equity, they provide us with an opportunity to make decisions in those areas that prioritize the people and perspectives that we want to center.
The third part of a Motivation Touchstone is a little different. It holds your key definitions, their sources, and their statuses.
When we’re using data to make decisions or answer questions, we need to translate terms, ideas, and concepts into something measurable. In order to do that, everyone interacting with the project – the people working on it, the people giving data to it, the people funding it, and anyone accessing the output/report – need to have a clear understanding of what the terms we use mean and how we went about turning them into numbers and back out into meaning.
Let’s say that our general mandate or mission statement for this data project is:
“To measure the current financial health experienced by the young women of Chicago”.
This project is undertaken by an organization that runs a variety of programming for young women in the greater Chicago area, including training groups focused on improving financial skills and capabilities. The project has a major funding partner; W.C. Bank, as part of their corporate social responsibility campaign “making sure everyone has a chance to be happy, healthy and wealthy”.
In order to turn this mission into a practical data project, we need to define exactly what we mean by each key term. Additionally, if we care about equity, we need to pay attention to who gets a voice in that defining process.
First, let’s underline some key terms that need to be defined:
We could certainly underline even more terms like “measure” or “experienced” and if your project has the timeline and scope to engage on the more philosophical elements in your mission, we highly encourage it. However, this is taken from a real example so we wanted to show you what these folks did.
Some of these terms need definitions simply to avoid misunderstandings, assumptions and mistakes like “current”. Others have essential and highly variable definitions that will completely change the meaning of the entire project like “financial health”. Additionally, we have identity and individual modifiers like “young”, “women” and “Chicago”. Different definitions of these terms will include or exclude various people’s data, and, by the way, reinforce our definition of those terms when we report them.
The next step is to decide if each term here currently has an “accepted” or “exploring” status. This will change during various phases of the project, and typically by the time the data is being analysed, all of the operationalized definitions will have been either consciously or unconsciously accepted in order to be used. By the way, making sure your analyst and model builders use the shared definitions is pretty crucial to making sure your results mean what you think they mean (and it’s a problem that happens all the time!).
A definition that is “accepted” simply means you have locked in what that term is going to mean for this project. It’s not that everyone in the world needs to share your definition of that term, but it’s absolutely critical from both a scientific method and an equity point of view that all stakeholders can access definitions you adopt. It’s also important to try to identify the source of that definition. It’s not expected that you prove from first principles what a “woman” is, or to posit a universal cutoff line for being “young” (my definition of young is constantly being updated and currently includes all of my adult children, and probably always will!). Most science is based on previous research and definitions and stats is no different.
There are a lot of practical decisions to be made. It’s ok to say that your definition of “Chicago” was chosen to match zip code boundaries so that you can incorporate other existing datasets instead of an indigenous definition of Chicago, or a historical definition of Chicago, or just inclusive of anyone who supports the Bears.
When a definition is under “exploring”, it means that you have yet to settle on the one or more ways that this project will interact with that concept. At the start of your project, every term begins under “exploring” if only for a brief moment.
Sometimes, it’s very easy to accept a definition. In the case of “current”, the organization simply picked a cutoff and didn’t feel that they needed any outside perspectives or input in that area. A little more tricky was deciding how to define “women” in this project. For that, they turned to some expert help in making this kind of decision and opted to base it on self-identification. They could have gotten participatory input from their project working groups instead, or they could have based their definition on some other expert, institution or norm. The key is that how this definition was arrived at is made transparent so that people can engage with it in relation to how they think about it.
Taking a moment to intentionally define these terms gave the organization an opportunity to sidestep their own assumptions about who is or is not a woman and include the input of whomever else they wanted to involve in that process. There’s no perfect expert, focus group or source to turn to for this, but that doesn’t mean it’s not incredibly important to do. It also allowed the project to move forward with a clear understanding of what they mean when they say “women”.
The researchers hadn’t really considered the implications of how to define “young”, and a lot of pros and cons to various measurements and cutoffs were tossed around in the meeting, from very grounded age ranges to systems based around experience, self-interpretation of age, or even removing the term entirely. The team decided that it would be both interesting and beneficial to explore that term further before locking it in. It would also give them time to consult with participants, colleagues and experts to identify potential equity issues around that term that they hadn’t thought of.
When the research team set out to define “financial health”, they realized that they actually had codified a very narrow and quantifiable definition in their granting agreement with W.C. Bank, their major funder. W.C. Bank felt that a credit score was not only a reliable indicator of financial health, but that learning how to improve credit scores would be one of the best ways for young women to achieve their life goals. The research team was concerned that this definition would not be shared by the women in their training program. Having two months of backup rent payments saved, or having disposable income beyond sustenance basics, or having secure income streams, or owning certain items/property, etc. might all be indicators of “financial health” that resonate more with their participants and have less to do with the banking industry. What to do?
Well happily, in this case, they raised their concern to the funders who were not only willing, but enthusiastic to include multiple metrics of financial health, some of which would be informed by their perspective and other directly from the participants. We’ll talk about how they went about embedding a variety of other views into their “exploring” definitions and how they then crafted research questions in another article, but for now, here are a few key takeaways: