Facebook Pixel

When it comes to data projects, one of the most powerful tools we use is what we call a Motivation Touchstone. It’s a document that you and your team can return to over and over for guidance when figuring out how to make effective and equitable decisions at every step in the data process.

A Motivation Touchstone is made of 4 parts:

Whose perspective (or perspectives) you want to center when making decisions.
A specific core motivation.
The restrictions you need to adhere to.
The rewards you hope to gain.

In this piece, we’re going to talk about: Rewards.

In part I on crafting a Motivation Touchstone, we talked about how getting clear on the Restrictions your project is facing will help you avoid pitfalls and design a more efficient and fair data project.

In this piece, we are discussing the other side of the same coin: Rewards.

Rewards in a data process are anything you (the people involved in the data project) are hoping to gain from the process. By identifying your hoped-for rewards, you are describing the criteria that you will use to ensure a “successful” project.

Let’s look at an example:

You are a small non-profit organization with a general mandate to improve the lives of women experiencing poverty in the city of Chicago. You want to do an impact evaluation of your latest program, running financial literacy learning groups. In order to establish our rewards, we only need to ask one question: why do we want to do this impact evaluation? If this impact evaluation goes “well”, what will have happened?

“Why do you want to do this data project” is a question that will have different answers to different people and groups involved in the project. Let’s break these potential rewards out into three levels:

Organizational
Team
Personal

At the organizational level, we might see potential rewards like:

  • Improve our future program effectiveness
  • Secure funding for additional programs
  • Meet evaluation requirements from our funders
  • Decide whether or not the investment in the program was worthwhile
  • Impress our board with dramatic and positive results
  • Report the results to spread awareness of the value of financial literacy
  • Report the results to improve our organization’s prestige in our sector

At the project team level we might see things like:

  • Identify the most impactful components of the program
  • Demonstrate our competency to the program director
  • Expand our program’s budget for next year
  • Cement a positive relationship with our current project participants

At the individual level, we can encounter a very wide range of hoped-for rewards:

  • All staff: Earn salary
  • Marcus: Learn new methodology
  • Raj: Use the results in our PHD thesis
  • Jaime: Work with a specific colleague
  • Sara: Get promoted
  • Tabitha: Create a data library for the office
  • Chen: Keep a promise to a project participant

Wow, that’s a lot of goals or potential rewards to be satisfied by one data project, yet this list is much shorter than the goals we uncover in even the simplest data science endeavours. You can already tell that some of these rewards are more palatable than others. Some seem more altruistic or ‘pure’. Some seem to be at odds with others. How can creating this list possibly help my data project, and what does listing these rewards have to do with equity?

Identifying all rewards is crucial to getting any of them.

First of all, remember that listing all the rewards people want to get out of a project didn’t actually create them. They were always there, lurking under the surface, and it’s going to be much more effective to at least address them internally, if not publicly.

In our previous piece talking about Restrictions, we used the metaphor of designing a rocketship. We needed to know what our restrictions were in order to create a spaceship that could be as good as possible given the situation. In order to have a successful mission to the moon, we also need to have a clear and shared definition of what defines “successful”.

If we asked three departments in our space agency what would make the moon mission successful, we might get three different answers.

Engineering: “The key thing is to demonstrate that this new rocket design is not only safer but also incredibly fuel-efficient. This will mean we can do more missions in the future for less cost”.

Geology: “The most important thing is that we are able to bring back a lot of moon rock samples. If we can prove that the minerals on the moon are valuable, we can get more missions funded”.

Public Relations: “If we don’t get pictures and video footage of our astronauts on the moon, we might as well not go. In order to do more missions, we need to gain more popularity and awareness.”

Each of these priorities or Rewards would lead to a very different spaceship design. The engineers want one optimized for fuel efficiency. The geology department wants one that can handle a very heavy payload upon reentry. PR wants to bring a ton of camera equipment and a live-streaming satellite connection wouldn’t hurt either.

Now, just like in our more *ahem* down-to-earth example with the non-profit organization, it will likely be possible to aim for many, if not all of these rewards, but only if everyone knows about them. In data science, we often find that stated “mandates” or “project aims” are heavily outweighed by a lot of secret or unstated goals. If the stated objective of a project is to “evaluate program impact” but the actual (or more urgent) goal is to get re-funded by your donors, you are setting yourself up for an inequitable and ineffective process.

Must-haves vs. Nice-to-haves

 

In our example, the project team has worked to highlight any reward deemed “must-have”. Anything unhighlighted is going to either be a “nice to have” or be abandoned as a goal for this project.

Organization:

  • Improve our future program effectiveness
  • Secure funding for additional programshere the team is being honest about one of the essential, and most common rewards the project is being created to achieve.
  • Meet evaluation requirements from our fundersthis is something that is mandated in the funding agreement.
  • Decide whether or not the investment in the program was worthwhile
  • Impress our board with dramatic and positive results
  • Report the results to spread awareness of the value of financial literacy
  • Report the results to improve our organization’s reputation in our sector

Team:

  • Identify the most impactful components of the program
  • Demonstrate our competency to the program director
  • Expand our program’s budget for next year
  • Cement a positive relationship with our current project participants

Individual:

  • All: Earn salaryif you don’t have this a “must-have”, are you saying you’d do this project whether you got paid to or not?
  • Marcus: Learn new methodology
  • Raj: Use the results in our PhD thesis sometimes individual rewards are must-haves in the project. It’s important to be on the same page about which ones will be treated as such.
  • Jaime: Work with a specific colleague
  • Sara: Get promoted
  • Tabitha: Create a data library for the office
  • Chen: Keep a promise to a project participant
Making Choices

 

Sometimes the Motivation step of the Data Equity Framework can appear a little soft or frivolous compared to the nitty-gritty choices in areas like data collection, research questions, methodology selection, analysis models, or data visualization. However, nine times out of ten, when we are hired to consult on a data project that has truly gone off the rails, a lack of a coherent Motivation Touchstone is the problem.

Once the must-have rewards are clear, they inform every decision at key choice points. If Marcus wants to learn a new methodology, that will impact project design. If improving relationships is more important than getting re-funded, the research questions may be very different. If funding is a goal, we may do more descriptive and comparative analysis. If identifying the mechanism of impact matters more to us, we may pursue a causal analysis. We’re going to report our information very differently depending on whether we’d rather make it into a prestigious journal or be accepted as a part of a local community.

Knowing what your goals are is as important in designing a survey as it is in designing a rocketship.

Why Rewards Matter for Equity

 

For your project team, it’s unfair to have hidden and competing agendas that they have to navigate. Team members won’t understand why certain decisions are being made unless everyone shares a clear picture of the true objectives of your work. Team members have a right to know whether their individual and team goals are or are not part of the strategy.

For your audience, whether that’s your board, boss, customers, funders, other researchers, or the general public, it is essential to be as transparent as you can force yourselves to be about why you are doing your data science the way you are doing it. Remind yourselves that it’s ok to say that part of the reason is to pay yourselves or get more funding to continue your work, or to build your reputation and awareness. Everyone knows anyway and you’ll gain trust points for being upfront about it.

Most importantly, Rewards reflect the project participants you are trying to prioritize. Evaluating and ranking your rewards provides an essential (though sometimes uncomfortable) opportunity to see if you are really centring the perspectives that you want to. Are all of your ‘must-have’ rewards defined and driven by the stakeholders that you want to center? If you would be uncomfortable divulging the ‘real’ reasons driving your project, that’s a big equity red flag.

At We All Count, we find it’s usually the case that internally driven objectives can be transparently shared and properly balanced with your other equity priorities in a way that makes everyone happy and provides a rich and satisfying “mission success” to a variety of stakeholders. You just need to know whose rewards you are going to embed into the heart of your project.