How do you know you're building the right thing?
When Jennifer Rufer was 22 years old, she underwent chemotherapy and a hysterectomy that she did not need. Her ordeal began when she went to her doctor because of irregular bleeding. She was producing high levels of human chorionic gonadotropin, or HCG. A pregnant woman normally produces high HCG levels but because Jennifer was not pregnant, the elevated levels of HCG can be a sign of a rare form of cancer that can quickly kill her. After weeks of chemotherapy and a hysterectomy, doctors still found no cancer and yet her HCG levels were elevated. Then came a stunning relevation: "They ended up finding out that I have never had cancer." The test was faulty from the beginning and she never had the disease. "I had been treated for no reason at all."
Years ago an organizational change meant that I was moving my team to another department. In large companies, that could mean a huge change for a team. Understandably, my team was concerned. So I set an appointment with my future boss who headed product for many years at the company and had a heart-to-heart talk.
My first question was, "How do you know you're building the right thing?" After thinking about it for a while, his answer surprised me. "I don't know," he said. Many teams struggle with this and forget to pause to ask this very important question.
In Jennifer Rufer's case above, the court awarded her $16 million after they found the test faulty and the pharmaceutical company who created the test negligent for failing to adequately warn doctors about false positives.
Product teams need to create "product tests" that can help build the right thing and avoid false positives. You can come up with your own product tests but here are some I have found to be generally helpful for products:
- Are you building for the right audience? Do you know who you're building for and are they the right audience? You can start gathering demographic information to understand what people do, where they go, how they found you, how they're telling their friends about you, etc. At Pluralsight, we have created personas to help the entire company understand who our audiences are. Our product teams have interviewed hundreds (maybe over a thousand) of our customers one on one so we can get to know them better. Our research team is building cohort analysis on our audiences so we know how to target them better.
- Are you honest with your process? I have seen many teams struggle building products because they do not have a process. A process helps teams stay honest and true to their audience. A process also allows teams to be consistent and rigorous as they build the product. At Pluralsight, we use a process called Directed Discovery that was created by Nate Walkingshaw, and we use it in all of our customer interviews, immersion visits, software development, and product releases.
- Are you iterating from the feedback? How and when are you gathering customer feedback? How does that feed back (pun!) into your product process? You can argue how much of the customer feedback you will want to incorporate into the product but the important skill here is to find the patterns and see what changes you need to make. The customer can help you see; you make the decision.
- Are you measuring feedback? Feedback can be measured quantitatively as you release a page, a flow, or a feature. Have customers rate these bite-size experiences from one to five stars then ask them for comments. From there you can analyze what people are saying about your product. Amazon's biggest asset is their product rating system. Measured feedback should be your biggest asset as a product team.
Product teams, big and small, need some structure to help organize the demands of your product, your customers, and your company. Think of a process for each of the questions above. My hope is that these questions will ultimately help you answer, "How do we know we're building the right thing?"
The story of Jennifer Rufer can be found here from ABC News.
Feb 16, 2016