Research is a feedback loop.
It's hard to underestimate the importance of research. It is part of every function in the company - whether it's done formally or informally.
What Is Easy to Get Wrong
Biasing The Respondents
It's very easy to do. In most cases you won't even know that it happened.
It's the way you word the questions, the choices you give for answers, the way you guide interviewees etc.
This especially tends to happen when, for example, designers try to research their own solution.
In interviews and observations it's especially hard to detect it by the person who is doing the research. If your goal is to learn what the user is trying to do, then you should never ask pointed questions like "Did you see that button?" Instead, ask why the person acted as they did.
Being Not Clear Which Decision Needs To Be Made
The job of the research is also to specify whether the question that is being asked (by whoever wanted the research to get done) is the right question to start with.
Choosing The Wrong Method
Research can often be done by sending out a quick email or Intercom message. The point is not to be as comprehensive as possible but as quick as possible and as comprehensive as needed. The innovation rate at the company depends directly on research - the quicker the feedback loop, the faster the innovation.
Looking At Numbers Instead of Talking To People
The bigger the company, the more specialized people get. The bigger the company, the more data there is. When you're scaling up, employees get especially tempted to only look at numbers. Great insights rarely come from numbers. For every hypothesis you should have a anecdote from a specific customer in mind.
Quantitative research answers (at best) the "what" and shows patterns but it doesn't answer the "why" - that's the job of qualitative research.
To fight this, you need to make talking to customers part of the process for everyone - support days, research days, etc. Even engineers should (if not talking to customers directly) at least be looking at actual customer interviews from time to time. This gives also a motivational boost - seeing the problems they are trying to solve first-hand.
Not Doing Open-Ended Answers First
It's tempting to just write down questions and possible answers and send out the survey. However, you might even not be aware of some of the possible options.
Always do a test run with an open-ended questions first to see what options are out there.
Mismatch Between The Reward And The Effort
Sometimes you need to motivate people to get them to talk to you. Rewards are one way of doing. But the rewards are heavily dependent on the target group.
Giving 20 USD vouchers to CEOs is not a meaningful reward. But giving 500 USD Amazon vouchers to someone who makes 600 USD a month will bias the results as well.
You need to have a strong filtering mechanism to make sure people aren't trying to deceive you about whether they belong to the target group or not.
Cheaping Out And Chickening Out
Research needs to be done as quickly as possible. The rate of innovation depends on it. If you need to spend 1,000 USD on gift cards, do it. If you need to add more friction to the signup flow (or even block signups) in order to catch people at the right moment, do it. Learning is more important than losing a bit of money.
Trying To Fill Awkward Pauses
Some of the best learning happens while people are forced to think about things. Don't try to fill the silence with your own thoughts. Let the person speak.
Mistaking Soft Commitments For Hard Ones
Just because someone says they will buy your hypothetical product doesn't mean they actually will. Ask people to confirm by pulling out their credit card.
Basics
Research starts with clarifying what decision needs to be made in the first place. The decision falls into four groups:
- Discover - what is the problem?
- Design - which option will be most effective for solving the problem?
- Develop - can people actually use our solution successfully?
- Deploy - why didn't our solution move the metrics?
Once you are clear on what needs to be decided, set clear limitations to:
- Evidence - what evidence do you need to make that decision?
- Data - what data do you need to collect (who, where, when, how)?
- Approach - based on the previous answers, how will you approach research?
Only then can you move on to selecting the appropriate method.
Methods
This is not a comprehensive list of methods obviously. These methods help you the most with strategy and keeping focus:
- Jobs To Be Done - great for understanding underlying motivations, blockers etc. for the use case you're trying to solve; think of it as win-lost interviews which should be done consistently
- Mental model mapping - great for putting together product roadmap and spotting holes in use case coverage
- NPS - Net Promoter Score is mostly about seeing the trend of what customers think of your product
See also various testing methods listed under product management. Researchers often shy away from these because they are not under their direct control.
Additional Reading
Product Discovery Activity Guide - Product Discovery Methods
The New York Times Product Discovery Activity Guide The Product Discovery Activity Guide is intended to inspire Product Managers, Product Owners, Product Designers, UX Researchers or anyone who is interested to innovate and collaborate together with teams in smarter, leaner ways. This collection consists of tools, techniques, methods, practices, and activities ...
pdmethods.com
UX for Lean Startups: Faster, Smarter User Experience Research and Design
UX for Lean Startups: Faster, Smarter User Experience Research and Design - Kindle edition by Klein, Laura. Download it once and read it on your Kindle device, PC, phones or tablets. Use features like bookmarks, note taking and highlighting while reading UX for Lean Startups: Faster, Smarter User Experience Research and Design.
www.amazon.com
The Research Mentality... and how to adopt it for product-led growth - Andreessen Horowitz
Building software has gotten insanely cheaper and faster. The infrastructure provided by the cloud or IaaS vendors has evened the technical ground for builders. A flourish of no-code tools have shortened the prototyping cycle down from weeks to days. As ...
a16z.com
