Just Ship It! Launch early to gain valuable feedback
“Ship it”, your manager says. “We’ve spent enough time tweaking the product, let’s just put it out there and see what happens”.
That thought can be scary, there’s no doubt about it. Releasing a product or feature when you have a full page of edits you could make can be daunting. But what do you really have to lose? Wouldn’t you rather find out what people think to see where your time and effort would be best spent?
By launching as early as possible, you collect user feedback, which, positive or negative, is really valuable. It helps you understand real users’ joys and pains, and inform your maintenance plans to improve the experience for all future users.
Learn how Customer-led Product teams use feedback to build amazing products in our free ebook.
Experts Agree
We spoke with UX expert Sarah Doody, who is an advocate of the ‘launch first, ask questions later’ mindset. Sarah once lived every UX designer and product manager’s nightmare: she launched a product that she figured was ‘good enough’ . She knew she was wasting time and money by waiting, so launched it to see what would happen. She found broken links, and email and payments errors that lead to a week of damage control and hands on support.
Was it a disaster? At the time, it seemed like it. But the silver lining was that she learned so much about the importance of launching early and gathering user feedback, which improved product and overall experience for future users. And, it still generated $12K revenue in a week, not bad for a $97 product.
Sarah shares advice on why you shouldn’t hold off on your releases, and how you should approach the collection of user feedback when you launch:
- Select the right test group to launch to
- Ask for feedback at the right time
- Don’t waste time on small details
- Define and measure success
Launching early without harming your brand
Reid Hoffman, founder of LinkedIn, once said that:
“If you’re not embarrassed by your first release, then you’ve launched too late”
A common argument against launching a bare bones product is that you don’t want to harm your brand’s image with that first embarrassing version. You’re told all the time that a frustrating experience can cause a user to turn away and seek alternatives.
While this is true, it doesn’t mean that you have to wait until your product is perfect to release it. You just need to make sure that you select the right set of users who you can trust to test your product and provide useful feedback.
Slowly increasing the number of users who have access can help you test an early product and gather feedback. The key is selecting the right group of early testers.
Who are these people, and where do I find them?
If you pick people who you have built some form of trust with – loyal customers, long-time subscribers, or even industry friendlies- then they will be more likely to look past the glitches. This is especially true if these people have the problem that your product will help them solve.
Curating these small groups of people who will find value from the release is an important first step in creating a safe environment for you to test in.
If you’re not sure where to find people for your user research interviews, watch this helpful video from Sarah.
How and when to collect feedback
Aside from analytics that monitor activity and interaction with the product, there are many great ways to gather feedback from your tester group. No matter which channel you decide to use, the key is to:
- Collect micro-feedback
- Make the ask as timely as possible
Microfeedback is really simple and specific. If you send users a ten question survey at the end of their experience, chances are many of them won’t feel inclined to answer with enough detail to provide real value. Instead, you must objectively consider what is the minimum amount of feedback that you can collect from them at key points in the experience.
“Obviously, microfeedback is meant to be small. So, keep the questions you ask very narrow and specific so the answers can be simple and short. Stick with questions that can be answer with a “yes/no” or a 1 to 5 numeric rating. If you really want to get cute, you could use little faces or emoticons. That said, always give the user the option to provide more details if they want. You’ll be surprised at how many people take the time to give you more information.”
– Sarah Doody
The part that is hardest to perfect is the timing. If you ask for feedback too long after the experience has happened, it will seem irrelevant or so distant in the past that details may be lost. Sarah recalls an experience with an Airline who emailed her four days after her flight to ask if she enjoyed it. Unless something dramatic or out of the ordinary happened, she wouldn’t be likely to write a response, especially given that this would be via email.
On the other hand, Instacart creates a great feedback experience. They send a text ten minutes after the delivery has arrived, asking for a simple 1-5 rating on the service. This is great because the user is asked when the experience is still fresh in their mind, and the 1-5 scale over text keeps it really simple.
You can ask for feedback during the process as well, as long as it isn’t disruptive. In-app feedback tools like Instabug send out low-friction surveys that don’t interrupt the user experience.
Journal, a search tool that makes it easy to find information across work apps, launched its product through a beta program recently. They have a few hundred users in the program, and have asked members that if there are glitches or positive feedback, that users share using the chat bot.
By selecting users who had a great enough need for the product to care about its success, Journal is able to collect great feedback in the moment when it’s fresh on a user’s mind.
“Everyone building a product has a unique set of constraints to accept and work around. In our case, infrastructure and cost constraints were the most influential in deciding to keep our beta closed.
We’re also building a product in a space where the bar is high to deliver a buttoned-up experience from the outset. We think it makes sense to take a little extra time in stealth so that we can go to market with a product that people will both find useful and really enjoy.
By limiting the number of people who get access to the product, we’re also increasing the time we get to spend with each user on a 1:1 basis. This will need to change at some point in the future, but has enabled us to gather invaluable qualitative feedback up to this point.”
-Sam DeBrule, Co-founder of Journal
Something to be mindful of with chatbots: Sometimes, it can be hard to collect contextual feedback that you can directly pass to your product team. For example, If it would make sense for your user to attach a screenshot to show you what they’re experiencing, make sure they would be able to! If there is friction in the feedback loop, the user won’t be as inclined to help.
Collect candid user feedback with a Facebook group
Sarah also suggests launching a Facebook group, which is a great place to collect candid feedback from customers as they voice their problems and help each other solve them (bonus! This can help take a load off for your CS team). It’s an amazing passive research method, where you can be on the sidelines observing what people love, don’t love, their workarounds or hacks. These groups can also be useful at the beginning of the process to identify potential first users when you have a new product that needs beta users.
If you choose to create a Facebook group, be aware that users will complain and other people will see it. If you jump in and proactively answer that problem, you can show people that you’re on it, and build trust. This is very important with that early version of your product, especially if this is the first time these people are interacting with your brand.
Resist the temptation of perfection
Always remember the Pareto Principle: 80 percent of your outcomes come from 20 percent of your inputs.
Critically examine your to do list and determine what is going to move the needle the most. Chances are, it’s not perfecting small details.
“I completely emphasize to product managers, founders, and anyone who is launching to resist the temptation to spend time perfecting something that is ready. It’s fine to spend a day optimizing and making improvements AFTER you’ve had enough people go through to have prescriptive data. But until then, you have to let it go into the world and let it be not perfect. If it’s working, making money and allowing your time for other priorities in the business, then there is no reason not to.“
The trend is everywhere, no matter what you want to call it; pilot programs, beta testing, exclusive releases. They all aim to help companies control their releases and gather that feedback.
Spotted: mobile app trend!
This way of approaching releases has increased the popularity of a mobile app release technique called feature flagging. Feature flags are a set of switches that turn features on and off, allowing apps to release new features to small sets of users at a time to collect feedback. If worst comes to worst and the release is a disaster, apps have the power to simply turn the feature off. Once they have collected feedback and made appropriate adjustments, they can roll out to the rest of their users with confidence. It’s the best of both worlds!
Take the words right out of their mouth
Once you’ve conducted enough testing and talked to enough users, you will understand the language they use, how they talk about their problems and your competitors. This is really useful for when you officially release to the rest of your user base; you can use their language to develop marketing materials that you know will really resonate with your audience.
To help you understand your users, Sarah put together 30 questions to ask in a user research interview in this free guide. Sarah also has a training program, User Research Mastery to take you step-by-step through the user research process. You can get 20% off with the code YOUSAVE20.
Defining post-launch success
Lastly, it’s important to have some type of measurement for your release. When you do that initial launch, you should have an idea of what data you would like to collect and how you are going to define success.
Get in the habit of updating the metrics daily or weekly. Don’t just let them live in a spreadsheet or dashboard that you never look at after the first two weeks! It’s so easy to do that, but then you lose out on the opportunity to continually improve the product, even after your beta is over. Knowing what’s working and what’s not won’t only help you prioritize your time, but also help guide you as you continually iterate and make improvements to your product.
So what are you waiting for? Ship it!
All of the time spent holding off on releases is time that could have been spent making money and collecting feedback that will give you confidence to fully launch.
Remember:
- Select a small group of users that are the right fit for your product and brand
- Collect microfeedback and make sure the ask is timely and relevant
- Perfect and correct after you have enough data
- Continue to measure results and success
Get more tools for aligning user needs and business goals in our free guide.
Big thank you to Sarah for sharing her ideas and expertise!
Sarah Doody is a UX Design Consultant, Writer, Speaker and Founder of The UX Notebook. She also teaches UX through in person workshops and online courses. Find her on Twitter @sarahdoody.