How to Combine Quantitative and Qualitative Data
Why Mobile Surveys?
Analytics tools have truly changed the way we see and interact with our audiences. Now more than ever, we’re loaded with quantitative insights about who our users are, what their behaviors are like, and how they interact with our apps.
But what’s the next step after collecting all of this information? Quantitative data gives you the whats and hows of your situation. But using surveys to get qualitative responses from your users will illuminate the whys, and give you clues about what to do next.
Surveying your users is both an art and a science. Segmentation is the key to creating custom-tailored surveys that get heaps of responses—and data that you can use to optimize your app. Tailored surveys get higher response rates than generic surveys because they’re more relevant to the groups you’re targeting.
In this blog, I’ll share the steps you can take to personalize your requests for customer feedback, and examples of how to leverage user data for better survey design.
Segment and Discover
You’re likely already familiar with demographic segmentation, the process of dividing users into groups based on factors like age, gender, location, or other similar buckets. But your users are more diverse and complex than that. They all come with unique needs and motivations.
First, know your segments. If you are using an in-app survey tool like Instabug, you can tag users with custom attributes to keep track of their segments and behaviors. Then, you can send targeted surveys to specific groups users.
Leverage the data you have to gather focused insights from your users. Your numbers will guide your investigation—take your quantitative behavioral and usage metrics and use this information to influence the questions you ask and the qualitative insights you seek.
The Art of Asking (Only) The Right Questions
Choose your questions wisely—have a reason for everything you ask. The results you get from your surveys should influence what you plan and do next. Define what you’re going to need in order to take action based on the data. Too many surveys ask loads of questions, but have no plan for what to do with the answers they get.
All businesses have different motivations for asking specific survey questions, but the following templates should give you a place to start thinking about the kind of insights you need to develop a better product. Thinking strategically will help you direct your focus and design more effective surveys.
Examples of Mobile User Survey Segmentation
Let’s look at specific types of behaviors you may have observed in your app. You may have gotten some of this information from your analytics, but not known where to go next with it. The following questions are a launchpad for you to start your own fact-finding mission. The answers you get from these questions (or others you’ll be inspired to ask) will help you learn how to optimize your app for your users and encourage the behaviors you want.
1. Purchasing Behavior
Purchasing behavior is, of course, a powerful starting point for ecommerce apps. But do you know what to do with that shopping cart data? Let’s look at how you can identify conversion-influencing factors by asking targeted questions.
What you learn about your user habits will guide your iteration and streamline your shopping and checkout experience. What made certain people willing to punch that “checkout” button? What do you have to adjust to get other segments to behave the same way? Find out what makes them feel comfortable with the experience and optimize, optimize, optimize.
| Purchasing Behavior (possible segments): |
| Never bought |
| Abandoned a shopping cart |
| Bought once |
| Repeat buyer |
Potential questions for these segments:
- What other products or categories would you like to see in [App Name]?
- What would make our checkout process more convenient?
- What convinced you to buy from us?
- Was it easy to find what you were looking for?
- What do you like most about shopping with [App Name]?
2. Usage Frequency
We all expect that our users will have variation in how much time they spend in your app and how often they log in. Usage frequency isn’t always a particularly useful metric on its own—it’s important to remember the context and what healthy usage would look like for your users’ specific situations. You’d hardly expect daily usage from a travel deals app, for instance. Different use cases will produce different habits. By determining which factors influence usage scenarios, you can devise strategies to increase functionality for your users and increase your app’s relevance to their lives.
| Usage Frequency (possible segments): |
| Power users |
| Regular users |
| Infrequent users |
| Dormant users |
Potential questions for these segments:
- Which feature of [App Name] is most valuable to you?
- On a typical day, how many apps do you use?
- How does [App Name] help you achieve your goals?
- What feature would make [App Name] more useful to you?
3. Engagement with Features
Learning about how your audience interacts with your features will give you clues about how your app fits into the lives of your users. What makes your app most relevant to them? Answers will vary by segment. When you dig into the details, you’ll discover the different ways in which people relate to your product.
You’ll also learn if you’ve been downplaying the usefulness or applicability of a certain feature, or even if your marketing messaging and functionality are properly aligned. There’s another important factor to consider here and that’s discoverability. How easy is it for your users to find what they’re looking for? Have they been notified of new features? Sometimes a low adoption rate means nothing other than that your users don’t know how to use it or where to find it.
| Engagement with Features (possible segments): |
| Only uses one feature |
| Uses most popular features |
| Uses all features |
| Hasn’t tried new feature |
| Unusual use case |
Potential questions for these segments:
- How easy or difficult was it for you to learn how to use [Feature X]?
- Can you describe a situation in which [Feature Y] was useful to you?
- What goals does [Feature Z] help you achieve?
- How easy was it for you to find [Feature W] in the menu?
- How would you rate [Feature Q]?
4. Loyalty
What has made your oldest users stick around for so long? Can you identify what makes your users perceive you as being more valuable than your competitors? Discover loyalty-earning factors among your segments and get clues about what will encourage new users to be in it for the long haul. You’re likely to find similarities between your longtime users and NPS promoters, because these groups typically have some overlap.
| Loyalty (possible segments): |
| New user |
| Longtime user |
| Ambassadors/Promoters |
| Likely to churn |
| Churned users |
Potential questions for these segments:
- What can we do better?
- What do you think of the design of [App Name]?
- Is there anything about this app that you find difficult to understand?
- Which features of [App Name] are most valuable to you?
- Is it easy for you to find support when you have a question?
- What do you use [App Name/Feature] for most?
- How does [App Name] add value to your life?
5. User Satisfaction (NPS)
This question is familiar to most of us: “how likely are you to recommend this product to a friend or colleague?” Asking this question in the first place is more than just a way to get a quick metric showing you how users feel about your brand; it’s an invitation to share their thoughts and feel valued as users.
Following up with your segments after surveying will help you dig into the “whys” behind the situation and get you deeper into the minds of the users of your product. A question with a similar theme that will also get interesting answers is “how would you feel if you could no longer use this product?” Discovering what makes people say they’d be incredibly disappointed to lose you will help you identify hidden gems and possible marketing angles for your product. Asking them what the reasoning behind their score is will help you glean further insights. Their answers to questions about their reasoning will show you what most heavily influences their feelings about your app.
| User Satisfaction (typical NPS segments): |
| Promoters |
| Passives |
| Detractors |
Potential questions for these segments:
- What could we improve before you’d be willing to recommend this app to a friend?
- What is the most important reason for your score?
- If a friend asked you about this product, how would you describe it?
- Where do you see potential for improvement?
Collecting More Mobile User Insights With Better Surveys
The above are common types of behavioral segments that can be found in most apps, but each app is unique! You’re going to find many, many groups within your audience that give you valuable insights—but which groups are most useful for you will depend on your users and goals.
You can use behavioral segments within your Taplytics dashboard for A/B testing and further insights. Remember that behavior will evolve along each user’s journey, and that users are likely to change segments as well over time. Your app will evolve and your users will change along with it.
Always be strategic with the phrasing you choose for your questions. It’s important to be clear and concise; long-winded and confusing questions will dramatically cut your response rates. It’s also good practice to choose survey tools that don’t disrupt the user experience. If you send your users out of your app to do the survey, you raise the risk of distracting them and decrease the likelihood that they’ll log back in. Keep it quick and simple—the shorter and easier it is to fill out, the higher your completion rates will be.
Don’t forget to thank your users for sharing their thoughts and their time with you! As often as possible, reply to your survey respondents. By closing the feedback loop, you’ll show them that you genuinely value the time they spend using your app and helping you to improve it. Following up also gives you a chance to explore intriguing answers more deeply.
For help interpreting your responses, check out Instabug’s 6-part guide to mastering qualitative feedback.