Experiment Results

The Glorious Data

You didn't come all this way to just go home now. No, you want to see the fruits of your labor. You want to see clicks going up and sales hitting new heights. Well, all of that magic happens on our "Results" pages.

Once you have an experiment running, it is displayed under your "Active Experiments." To see the results, just click on the experiment name.

This article goes over how to navigate the "Results" page and shows you some of the common things you might want to do to analyze your data.

Understanding the Data

When you hop into the "Results" page for the first time, there probably won't be much there. You'll likely see something that looks like this:

Don't worry about the lack of numbers. You've only had your experiment running for a few minutes. After a while, your "Results" page will start to look more like this:

So, you don't have a million sessions yet? Not to worry. You might as well start to understand what you can do with the "Results" page now. The "Results" page is broken down into five parts:

  1. Header

    The header for your experiment's "Results" page is simple and serves a few important purposes.

    It tells you which experiment you're viewing and allows you to easily change its name. You can tell, at a glance, if the experiment is active or paused. You can set the page to either include or exclude your development devices from the page. This is particularly useful when you are instrumenting code goals and you want to test them before pushing your app to the store. Finally, you can set a date range for the page. This allows you to view the entire range or a subset if you want more granularity in your data.

    Experiment Overview

    This small box in the top left corner of the page lets you know how many unique users have viewed this particular experiment, how many total sessions were exposed to the experiment, and how long the experiment has been running.

  2. Experiment Results Summary

    If you need an at-a-glance view of how each variation is converting for each of your goals, look no further. This is the straight data without any confidence intervals or anything else. This is great for a simple comparison view.

  3. Refine Results

    Not all of your users are the same; so it's not always best to view the results of your experiments in one big group. On the "Results" page, you can refine the results to show all of your data for particular segments of your users. For example, you can segment by your app version, the devices your users are using, and the OS version they are running.

    This can be very important for understanding how different people react to your experiments and can show you how to target your experiments in the future.

  4. Detailed Results by Goal

    For each goal that you set up, Taplytics shows you a detailed results panel. In this panel, we tell you how each variation performed vs. the baseline. Here, you can see the total conversions, the percentage conversions (and the range that this should fall within in the future), the percentage change vs. the baseline, and the overall chance of beating the baseline. We also tell you if you have reached statistical significance or how much longer the experiment needs to run to achieve it.

    Beyond the hard numbers, we also give you a graphical representation of the data in the form of a line chart. The chart defaults to a view of the entire timeline of the experiment's life, but you can focus in on any area with the scroll bar at the bottom.

Customizing the Results Page

There are a number of ways that Taplytics lets you customize the "Results" page. The first and simplest way is to name the visual goals that you have. 

When you first come to the "Results" page, you will be greeted by goals with placeholder text that say "Button Click on View".

In case you don't ever decide to change this placeholder text, we show you the button and the view that we're referring to when you hover over the respective names.

When you decide that you want to customize the name, click the little pencil icon next to the placeholder names. This opens a textbox where you can rename your your goals.

You can further customize your results by setting which part of your app is at the beginning of an experiment. This is particularly important for understanding deep parts of your app, like shopping carts and purchase funnels. To customize this part of the results, click the "Customize" menu item in the left-hand navigation menu.

From this page, you can set where your experiment starts by choosing a specific view from the dropdown menu. The views are named as they are in code. If you are non-technical, this may be an area where you want to consult someone on your technical team to choose the appropriate view. Once you've picked the view you want, the "Results" page will only count a user's view or session if they get to the view that you set. Awesome, eh?

Now you're all set to get rocking on some wicked A/B tests. If you need more detailed technical information, feel free to dive deeper into the technical documentation. If you would like help creating experiments that will really drive more value from your users, feel free to shoot us an email. we're happy to help!