Make Sure First Screen Experiments Run Properly in App

I get a lot of questions either about how Taplytics handles first-screen experiments or how, as a user of our platform you should think about them.

It’s a great question because there are a lot of inherent challenges around running an experiment on the very first load of your app. But the challenges need to be dealt with because some of the most important experiments that you can run are the first thing that a user sees when they open the app. More specifically, no matter what category your app is in, experiments on your first-time user experience will probably be one of, if not the most important type of experiment you will run.

First screen and first load experiments are extremely important

It’s an important type of experiment because first impressions are everything, especially when it comes to mobile apps. For better or worse, we live in a day and age where the switching costs to using different services is almost zero. The result is that you have seconds to give your users what they are expecting, or else they will close and permanently delete your app.

Because of this willingness to switch and users’ fickle nature, you had better make sure that the A/B testing platform that you choose is able to handle all of the difficult aspects of running an A/B test on the very first load of your app. If you don’t, you’re not just throwing away money on a bad platform, you’re going to be losing users that you fought hard to attract. Users that you probably spent significant ad dollars to acquire

What are the challenges of running a first screen experiment?

1. Making sure all properties and settings load before the screen appears
2. Ensuring users never see a switch from one variation to another after load
3. Making sure to place users in their appropriate variations
4. Ensuring that the whole process fails gracefully when a user opens the app in sub-optimal conditions (i.e., bad internet connection)

We have spent a lot of time working on this problem because the most important thing for us is ensuring that your users get the amazing experience that they’re expecting. If an A/B testing platform gets in the way of that, it is hurting you more than it’s helping.

So how do we handle these challenges?

First off, we make sure that we are using the best possible server technology to ensure reliable delivery of experiment data. More importantly, though, we take ourselves out of the equation. We host all of our experiment data on Amazon and Google CDNs, which means that your experiment data will always be accessible. The other important piece beyond ensuring that data is deliverable is how to deal with it instantly upon first app load. This challenge is where the Taplytics SDK has a few tricks up its sleeve.

Specifically, we have created a function in our SDK called delayLoad

The sole purpose of the delayLoad function is to ensure that your users get what they are expecting when they load up your app, and it also makes sure your users never see a switch. This function takes advantage of the launch image already used by your app. It holds the image on-screen until our SDK can load all of the necessary experiment data, and we’re ready to give your users an amazing experience.

By default, the delayLoad function will hold the launch image for a maximum of two seconds. At which point either the experiment data has loaded, and your user gets their appropriate variation or the data couldn’t load and we move your user into the baseline variation.

What if I want to set a different maximum load time?

There are a lot of reasons why you might want a different time setting for the delayLoad function. You may be running an experiment that loads hi-res images on the first screen, or your users may be sensitive to any amount of delay in app launch time. For either of those cases, and any other that you come across you can customize the time setting to whatever makes sense for you. To learn more about how to customize this setting you have few options. You can either look at our SDKs’ header files, check out our documentation or the instructions are right on your app’s settings page in the Taplytics dashboard.

If you have any questions about this or any other part of the Taplytics platform, please don’t hesitate to submit a question in the comments section below or send me an email directly at cobi@taplytics.com.