Google Play Experiments vs. A/B Testing in SplitMetrics

The question I get very often from fellow mobile marketers is what’s the difference between app store pages tests in Google Play Experiments and SplitMetrics experiments? Let’s dwell on distinctions between these mobile AB testing platforms.

It’s clear that Apple App Store doesn’t allow A/B testing of app pages, so marketers have to bypass the App Store with custom coded landings or software like SplitMetrics. Google Play in its turn provides experiments within the store itself and these tests are free, so why go elsewhere?

Short answer: indeed, Google Play allows mobile publishers to run experiments on their app pages in the store, but these tests have significant limitations.

Comparing Google Play Experiments and A/B Testing in Splitmetrics

Above all, Google Play offers no options beyond the very basic conversion experiments. You can’t test and optimize conversion for search and category or run an experiment to find out how you measure up to your competitors.

SplitMetrics options are way more extended. For example, if you want to test icon and title at Google Play search page to compare your app’s performance with your competitors’ CTR in the store, you can just launch a specialized experiment from SplitMetrics dashboard. Our A/B testing platform provides more than 12 types of other experiments.

Read also: How to create an experiment with SplitMetrics

With SplitMetrics, you can play around with your Google Store app page and see whether people would download more if, say, you added Apple Watch support or increased the number of your app reviews. Essentially, you get to test any creative app page alternative your marketing team comes up with. Moreover, the creation of any experiment is a matter of minutes.

Pre-launch app product page experiments

Another Google Play disadvantage is that you can test only published apps. If you want to run an experiment prior to the launch of an app (validate the concept and collect emails), Google Store won’t be of much help.

At the same time, SplitMetrics offers all kinds of pre-launch app product page experiments from icon to reviews number. It empowers your product page in Google Play before your app is live.

Product page conversion evaluation

You can’t understand what actually worked with Google experiments. Sure, you can see overall conversion results in the Play Store, but you don’t know what visitors did on your app page, how long they stayed, how deep they explored and scrolled.

For example, it could be your third screenshot that triggered the majority of conversions. With that in mind, you can decide to move it to the front-runner position and optimize for an even higher conversion rate on your store app page.

Analyze user behavior

SplitMetrics collects analytics on user behavior before install (more than 15 metrics) to help you build a strategy for your app optimization. You can see session length, page scroll heatmap, screenshots scroll depth, user segmentation, and much more within experiments.

Export experiment data

Another Google Play tests shortcoming is that you can’t export store experiment data. With SplitMetrics you can easily export a raw CSV file or forward a PDF to your manager if you need stats for your internal experiment reports. Alternatively, you can view all test results on its appealing and intuitive dashboard.

Control over the quality of traffic

You don’t get full control over the quality of traffic you’re sending to your experiment in the Android Store. Although testing on Google Play is technically free due to organic traffic, there’s no way to see how different customer segments react to your app store page. You can’t test app page on your main audience – the one that monetizes better and has better LTV.

When you drive paid traffic to your experiment page under the test, you can get accurate data on how each of your profiled user demographics interacted with the app page. SplitMetrics doesn’t only support any traffic source, it also provides built-in Facebook integration to facilitate setting up ad campaigns for A/B experiments.

Other pros and cons of Google Play Experiments

Running Google Play experiments, publishers get no customer support or ASO help. SplitMetrics has been working with hundreds of app store developers on a variety of app page design experiments. We want our clients to succeed, so we generously share the wealth of insights we’ve collected.

Read also: ASO guide

Google Play used to allow to run only one experiment at a time. This problem is solved and publishers can have a few simultaneous tests. Yet, marketers who want 360-degree optimization opt for SplitMetrics as it offers way more complex A/B testing and ASO solution.

In addition, you have to test both Android and iOS apps separately because Apple Store and Google Play users act in different ways. Lots of publishers lose millions of installs after applying Android results to their Apple store pages. Isn’t it easier to run experiments for all platforms from a single dashboard with an intuitive self-service tool?

The answer is definitely ‘Yes’.