Want to get a tied-and-true guide on validating mobile app and game ideas, features, or functionality? Watch the latest App Growth Point “8-Step Framework to Successfully Validate Mobile App and Game Ideas” with the mobile industry experts from Moburst and SplitMetrics and make use of the presentation together with the Q&A session at the end of the blog post.
Hagar Seri, Head of ASO & Mobile CRO at Moburst – a digital full-service, mobile-first marketing agency that helps companies to scale and become category leaders. Hagar developed the agency’s CRO service and implemented new, cross-team methodologies to boost clients’ organic performance.
Nadia Nazarova, Head of Product at SplitMetrics with 12+ years of hands-on experience in digital marketing, product marketing and product management, helping top app and game publishers improve their ASO, validate app concepts and hypotheses via A/B tests with SplitMetrics Optimize.
Mari Trypolskaya, Team Lead and Account Executive at SplitMetrics with 7+ years of experience in sales and business development. She helps large- and mid-size developers boost app store optimization, run A/B tests to validate app and game ideas, concepts, and hypotheses with SplitMetrics Optimize..
Host: Lina Danilchik, Marketing & Communications Lead and the App Growth Talks Host at SplitMetrics. She provides mobile marketers with best practices and tips on app growth, ASO, user acquisition and Apple Search Ads.
What resources for benchmarks do you use except SplitMetrics for gaming?
We put together benchmark reports on the basis of our own A/B testing platform SplitMetrics Optimize. For gaming developers, recommend taking a look at the latest ASO Benchmarks and Mobile Trends 2022 Report and Mobile Gaming Design Trends & Tips.
Should your app icon or screenshots look like other apps in the same category or should it look different to stand out? How do you make your app stand out in a crowded app store?
For smaller apps that are still at the beginning, and do not have a defined visual identity, it’s worth testing a few options – standing out vs. following the category’s best practices.
What are your minimal detectable effect and confidence level suggestions when conducting A/B testing on Android and why?
The MDE depends on the testing hypothesis. The significance level is 5%.
When your competitors don’t make any A/B tests or don’t update screenshots/icons, what would be the best approach for reaching for new ideas and building hypotheses ?
That’s a great question, not all big competitors are optimizing creatives and text metadata on a regular basis, in that case it’s worth checking the competitors that are ranked similar to your app to find other opportunities. But besides the competitors, in order to build hypotheses it’s also important to evaluate the market and audience, and try to correlate it with the app’s USPs.
Another question for Hagar, how do you identify what a/B tests competitors are running? With graphics etc.
There are a few ASO tools available that can help, such as AppTweak, SensorTower and others.
What is the minimum volume needed to conduct an A/B testing?
It depends on the testing methodology. For Bayesian – 200 visitors per variation, for Sequential – 500 visitors per variation.
How often should the icon, screenshots, and video be changed in the App Store?
It’s good to have them updated on a monthly basis. I would add that it’s important to update one asset at a time, to truly evaluate their impact on CVR.
If we are having an app that is the leader in the app category how should we test screenshots? because honestly, all our competitors are copying our screenshot ideas.
Testing definitely wouldn’t hurt. At least, you will find out that all your product page elements are optimized – and any change might decrease the conversion rate. Additionally, you may identify something new that works for your app – you can do that by testing new hypotheses based on benchmarks in associated categories, general design trends, and visual salience insights.
Please can you explain the meaning of audience tests for brand new app ideas? For example, I have a recipe app idea for a specific type of diet ( e.g diabetes) , how would I go about that?
So you probably would like more people with the possible interest in this recipe to find it, right? But first you need to understand who are those users who might be interested in it. The research helps you understand the baseline audience parameters (e.g. female and male, 18+, interested in health, food, cook, diets, etc.). However it’s too wide, and by running an audience test you can specify your users and understand the specific parameters for finding them and bringing to your recipe.
When the emulated tests are served on social media platforms, do these have to be only those that the company is active on? E.g. if our company does not have a tik tok account, will the tests be also served to these users?
Emulated pages are not presented on social media platforms. They are actually web-pages, and when running a test, you need to set up a simple ad campaign on any social media platform (Meta, Google, TikTok, Snapchat, etc.) and from your ad banners users will be redirected to the emulated pages.
How does SplitMetrics work for Indie Developers with apps in the iOS store? The Dev would need to purchase ads like Facebook ads pointing to your test site for the testing? What is the cost for this?
The publisher/developer should drive paid traffic in order to collect user behavior metrics. The cost per 1 test depends on your CPI from the particular channel and conversion rate – the higher the CR, the less budgets you would need for this test.
