How to run a CPI test: the Facebook edition

Before you put more time and money into a game concept, testing can verify whether you should continue building out your game or drop it entirely, since the results often correlate to potential audience size and predict how well you can convert from ad impressions to installs. It’s never easy to discard your prototype and start from scratch - but data doesn’t lie, so rely on your metrics from marketability testing to prove whether or not to move forward with your concept. 

At Supersonic, we’ve created a highly sophisticated process for predicting a game’s marketability, and we test across many different platforms to get the most accurate results. Here, Nir Regev, Publishing Team Lead at Supersonic, shares our best practices for running a CPI test on Facebook, one of the biggest and most used testing platforms in the hyper-casual industry, discusses how to understand your results, and benchmarks for success.

Setting up your CPI test

Testing the right number of videos, setting an appropriate budget, and targeting the correct audience are the foundations of an accurate and efficient marketability test on Facebook. For even more tips on confirming if your prototype is a product-market fit, check out this guide to marketability testing.

Videos

Each creative set should contain 4-8 videos. Don’t test more than 10 at a time because any more than that will spread the audience too thin - and if each creative reaches an entirely different user base, you won’t be able to get a sense of the true marketability of your game. 4-8 videos is the perfect amount to attract a wide audience and give you an understanding of how accessible your game concept is. 

During the prototyping phase, your goal is to get a sense of how well your gameplay resonates with users and to confirm the concept has potential. Focus on core gameplay in your creatives, with minimal editing and additions (e.g. no sound effects). Try testing several scenarios and variations, like fail scenarios or different characters. You can even test elements not in your game yet - if these features succeed, you can design them into the build when you start growing your game in the next stages. To nail your videos at this stage, check out our blog post on how to build and optimize creatives for marketability testing. 

"During the prototyping phase, your goal is to get a sense of how well your gameplay resonates with users and to confirm the concept has potential."

Budget and targeting

Set your daily budget based on the operating system: usually we suggest $15 per ad on Android and $20 per ad on iOS. These target budgets ensure you’re able to reach a large enough audience without overspending. At Supersonic, we recognize that this spend can be an obstacle for developers looking to test their games - especially if you have more than one concept. That’s why we give our partners $300 to run a CPI test after submitting a game to our platform that seems promising. If the first test shows potential, we invest in a higher UA budget and start testing across more platforms to be as accurate as possible when predicting the marketability power of your game.

As for targeting, hyper-casual games are meant to be accessible - so you need a large user base to scale quickly and profitably. When setting up your test, keep the default placements and audience data suggested by Facebook and select a wide age range.

Duration

Once you’ve set up your CPI test, let it run for 6 days. The learning phase on Facebook takes 3-4 days before CPIs stabilize - 6 days is enough time to give your game a fair chance to prove its marketability without letting the test go too long or cost you too much. After letting the test run, it’s time to analyze your results.

How to understand your test results

Looking at the metrics that Facebook provides you about your audience and the performance of each placement from your CPI test, is key to understanding whether you should keep or kill your game.

Audience

Facebook gives you all of the audience data to analyze who interacted with your creatives, showing you how accessible your concept is and in turn how well it will scale. You’re looking to reach an audience that tends to be loyal to hyper-casual games and can be reached by SDK networks. If the audience skews heavily towards one demographic, it could be difficult to achieve scale because you’d be relying on a limited user base. 

The more equally balanced your audience is, the better. To achieve this balance, adjust your creatives to cater to a specific audience - for example, test a creative featuring a female character to try and attract more female users. Quick adjustments like this can help correct the scales so your game is set up to grow efficiently.

CPI

If your CPI is stable and low at the end of the test, it’s a good sign your game has high marketability power. But if your CPI is fluctuating throughout the entire test period, it’s difficult to predict the cost and profitability of your concept. It could be worth refreshing your creatives and try running the test again to see if you achieve more stable CPI results.

Now let’s talk benchmarks - we suggest looking at results on Android these days since the iOS privacy updates have created an uncertain testing environment. A CPI of less than $0.25 on Android indicates your game has potentially good marketability power. At this range, you can start focusing on building out your game and improving in-game metrics, like retention and playtime.

If your CPI is $0.25-$0.35, the game could still be a product-market fit, but in the next stages you’ll need to focus on lowering CPI while simultaneously building out the game. Look at your in-game metrics to determine if the game is worth continuing to build out - if retention and playtime are low, it’ll take too much of your time and resources to both reduce CPI and boost these metrics to reach the benchmarks. 

A CPI of $0.35-$0.45 brings the marketability power of your concept into question. At this range, you need to focus solely on lowering CPI in the next stage instead of building out the game. For most prototypes, it isn’t worth it - it’s better to go back to the drawing board and test a new concept that could crack the code of CPI.

Placements

Your creatives are automatically shown on Facebook’s different ad placements: the Facebook feed, Instagram, and the Facebook Audience Network. The creative performance on each placement can indicate the probability of scaling on various types of platforms. For example, if your CPI is very low on Instagram, your game could scale well on social media.

The more balanced the performance across placements, the better - it indicates your game will scale well across both social platforms and SDK networks when you move onto the next stages of launch.

"The more balanced the performance across placements, the better."

It’s worth noting that Instagram and Facebook tend to be more affordable - if your CPI is low on these placements during testing, it’s a good indicator that your game will scale affordably later. Meanwhile, the CPM on Audience Network tends to be higher (sometimes 4x higher than Instagram or the Facebook feed). That said, the quality of users on Audience Network is often better, which can drive higher LTV and boost in-game performance in the later launch stages.

Predicting the future

Facebook tests are the best indicator of your game’s marketability power for the least amount of operational overhead and money. CPI tests are a predictive tool, though - they only show you a piece of the larger picture. 

Every game is a unique case, and the best practices listed here are to help guide you in setting up and looking at the results of a CPI test. The more data you can collect and analyze, the more accurate your CPI test is likely to be as a prediction of future marketability and profitability. Take all of the data into account - from CPI to audience demographics - when deciding whether to keep building out your game or cut your losses early.