What is Marketability and why Engagement matters in your first CPI test — explained by ByteBrew

When testing ads and creatives, a prototype’s marketability is a thing that shouldn’t be underestimated. In a new column for Game World Observer, ByteBrew Chief Strategy Officer Christopher Lefebvre explains what marketability is and what metrics developers should pay attention to.

Christopher Lefebvre

In my decade of experience in the industry, I have worked in user acquisition, monetization, mediation, and prior to joining ByteBrew, in publishing. Overall, I produced and tested hundreds of prototypes and launched 20 or so chart-topping mobile games, albeit in radically different market conditions.

The below is not a “formula”, as it is not a guaranteed statement of fact, but more a framework to help interpret a few of the many variables that impact a prototype’s potential to reach a full launch.

What is Marketability?

Marketability = Creative(Theme+(Execution(Mechanic X Art)))

From the player perspective, the front line of Marketability is “Creative”. A good creative makes how your game is played clear within the first second, ideally in the first screenshot, and why the game is compelling within the first 3-5 seconds.

If the creative has captured the attention of the player, the next factor impacting Marketability is “Theme”.

By Theme, I don’t mean what you think the game is about, or what you want the player to think the game is about. Theme is the first word that will form in the player’s mind when they see the Creative. This visceral response will be what determines if any further attention is paid.

It will most impact the breadth of appeal in your campaigns. The broader the Theme (i.e. “food”), the broader the market appeal, which can be measured by the degree of similarity of marketing KPIs (i.e. Impression distribution, CTR, CPI, IPM).

Themes with more niche appeal, (i.e. Space) will see impressions distributed to a narrower subset of players with better performance greatly weighted among specific audiences. Consider Theme a line in the sand a potential player must cross to truly begin assessing the story the creative is telling.

If the player hasn’t been turned off by the Creative or Theme, the game’s “Execution” just might. Execution, composed of the game Mechanic and Art, will most impact depth of appeal, which determines how many of those players within your breadth may potentially install the game.

In assessing the potential impact of Art in Marketability, think to the primary role of the Creative, which is to be clear and compelling, and consider how the game’s Art is helping or hindering that effort.

• Is your color palette conducive to understanding game elements?
• Are there unnecessary UI elements drawing attention from core gameplay?
• Are you experimenting with multiple camera angles?  
• Do you know the potential impact on Marketability your character models will have?  

With Art, more than any other variable, the key is to be able to standardize and isolate these and other critical sub-variables in order to better diagnose the impact of Mechanic on Marketability.

Mechanic, in parallel with Creative, has two functions which impact Marketability. How the game is played must be immediately clear, immediately followed by why it is compelling.

When variables like Theme and Art have been effectively standardized or isolated, the importance of the Mechanic in Creative is paramount, and cannot be fully understood without Engagement data to support interpretation.

For example, a game has higher than target CPIs, say, $0.50 when the target is $0.30. Imagine that Facebook Ads Manager campaign, when broken down by Age & Gender, indicates that each demographic received a similar size of impressions and achieved a CPI of $0.50 each.

While the CPI is $0.20 higher than the target, the game still has a Marketability potential with proper optimization, which is significantly better than if a small cohort of unbalanced demographics were responsible for below-target CPIs.

Then, let’s say you have made no “mistakes” on Art. Character models are similar to those used in the past, and they don’t overly skew results or impact Theme. The game has a simple color palette and nothing visually distracts users from the Mechanic. Everything indicates that your creative should be telling a clear and compelling story, but how can you prove that?

Why Engagement data matters in your first CPI Test

Continuing from the above example, imagine your Engagement Data (Sessions Lengths, Playtimes Retention), don’t indicate a clear problem. To see whether the Mechanic is clear to the players, we can build a funnel and analyze the data obtained.

Graph of the player’s linear progression through levels 1-7

Analytics tools can identify the amount of early churn. The graph above shows that a large number of users drop out at Level 2 and 3, after which a flattening occurs. Additionally, we see an increase in the average time to complete on early levels, but a normalization of this time among those players who managed to progress.

This indicates that there are likely issues with the clarity of the Mechanic, which is hindering both Engagement and Marketability. Potentially low effort adjustments could have a high impact on these metrics. But, how do we know what needs to be clearer?

Custom funnels of events tracking the progression of one level

By making up custom events, we can track any and all individual actions the player may take. In this case, we built a custom funnel of five events to track the player’s ability to progress as intended and identify all common paths outside those that we intended. This is particularly useful in genres like hybrid-casual, where there are more non-linear paths for the player to take. From here, we can quickly identify where the loss of clarity occurred.

Map of events showing the progression through one level and indicated the dropoff of players

Let’s break down a situation where Marketability KPIs remain the same, slightly above target, but the Engagement data is at or above target playtimes, but below target retention.

Graph of the player’s linear progression through levels 1-7

The above graph is more concerning, as it reveals the issue with how compelling the Mechanic is. We can see minimal churn in the first level(s), but it consistently increases thereafter. The Mechanic was reasonably clear, but overall, most players didn’t find it compelling. While this is often more detrimental to the project’s future, it’s worth ensuring that the Mechanic allows for some degree of mastery.

Graph showing the number of players who chose a shortcut while progressing through the level

The above graph shows the number of players, per day, who completed said level demonstrating the mastery of the Mechanic that we feel makes the game more compelling.

If we see low Engagement, this means players ignore the more compelling elements like shortcuts. So we need to consider some low-effort tweaks that could improve the metrics. For example, making changes to Art could help tell a better story in the Creative and lead to more players discovering compelling aspects of the Mechanic.

Too often, without Engagement data, decisions are based solely on Marketability KPIs, or only on high-level Engagement KPIs, without fully understanding why we see certain results. This can increase the risk of killing a potentially promising prototype, focusing resources on prototypes with low scaling potential, or failing to increase the likelihood of scaling our subsequent titles.

By building funnels, cohort reports, as well as other tables and graphs to collect data from custom events, developers can conduct a deeper analysis. This could remove the risks from the equation and increase the probability of a prototype reaching a full launch. ​​If you want to learn more, please feel free to reach out to success@bytebrew.io.