Benchmarking in game development or where to start creating a game?
What are the four steps you need to take before you start writing a game concept and development, – said Playbeat product manager Alexandra Shtachenko.
Alexander read a report at the GetIT! conference, which was held in Odessa in early September. We are publishing a printed version of the speech.
Alexander Shtachenko
Benchmarking is the work of comparing your game with the games of competitors
Benchmarking in game development is a comparison of a future or current product with competitors’ products that are already on the market, studying their indicators, key metrics, mechanics, as well as the development approaches that are practiced in them.
This analysis is quite resource-intensive. Many do not know how to do it, where to dig up the indicators of a particular game, and this is possible. In this article I will tell you how and why to conduct benchmarking.
The first stage of benchmarking is collecting information about the market
Before we start doing something, we conduct an analysis, collecting information about the market and the projects present on it. Information is collected through three channels: through reports, news and blogs.
1) Reports
You can and should use the reports that are available on the network. Now I will tell you about some of their sources.
Newzoo is a cool resource that makes both paid and free reports on the mobile market, the US market and esports. Newzoo differs in the way it collects data. They select groups of people, do a survey, and then extrapolate this data to the whole world.
Superdata is a good site where they ask for money for almost all reports. The cost of the report on it is from $ 500 to $5000.
Sensor Tower, App Annie, AppsFlyer, Think Gaming, Prioridata are sites of analytical services and tracking. They also have data on them. I especially recommend the last one – Prioridata. You can subscribe to two games for free on it and view every week how many have downloaded them during this time and how much money they have earned. There is an error, but it is quite small.
2) News
News is also very important. You can see where and what games are coming out, what software is running out, what practices and innovations are being introduced into projects.
The first news appears in professional communities on Facebook. No need to crawl all the sites. You have entered, and someone has already posted something.
But I also visit websites. The main English–language resources for me are gamesindustry.biz and pocketgamer.co.uk , and the main Russian–speaking – app2top.ru .
I also strongly advise you to use quora. Here you can always ask your colleagues if you don’t know something. With a high probability you will get a good answer. It won’t be fast, though. They respond with a delay of almost a week.
3) Blogs
It is very useful to know the blogs where the data is collected. I will mark my blog first of all – progamedev.net . You should also pay attention to the blogs of companies and services that are engaged in analytics.
Once a quarter, they write some useful data that you need to download and see where you are and where the rest of the market is. Why are you below the Top 50 grossing, how much do the guys who are higher in it get, how much is needed for traffic, and so on.
The second stage of benchmarking is the study and selection of projects
Having dealt with the market situation, you should select projects for further analysis. As part of the selection, you must answer four questions:
- what kind of projects?
- where are they launched?
- what are their indicators?
- what is their audience?
1) What kind of projects?
The most important thing in benchmarking is to understand who we are investigating after all.
More than a hundred games are played every week. You won’t be able to see everything.
A dozen projects are being developed. You can look at them, try to understand why they are given a feature.
You also need to monitor the sotflonchami of leading companies.
By the way, I have a note in my blog. It’s called “Softlonch Hunters”. In it, I wrote how and where to determine the top titles that have just been released in softlonch.
2) Where are they launched?
The situation with softlonches is complicated. As a rule, they are not advertised. So they should be found first.
Then you will face the question of where and how to download the game that just came out. To answer it, you must have several Android and iOS accounts. And, of course, you should be able to switch between accounts of different countries.
Why?
If earlier games were first released in the Canadian versions of app stores, where we could watch them, now the situation has changed. The price of traffic in Canada has become very high. Therefore, they are being launched in Mexico, the Philippines, Australia, New Zealand, and even Sweden.
3) What are the game indicators?
You have selected the projects you are interested in and even downloaded them. What’s next?
As part of the next step, you should find out how they perform, whether they are worth attention or not.
Learning the game is not a day or two. If it’s empty inside, it doesn’t bring anything, why waste your time on it?
For this reason, you need to monitor their KPIs. It is very important where to get them. You can do this using the resources that I have already indicated above.
4) What is the audience for games?
The fourth aspect of the study is studying the audience of the selected project. It is necessary to find out who fell for the project, who played it, who pumped it, and so on.
How do I find out?
There are three ways to investigate. All of them are well described in the article “How to conduct a marketing analysis?” by Ilya Eremeev.
There are good tools that allow you to evaluate the potential audience of a particular game in principle. Facebook has a good toolkit, there is also Google Trends.
The third stage of benchmarking is the deconstruction of selected projects
Having answered the above questions in our research, we compile a list of titles of the area of interest (in my case, social casinos) with top monthly earnings and the audience attracted for the month.
Next, we leave only those titles that most fully meet our criteria. For example, in my research, I screened out all the games for large IP addresses and those that came out before 2015.
We begin to study the remaining projects under a magnifying glass, disassemble them into components, and also keep a log of all changes since the release (from functionality and prices in stores to art and UI).
In the course of their deconstruction, it will be useful to compile a comparative table of the mechanics available in these games.
Why is it important?
The fact is that with its help you can see the general mechanics of top earning projects. That is, the most important components that provide leaders with high earnings.
This is a list of mechanics that you need to put in your future design document.
The fourth stage of benchmarking is the breakdown of components by a.e.r.m. and USP/KSP
The next step is to create the basis for your future design document.
You decompose all found mechanics into a matrix a.e.r.m.
a.e.r.m. is the most important table in game design. Those game designers who don’t know about it can be safely dismissed.
If a game designer is not familiar with this table, does not live with these thoughts, then, most likely, he makes features for the sake of features, introduces them “because Supercell and King have them,” and not because he needs to raise the KPI of the game.
There are four main components in the table: acquisition, engagement, retention, and monetization.
In the game, we have to ensure a high KPI of each of the components. Each component is divided into three types according to which component of the game is present: the main mechanics, metagame or social.
When you assembled the feature set and decomposed it into this matrix, that is, you painted what is responsible for the acquisition of users in the core of the game, which mechanics are used to retain in the meta, and which work for retention in the social network, and so on… So, after that, you see if there are cells left unfilled in the matrix. If there are any, it’s too early for you to write code, it’s too early to pay artists’ salaries, because you’re not ready for development yet, you haven’t finished your research yet, you don’t know how to make your game successful.
In addition to a.e.r.m. mechanics of the studied titles, it is necessary to paint according to KSP and USP.
- KSP – key sales points (key trading offers)
- USP – unique sales points (unique sales offers)
The key positions are those that everyone has, this is what it is impossible to enter the market without. We call such features “must have”, but they cannot be done for show. Their quality should be at the market level. Preferably higher.
Unique features are those that no one has. When you are going to use this or that unique mechanics, be sure to check whether a patent has been filed for this mechanics. Otherwise, you will then explain yourself for a long time in court and give the developers some of your money.
***
When all this is done (market assessment, selection of titles, checking of titles and their analysis), you are ready to work on your own documentation and development.
The video of Shtachenko’s speech can be viewed here.
