How outsourcing companies can use neural networks in processes - OWL studio experience

Vera Velichko, CEO of OWL studio, spoke about the tasks in which neural networks can help CG artists today, and in which – you should not expect much from them.

Vera Velichko

Neural networks are one of the most discussed topics of the last few months and, I must say, I still haven’t decided for myself how to treat it.

OWL studio is a service company, we create graphics for games. Where AI is a gift from heaven for game development companies, an opportunity to save on the most expensive processes, for us it is a direct threat to business.

And also – this is my personal pain and longing. I was originally an analog artist, I studied at an academic school (brushes, canvas, oil – that’s all), but when it came time to decide how to earn a living with all these beauties, it turned out – in no way.

The path of the modern traditional artist is thorny, tortuous and runs mainly through good connections. Then the transition to CG was a great opportunity to monetize my skills, but the restructuring to the computer graphics toolkit took several years.

So, do you know what I’ll tell you about AI?

Disgusting!

My analog soul is not that indignant – she simply refused to have anything to do with this fiend of hell. But my soul is my soul, and as the head of the company of artists, I had to react to the situation as quickly as possible. The gaming industry is always surfing, if you don’t ride a wave, at best you will be left without a board.

Fortunately, the second art director of the studio, the father of the entire art department and the guardian of my nervous system, became interested in AI even before it became mainstream – and became my first assistant in the introduction of new technologies.

First of all, we had conversations in the team. I still believe that in a situation of constant and sometimes very abrupt changes, it is extremely important to be in dialogue with the team: if people understand what is happening, what situation we are in and what challenges we are facing, and the management is transparent in their actions and intentions, they get involved in the processes and give a much deeper and a qualitative result is better than when they simply act according to instructions.

The first conversations caused quite a violent reaction: many artists were afraid that they would be forced to work with neural networks regardless of their desires, that they would lose the most interesting part of their work and turn into “neural network operators”, that after installing neural network pipelines in the studio there would be cuts.

Of course, it would be difficult to motivate them to learn new tools if we had not previously discussed all these issues.

But after two weeks, half of the team changed their anger to mercy and began to look closely at the topic. Our initiative group independently studied the tools of work, we created a separate chat, conducted several streams and began experiments.

What tasks did we test AI on?

Search for concepts (throwing ideas to reduce the time for concept art)

Result: as in that joke. Do you know how ships are made in bottles? They take a bottle, throw glue, sticks, rags, pieces of paper, threads there, shake it all – it turns out all sorts of stuff, sometimes ships.

At this stage, it is unpredictable whether the result of the work will be useful for the conceptualist or it will be faster and easier for him to generate a concept on his own.

However, in one situation AI is certainly useful: in a state of white paper, when ideas do not come to mind. In this case, working with AI will be at least a good start.

Separately, it should be noted that at this stage we are free from copyright restrictions: the ideas generated by the neural network are then processed completely, we do not use this content directly in the final product.

A separate positive moment was the opportunity for customers to come to us with some sketches of their ideas created with the help of AI. This greatly simplifies mutual understanding.

Collaboration on the concept with the client

Search for options for details and stylization based on the sketch

Render

From our point of view, this is the most useful feature of AI at this stage: it can be used to speed up rendering processes. For example, rendering realistic characters, which previously could take up to three weeks, can now be done in four days, and most of the time will be spent searching for a concept.

AI perfectly copes with mechanical processes in an understandable framework: render an object in a certain style, improve the quality of the render, zoom in, etc. At the moment we see the main area of application at this point.

Working out details based on a sketch

Example of finalizing a concept created with AI

Generating assets identical to existing ones

The idea is essentially simple and has already been described in several articles-reports on working with neural networks: preparation of simple art, such as buildings on the map, props, stones, bushes and other similar content, which has a huge number of analogues and which is essentially simple and of the same type.

Usually, this is the least creative part of an artist’s work: you just need to create +10, +20 or +100 objects on the principle of “the same, only different”.

We have never been able to successfully apply AI for this yet.

It is important to understand here that if we work, for example, with lines of objects for a merge game, it is extremely important for us to observe the heredity of objects in the line, their readability, uniformity of detail, a clear difference between one grade from another, the difference of each line from all other lines on the project.

So far, neural networks cannot cope with “keeping in mind” such a number of parameters at the same time. It will take less time to work with your hands than to edit the AI results. However, we do not give up hope to implement AI into this pipeline as soon as such an opportunity appears.

Edits

Change the color, adjust the angle, pose, replace, remove or add some object – all these are edits. And previously, they were the main reason for overworking tasks and most dangerously affected the budgets (or profitability) of the project.

Purely psychologically, such edits usually seem simple to the customer, he does not see great difficulties in their implementation – while artists burn out on such processes as matches. Two or three iterations of edits can reduce the productivity of the artist for a couple of weeks ahead, and after the fourth, a vacation may be required to restore creative abilities and actively immerse yourself in the work.

Perhaps it was with these capabilities that AI “sold” itself to our team. Thanks to getting rid of the torment of endless reworking of their works, the artists were ready for a lot. A bridge of hope for a dream job was thrown across the initial abyss of xenophobia. After all, if neural networks take over all the most tedious, boring and frustrating work, won’t it finally become the opportunity to create that we all dream of?

Finalizing a character based on a sketch

What are AI’s weaknesses?

It’s still not a “make it nice” button

To achieve an adequate result, you need to be able to work with a neural network.

I watched the training videos of our team, our art director came to me and demonstrated all the most incredible and impressive AI capabilities for two hours…

All he has achieved is my sincere conviction that I will not do this.

But it became obvious that appropriate skills are needed for adequate work with AI. You need to understand the logic of its work, understand the tools and settings, be able to correctly form introductory notes, and constantly be in the flow of information – AI is evolving literally before our eyes.

The logic of the concept

If, for example, we create a character in some kind of body kit – armor, clothing, armor, with gadgets of one kind or another – we can not even hope that the neural network will create at least some logical design. Most of such a “body kit” would be impossible to use, and some would not be able to exist in principle.

The more important the logic of the concept on the project is, the more you will have to edit.

If the project is serious enough and the logic of the world is given significant meaning, the neural network is suitable only for the conceptual stage. An attempt to use it to make finales can turn out to be more expensive than drawing in the traditional way.

The quality of meaning

Purely on the principle of its work, the neural network generates a given meaning from the details. If we ask her to create a picture that satisfies a large number of parameters, the neural network will try to combine all the listed parameters in one picture, and the more parameters there are, the more details there will be in the picture. While a person can operate with symbolic, allegorical, intuitive emotional interconnection, and thereby create the simplest possible images that contain many levels of meaning.

Where AI does not cope at all and human hands are needed?

Narratively strictly defined task

There is a very fine line: a strictly defined task with an unambiguous mechanical way of performing, and a strictly defined task implying a creative approach.

That is, if we need to perform specific actions, for example, to render a sketch, add or remove a certain object, the neural network copes perfectly. But if we need, for example, to create an illustration for a specific plot, to keep the right mood in it, the correct reading of the setting and the history of the game, compliance with the features of the gameplay (for example, save zones for the interface, or a certain design of the location necessary for the gameplay) – “torment” the neural network to achieve the result, it becomes order is a more resource-intensive task than to do with your hands.

Promo illustration for Guns of Boom, in which it was necessary to take into account a large number of narrative details and clearly work out the composition

Stages of work on art

A task with a lot of nuances

One of the projects our team is currently working on uses a very simple style – a simple, medium-volume render, a well-trained artist can render a character in a few hours.

However, high purity and accuracy are required here: the transfer of materials, textures, the nature of highlights, shadows, effects – all this is strictly defined and it sometimes takes almost more time to polish all these details than to render.

And usually it happens like this: there is one style-forming artist who does everything quickly, easily and in the bullseye. His style is taken as the basis of the project. Then all the other artists learn to imitate him, and each of them spends most of their time imitating someone else’s style – until it becomes natural. In such situations, at the stage of the production setup, the deadlines for art are terribly “creeping” and result in serious overruns, but then, if the setup is completed successfully, the development is accelerated several times. The simpler the render itself and the more important it is to take into account all the nuances, the less sense there is in using a neural network.

For the same reasons, AI is not suitable for generating match3 chips and other puzzles of a similar nature, lines of merges, infographics, iconography, interfaces.

Interfaces created in our studio for the Pocket Walley project without AI participation

Tasks that go beyond analogues

Everything is simple here. AI does not create fundamentally new concepts and works best in the area of typical tasks. As soon as we are talking about the search for visualization of any ideas or images that have no analogues, the quality and adequacy of the result of AI work drops rapidly.

Result

The most important task for us at this stage is the issue of compliance with the copyrights and NDA of our clients. Since at this stage there is no final legal framework governing the use of AI-generated content, we do not use any pipelines in our work without prior agreement with the client. Also, for us, the issue of paramount importance is the preservation of the inviolability of our clients’ content – we cannot use a neural network trained on a graph belonging to our client anywhere except on his project.

Thus, the only safe source of art for AI training becomes the personal portfolio of the studio (projects whose copyrights fully belong to us), as well as the personal portfolios of our artists. Considering the size of our team (more than 50 people in the main team), this is a fairly large library. But in the case of deep integration of AI into the processes in the studio, we also have to work on creating standard assets, on the basis of which we can “train” the neural network for the most common tasks.

Nevertheless, despite all the technical, legal and conceptual difficulties, AI has already shown excellent potential in its work and allowed us to open new directions for our clients:

• consultations on the use of AI on the project: in what tasks, from our point of view, neural networks can be used, what resources will we save at the same time;
• training of the network based on the client’s art, for subsequent rapid generation of typical content, then – operational work with AI and correction of the result to the final;
• edits and corrections due to AI, which significantly reduces the risks of breaking deadlines even in a search situation;
• work with AI using open sources in situations where it is acceptable for the client (for example, for quick generation of creatives, test content, marketing graphics, etc., reducing the cost of such work several times).

All this allows us to offer our services to those clients for whom it was previously impossible to use an outsourcing studio on a budget.