How Unity wants to ride AI wave by integrating natural language interfaces and data training capabilities into its engine
Unity has finally shared details on the adoption of AI technologies and new tools coming to the engine and its editor. Let’s take a look at how the company plans to expand in this area and what growth opportunities it sees there.
Unity first teased its Unity AI initiative in March with a short video (can be seen below) and welcomed creators to join a special Beta program. According to its description, the company is “building an open and unique AI ecosystem that will put AI-powered game development tools in the hands of millions of creators.”
CEO John Riccitiello then told Reuters that in the future, developers will be able to visit Unity’s marketplace, select a specific AI product (e.g. artwork generator) and use it to create content. But other than that, details about the company’s AI plans remained unclear.
However, in its latest financial report for the quarter ended March 31, Unity went into detail about the adoption of AI tools. Here are the key takeaways.
How will AI be integrated into Unity Editor and other products?
Unity believes that AI will accelerate its growth and profitability, expecting it to be “tailwinds for many years to come.” During an earnings call, Riccitiello also noted that Unity has “sustainable competitive advantages” in this field due to the very nature of its platform.
So the company sees four main areas where it can benefit from AI:
- Unity Editor — reducing the complexity of creation tool chains to attract more developers. The company expects creators to make the first drafts for content (code, animations, physics, etc.) using the new natural language genAI tools and then edit and improve them with classic deterministic tools. This should make the creation process more “iterative and collaborative”;
- Unity Runtime — integrating Barracuda, the company’s cross-platform neural network inference library, into the runtime. It allows trained models to run locally on devices so that developers don’t have to worry about cloud compute costs or latency. Unity thinks a combination of Barracuda and the runtime can be applied to game development (e.g. creating more responsive and “real” characters) and other industries (e.g. simulating manufacturing lines);
- Unity Network (ads, analytics tools, publishing systems, etc.) — using machine learning/AI-powered services to help the company’s clients find new opportunities and growth points in these areas;
- Training Data — training models based on datasets captured during the creation and operation of games and other real-time experiences. The company thinks this will allow its clients to use Unity as a “data creation, simulation, and training engine for their own needs.”
Why does Unity think that AI will be beneficial to developers?
Riccitiello told investors that new AI features in Unity will allow developers to:
- Create content in the Editor faster and cheaper;
- Create things that were never before possible.
“So developers will do more on our platform,” Riccitiello said. “And that applies to the game development side and the digital twin side.”
The Unity CEO believes that AI could improve both game worlds (by making them more compelling) and player interactions with those worlds (for example, by making NPCs more “intellegent” through non-scripted dialogue options).
Speaking of digital twins, virtual copies of physical objects and systems, Riccitiello noted that AI could improve the way things are designed and produced in real time and for the real world.
What that gives you is a digital twin which is just not catching you up for the moment but helping you do next what is necessary to run your factory better, to design a better building, to teach people in a better way, to run a better airport or at least a smarter airport. All the things people are using now get better, faster and more future looking in a way that really enhances their business. CEO and president of Unity
How will Unity combine AI and deterministic tools in game development?
Unity sees this combination as something most professionals will use in the future. “Deterministic” means all the tools included in the engine that allow developers to get determined, non-random results of their work.
What Unity plans to do is introduce a natural language interface within the Editor to “allow people to go back and forth between using a relatively complex tool that is deterministic where they have to work with it to yield an outcome and natural language which would give them a first draft in what they’re trying to do.”
As Riccitiello explained, some things like creating a volumetric fog for a game will still be easier to do with classic deterministic tools. But if a developer wants to, for example, generate a specific asset to test their ideas or vision, using a natural language interface would be preferable.
We're obviously, by far, the leading platform for this type of content creation. And what we expect is people to use both because just using natural language is really almost impossible and it's likely to be impossible for a very long time to be able to create something that benefits from team production processes and get something that is consumer ready. CEO and president of Unity
Unity expects game developers to become more productive by leveraging the upcoming AI capabilities of the Editor. On top of that, the company hopes that its natural language interface will attract new users and make the engine easier to use for non-professionals or, say, artists that don’t want to or don’t know how to interact with the coding side.
“So I see a lot of opportunity for us to add ratable revenue as our developers start to use more AI tools and to build more complex and richer, more advanced worlds,” Riccitiello said
What challenges does Unity see in this area?
Video games are very different from static mediums like art or text where chatbots and generative AI tools are already showing some great results. Unity noted that in order to train data for models that could be applied to game development, you have to take into account not only the outcome of player actions, but also all the inputs, physics calculations, lighting, and other things that dynamically happen within the game.
So the biggest challenge here is to create tools that would allow to train data inside the Unity Editor and at runtime. But Riccitiello still believes that natural language interfaces won’t be able to create complex games and products in the near future.
I have no doubt that pure natural language will be able to produce really simple games as long as enough rules are put into the tool to create it, but the rich content that we've gotten used to is going to be harder to train. Now it doesn't mean it won't get trained. But companies like Unity are going to be deeply involved in that process, and others are going to be outside looking at it. CEO and president of Unity
Despite all the challenges, Riccitiello stated that “AI is going to have a profound effect on the industry” because: 1) it will allow developers to create game types that weren’t previously possible; 2) it will cut costs in high-end production.
Why are some experts skeptical about Unity’s AI adoption?
To summarize Unity’s stance on AI, the company believes that the upcoming features will create new revenue streams and democratize pricing for its products and services. “AI is going to be positive for the industries we compete in,” Riccitiello said. “And it’s going to be positive for Unity’s market share in those industries. And it’s going to be positive for the revenue that we yield from those market share point gains.”
However, not everyone shares the same optimism as the company and its leadership. Benchmark equity research analyst Mike Hickey called Unity a “loser in an AI World” (via Barron’s).
“We believe AI and machine learning could be destructive to both Unity’s growth and profitability,” he said, adding that emerging startup companies could take market share by making similar products in the AI field at lower expenses.
AI is still a very sensitive topic, generating both enthusiasm and controversy among developers and content creators. But it is clear that more companies are going to jump into this space to grab a share of this market.
Unity’s plans are definitely ambitious, especially in terms of deploying AI capabilities on players’ devices and training data at runtime, but it remains unclear how all these things will work in practice.
