Camping at the location of orbs is a solid strategy: The player must pick up orbs to win (imagine if Pac-Man’s ghosts merely lingered near the entrances to each corner of the map). It also makes the game less fun. Players no longer experience an exciting chase. Instead, the AI might spring an unpredictable ambush. Trachel and Peyrot say their goal is “not to create superhuman bots—that would not be fun and engaging for a novice player—but instead to find ways to incorporate machine learning into game AI tools already used in production.”
That might sound dull to players craving better AI. Yet the machine-learning techniques shown by Trachel and Peyrot remain helpful for tuning difficulty even when the foes that players face in the finished game don’t use it. Julian Togelius, cofounder and research director at Modl.ai, has spent nearly five years using AI to test games. Modl.ai uses bots to hunt graphical glitches, find flaws in world geometry, and sniff out situations that make it impossible to win.
“You can tell us what kind of failure state you are interested in. And then basically it runs. You send off a job, and it runs depending on how much you want to explore,” says Togelius. “And of course, we can cluster these for you and provide a report, saying here’s where you seem to have issues, and so on.”
Modl.ai’s testing bots use machine learning to adapt to each game tested, though its current implementation limits those adaptations to each specific title. Togelius says the company is prototyping the addition of deep learning that will train bot behavior across multiple games. Once in use, Modl.ai’s bots will learn to emulate the behavior of real players, which should more efficiently uncover issues that players would find.
For True Machine Learning, Game Engines Need a Revolution
When it comes to difficulty, then, machine learning can be both a problem and a solution. But crafting a fair, fun challenge isn’t the only hurdle facing developers who want to use machine learning in games. The problems run deeper—so deep, in fact, they may force a rethink of how games are built.
Performance is one barrier. Machine learning requires lots of training data for worthwhile results, and that data can only be acquired by playing a game thousands or tens of thousands of times (though bots can lighten the load, a tactic Trachel and Peyrot used in building their demo). And once the training data is collected, the resulting model can become burdensome to execute in real time.
“Yes, performance is clearly an issue, notably with large ML models that process frames for each tick of the game clock,” Trachel and Peyrot said in an email. “In our case, to avoid performance issues, we used a small neural network that was only inferring at precise moments of the game.” Scaling up to the huge open-world environments that modern players expect is another matter entirely.
Togelius says the way modern game engines work exacerbates the problem. Machine learning, he says, “will by necessity be slow because game engines are not built for this. One of the many reasons we don’t see more interesting modern AI in games is because Unreal and Unity and all their ilk are basically terrible—anti-AI in so many ways.”
Animation is another issue. Most modern game engines expect animations to be strictly defined frame by frame. This works well when animators know with certainty how game characters will behave, but an AI controlled by machine learning might behave in ways the animators didn’t expect. Designers can work around this with a physics-based approach to animation, but this places even more performance strain on a game console or computer’s hardware and comes with its own development challenges.
In short, developers face a monster of their own making. Game engines are built to use behavior trees and prescripted actions to craft worlds of AI-controlled NPCs that work well even on meager hardware. But as machine learning gains steam, these classic solutions will need to be reconsidered.
“If you go talk to a machine-learning researcher who doesn’t know game design, they’ll be like, ‘Why don’t you use new things and get NPCs that are more lifelike and adapt to how you play,’ and so on,” says Togelius. “But you can’t just plug this into an existing game. You have to rethink what the game even is.”
