How Forza 5 and the Xbox One use the cloud to drive machine-learning AI

[BajiRav]

Ars

 

 

Under the hood, those AI opponents house the aggregate learnings of millions of cloud-processed races.

 

For months now, Microsoft has been touting how Xbox One games will be able to make use of cloud computing resources to handle certain time-insensitive tasks that aren't really feasible to calculate on a single local box. While Microsoft has gone into some detail on how this process works, a lot of the company's talk has taken the form of vague hand-waving about how the magic of the cloud will make everything more powerful.

So when we had a chance to talk to Forza Motorsport 5 Director Dan Greenawalt recently, we wanted to take a deep dive into how, precisely, the game makes use of Microsoft's cloud resources to power its adaptive, machine learning-based AI system, called Drivatar. And we learned how processing a massive amount of data on Microsoft's servers allows for possibilities that Greenawalt says weren't really possible on previous consoles.

It?s learning

The Drivatar team at Cambridge's Microsoft Research campus has been creating adaptive AI opponents based on real users since the first Forza Motorsport was released on the original Xbox in 2005. But previous games were only able to learn from local races and had to process and learn from that data on local hardware. Greenawalt says being able to aggregate and process data in the cloud generates results that are "like night and day" from what has been possible before.

"We weren't thinking about cloud with the original Drivatar, but the original Drivatar was a learning-based system," he said. "We want it to be very human. The issue is it needed to really crunch a lot of data, and the Xbox, the Xbox 360, and the Xbox One aren't powerful enough to do what multiple servers can do to crunch the data. I didn't know I needed the cloud until we had it."

Before the magic happens, Forza Motorosport 5 records incredibly detailed data about your gameplay locally. "Every time you race, every time I race, the box, not the cloud, records telemetry about where you were, what car you were in, how you were using the throttle, where the other cars were on the track. Incredible amounts of detail," Greenawalt said.

Besides data about your car, the game is also keeping track of relational data, correlating your actions to those of the other cars on the track. This can help answer questions about when and why players take certain actions, Greenawalt said. "Why would you dodge all the way to the outside of the tracks? Is it because you're a bad driver, or because you're going around somebody? It's just data, but we over-record the data because it's possible that any of those other pieces of information, like where the other cars are, will be important to the actual data itself."

After each race, the Xbox One uploads a few kilobytes of data representing the race to the Microsoft servers, where the Drivatar algorithms are constantly churning through the aggregate for patterns. "If you think about it, you're trying to boil the ocean," Greenawalt said. "That's what the data is, you're talking about millions and millions of data points, and what machine learning does is draw correlations and generalizations in a massive field of data. That's not something that a PC by itself is necessarily set up to do a good job of. It can, it just takes a lot of time."

Right now, that ocean of data is more like a gentle stream, with about 400 members of the Forza team contributing Drivatar data. Even so, the system has already been able to learn some behaviors just by analyzing thousands of individual races, completely on its own.

"We didn't design it to do anything," Greenawalt said. "It looks at all this data and it crunches the numbers and correlates and generalizes and comes up the behavior. Like, 'Oh, I saw 20 percent of the population, when there was a car in front of it on Laguna Seca, they actually went for the outside and then dove on the inside.' ... We look at it and we go 'Oh, he faked him out.' The computer doesn't need to know [that name of the move], it just knows you did this in that situation. So it's learned how to fake."

Enlarge / The precise way you take this turn is just another drop in an ocean of machine-learning data but it also key to crafting the "personality" of your own Drivatar.

 

Greenawalt recalled a concrete example of a time where the Drivatar AI surprised him with a learned move. "I saw three guys going in, going three abreast, and I was like 'Let's see what happens here.' They were totally aware of each other and they managed it, and then one guy got up ahead and I kind of tucked in behind him. Then I saw the two guys basically faking each other out through the corner. One guy kind of went to the outside, so this guy went to the inside, the [the first guy] dove inside, [the other guy] went around, and you had a double pass."

"This is what humans do, but you never see AI do that unless it's scripted," he continued. "I've never seen it in a game unless it's scripted. And it learned how to do it because we do that. We do it to the AI, and that's what's really cool to see."

Squashing trolls and personalizing AI

The danger from learning based on real humans, of course, is that the system might end up learning from groups of racers who are either unintentionally bad or intentionally goofing around in the game. Even in the limited sample of developers currently contributing to the system, Greenawalt admits "we have asshats in our group that I work with every day. They're already trying to train it and troll it."

This is where some minimal human judgment comes in to prune the more lackluster learned behaviors generated by the Drivatar system. "We manually tell it... 'Don't learn that type of thing,'" Greenawalt said. "It creates a new behavior, it says 'Hey, I've seen a lot of this, I'm going to do this thing.' We say 'Don't do that whole thing, and if you ever see anybody do this thing, just forget about it.'"

The behaviors that pass muster, though, are then downloaded back to a local Xbox One in the form of distinct Drivatars that are meant to represent the aggregate driving skill of actual users. These personalized racers get more accurate as they're fed more individual race data, but Greenawalt says the system is pretty good at figuring out what your Drivatar should do based on what you yourself have done in similar driving situations.

"Have you ever seen those towers of champagne glasses? Imagine every champagne glass is a [racing] condition," Greenawalt said. "So the more you pour into it, the better it gets all the way down, and then we start retiring data. ... It's like a rolling average of their skill. The generalization is pretty good. There are three races to set up your Drivatar, and those races give us a good set of corners and cars for us to generalize on."

More than any additional graphics or physics modeling capabilities, Greenawalt says being able to take advantage of easy cloud APIs for Drivatar is the biggest thing differentiating the Xbox One from previous generations of console hardware. And he says things like Drivatar are just the "tip of the iceberg" for how these functions can be used.

"This whole Drivatar idea could work in a first-person shooter," he said. "If you think about the routes you take in a first-person shooter, there are certain paths, there are weapons you like, your accuracy, your percentage. I think it's possible, I don't know it's going to happen, but the truth is this was our take on what to do with the power of the cloud. Other creators are going to come up with other things."

[trooper11]

They certainly make a good case for using the cloud to generate a better AI through learning from human players.  I do think that is something that could be very useful for certain genres and developers. 

 

Its good to hear more about how this stuff works vs arguing over vague ideas.

[Fourjays Veteran]

This does sound very clever and makes a good case for the kind of things the cloud can actually do. Would want to see it in practice, but assuming it all goes to plan then the AI in Forza could be the best in a racing game since GP4. Would sorely tempt me as I've missed racing against good AI ever since.

[vcfan]

Truly next generation stuff.

[spudtrooper]

Great writeup.  Xbox live / X1 could very well be the end of dumb ai - any game could implement this techology to create better games.  I haven't really played a racer since Project Gotham on launch 360 but i am looking forward to this game.