A Waymo engineer told us why a virtual-world simulation is crucial to the future of self-driving cars

Waymo

Neuroscience led James Stout to computer science, which led to Google and Waymo.

• James Stout was convinced that he'd become a neuroscientist, but a shift to computer science led him to Google and later Waymo, where he pioneered on the company's virtual-realty testing simulation, Carcraft.
• Carcraft has tested vehicles in five billion miles of simulations, while Waymo has racked up a mere eight million miles of autonomous driving in the real world.
• More recently, simulated testing has begun to merge with actual testing and Waymo has been able to install sim systems in real-world vehicles.

Editor's note: Business Insider had the chance to speak with Waymo employees from different parts of the company to learn more about their work. What we discovered were some of the coolest jobs at Alphabet, Waymo's parent company. This is the second in the series. For a brief history of Waymo, click here.

Waymo has racked up over eight million self-driving miles since the former Google Car project kicked off almost a decade ago. But that impressive real-world number is nothing compared to the mileage amassed in virtual reality.

"We do eight million miles every day in simulations," said James Stout, a senior staff software engineer at Waymo.

Stout has worked at Google since 2009. He joined the Google Car project full-time in 2013 and applied his background in neuroscience and artificial intelligence to the daunting task of teaching a vehicle to drive itself. Thus far, he and his teammates have navigated an astonishing five billion miles in a "Matrix"-like virtual real called "Carcraft."

"It's running 24 hours a day on Google's data centers," he said. "And there are 25,000 cars in the virtual fleet."

The trick, according to Stout, is to isolate what he called the "truly interesting" miles in that welter of information. And his group doesn't make it easy for the virtual vehicles, which in a system that Neo would appreciate can enter training rooms made from computer code before they've dispatched to a 'real' world to tackle myriad challenges.

Different universes, different worlds - and fuzz

Waymo

How a Google Car sees the world.

"We have different universes and worlds running," Stout said. "And we're testing small variations over and over while creating situations that the vehicles have never seen before."

This is where Carcraft moves beyond anything the Wachowskis dreamed up.

"Fuzzing comes into play," Stout explained.

The concept is esoteric, but without it, Waymo's actual-world combinations of laser-radars, sensor, and programming - what CEO John Krafcik calls a "driver" - can't replicate something the human brain has been managing for over a century: pilot a powerful machine with four wheels through a wide range of environments. (Including the Moon, where astronauts drove a lunar rover in the early 1970s.)

Stout said that he and his team will start with the skeleton of an idea - say, a four-way stop, something that's widespread and routinely dealt with by human drivers, but that can involve numerous vehicles and entails complex decision-making.

The many real-world inputs, drawn from what Waymo vehicles have experienced, can then be fuzzed and varied, multiplied, and then analyzed as edge cases. This results can then be fed back into Carcraft and applied to real-world Waymo testing, making the company's driver more robust and capable.

"I look at it like a scientist," Stout said. "The goal is to explore those possibilities."

Waymo

A fuzzing sample.

Humans are good at driving, but Waymo's driver have some serious advantages

There are some obvious advantages here for Waymo's self-driving objectives, which recently landed the company, spun-off from Google's parent Alphabet in 2016, a $175-billion valuation from Morgan Stanley. Waymo is moving toward a commercial rollout in 2018 and 2019, with an intense focus on safety and versatility.

Waymo's driver really is that: a sort of disembodied robot pilot and navigator that could be installed, according to Krafcik, in multiple transportation platforms, from passenger cars to semi trucks. Humans already know how to drive everything that can be driven. But we have a poor safety record: nearly 40,000 people die in vehicle-related accidents every year in the US alone.

Very few humans besides possibly professional race-car drivers are running 24/7 simulations in their brains of what fuzzy edge cases they might encounter while behind the wheel (and even race-car drivers have to sleep, when they might or might not dream of driving). Autonomous vehicles are among the toughest problems that computer scientists can take on, so Stout and his Waymo colleagues are using everything they know and amplifying it with massive amounts of computing power.

Waymo

How Waymo maps the world in simulation.

Beyond having Carcraft as its collective mega-mind, the Waymo automated driver never gets tired and is better designed to deal with the cognitive difficulties of moving a vehicle from point A to point B.

"The car is always paying attention," he said. "Human front-facing eyes evolved for hunting, but the self-driving car is looking 360 degrees all the time."

The Waymoers I've spoken with over the past year are some of the most thoughtful people I've ever encountered, and Stout was no exception. His interest in neuroscience began in high school, but when he discovered that computer science enabled quicker feedback, he made the shift halfway through his undergraduate studies at Brown University. (Stout is also involved with hands-free computing and speech-recognition, which he blogs about at handsfreecoding.org.)

At Google, he worked on search and Google Earth, leaning how to do what he called "very big stuff."

But after he spotted some early Google Cars - modified Toyota Priuses - zooming around parking lots at the Googleplex in Mountain View, the self-driving project became his dream.

"What made me want to join was that the people on the team were smart and charismatic, but also humble," he said.

The 20% solution

Waymo

What Waymo explores in Carcraft can then be applied to real-world testing.

He wasn't sure how to get hired, but he figured he had some ways to make himself useful. He concentrated on simulations, something that the Google Car team wasn't then regularly running. At first, it was an undertaking that consumed 20% of his time.

Google allows employees to work on these side projects with the understanding that they could yield major dividends, and for Stout his 20% efforts planted the first seed for Carcraft and led to him joining the team.

Fast-forward five years and Stout still eats some oatmeal for breakfast and might review a technical text to get his mental gears turning before heading into the office. Then he hits the ground running, as he describes it.

"The team is always working hard, and there are always interesting discussions going on," he said. But it never gets old.

"Overall it's super exciting."

And for Stout, the biggest and potentially most satisfying payoff is anything but virtual.

"I want my parents to be able to hop into a self-driving car," he said. "That's what we've all been waiting for."

Get the latest Google stock price here.