San Francisco has some of the country’s worst traffic. The lights always feel out of sync. The pavement is riddled with potholes. And pedestrians, cyclists, one-wheelers, and scooter-ers spill into the streets like the fog descending from the hills. It is, in all, a horrific place to drive. And for the same reasons, it’s a tremendous place to teach a car to drive itself. To borrow a phrase from a rival city, if your robot can make it here, it can make it anywhere.
That’s why Zoox, a much-hyped self-driving car startup based in Silicon Valley, does much of its testing in the Financial District and North Beach, two of the city’s most vexing neighborhoods. In a three-minute video shared exclusively with WIRED, we see the view from one of Zoox’s test cars, a Toyota Highlander SUV retrofitted with its sensors and computing systems, face down some of San Francisco’s gnarliest thoroughfares and San Franciscans’ most befuddling moves. The vehicle scoots around double-parked cars, makes left turns across traffic, and safely slides between hordes of pedestrians. It does it at night, in the rain, and on hills so steep, you can hardly see the intersection up ahead.
“We’re handling the spectrum of complicated situations you need to drive in in a city like San Francisco,” says Jesse Levinson, Zoox’s CTO. “We have built the software and hardware frameworks that can handle this.”
[#video: https://www.youtube.com/embed/868tExoVdQw&feature=youtu.be
This is a big deal. Where most robo-car developers, like Waymo or Uber, are partnering with traditional automakers like Jaguar Land Rover and Volvo to design their vehicles and companies like Lyft to put them into operation, Zoox intends to run the entire operation solo. The company will run its own ride-hailing service, using a vehicle it’s designing itself (the current iteration looks like a long wheelbase golf cart festooned with wires).
Levinson has been working on self-driving car technology since the mid-2000s. When he was a grad student at Stanford, he helped the university’s team take second place in Darpa’s 2007 Urban Challenge, a foundational event for the self-driving industry. In 2014, he co-founded Zoox with Tim Kentley-Klay, and has raised $800 million to date, according to Bloomberg, but the company only started driving in the city about a year ago. “We’ve been able to conquer some difficult terrain really quickly,” he says.
The Zoox team (about 500-strong, many pulled from places like Tesla, Nvidia, and NASA) has expanded the territory its cars cover and is steadily making its software more efficient, to reduce the computer-power any car needs to drive safely. They’ve applied machine learning to teach that software how to identify cyclists, pedestrians, and other actors, and predict how they’re likely to behave. They’ve worked to prioritize safety but value efficiency. “Part of driving well in a city is not being so conservative that you don’t move,” Levinson says. “People are just gonna honk at you, and you won’t have a product.”
The car’s victory over sundry obstacles—six-way intersections, GPS-blocking tunnels, highway on-ramps—indicates Zoox is well on its way to having that product. But, as with all self-driving tech demos, this video needs to be consumed alongside a few grains of salt. It shows about nine minutes worth of driving (shown at 3x speed), a tiny and select sample size. It doesn’t reveal how smoothly the car drives. One self-driving expert who reviewed the video (and asked not to be named discussing a competitor) noted the car does not seem to spot and classify every single pedestrian. Overall, though, the car looks good. “It’s on par with the state of the art,” says Matthew Johnson-Roberson, who co-directs the University of Michigan Ford Center for Autonomous Vehicles and watched the video.
“It is an impressive set of scenarios and the vehicle does exhibit a very sophisticated level of ‘driving skills’ in this video,” says Huei Peng, who studies autonomous vehicles at the University of Michigan. But there’s a big difference between handling a sticky spot once and handling it every single time, in every combination of all the variables. The real proof of a self-driving system doesn’t fit onto a highlight reel. It’s based on millions of miles and thousands of hours of driving.
Levinson says as much, and while he isn’t yet ready to give a broader look at how his system performs, numbers Zoox filed with the California DMV this winter show its system made major progress in how far it could drive between human safety driver takeovers in the second half of the year. That needs to be proved over time, Levinson says, before Zoox will feel it’s ready to launch its service. “You can’t have service if you accidentally hit something once in a while.” (Based on public reports logging autonomous vehicle incidents in California, Zoox’s cars have never caused a crash.)
But one thing is clear from the video. San Francisco is a wild place to learn to drive. And if you can master its mean, meandering streets, you’re ready to be a pro driver.
- The ultimate carbon-saving tip? Travel by cargo ship
- Laser-shooting planes uncover the horror of WWI
- The Pentagon's dream team of tech-savvy soldiers
- PHOTO ESSAY: The annual super-celebration in Superman's real-world home
- It’s time you learned about quantum computing
- Get even more of our inside scoops with our weekly Backchannel newsletter
