As a senior at the University of Minnesota, Jeff Dean built an artificial brain. Kinda. Using what was considered a supercomputer at the time, he mimicked the networks of neurons inside your head, creating a system that could analyze information—and even learn. The trouble was it didn’t work that well. Those computers didn’t provide enough juice. They couldn’t juggle enough data. “We just trained it on toy problems,” he says of this neural network. “The computational power wasn’t all that great.”
But this was 25 years ago, before Dean went to Google and changed the very nature of computational power.
As one of Google’s earliest engineers, Dean helped create the fundamental computing systems that underpin the company’s vast online empire, systems that span tens of thousands of machines. This work gave him celebrity-like status among Silicon Valley engineers—people recognize him as he walks through the Google cafeteria. Now, armed with those massively distributed systems and the ideas that drive them, he has returned to the world of neural networks. And this time, these artificial brains work remarkably well.
Together with a team of other big-name researchers, Dean is building neural nets that can identify faces in photos, recognize spoken words, and even understand natural language. And many other tech companies—from Microsoft to Facebook to Twitter—are creating services with similar capabilities.
The basic algorithms that drive these systems aren’t that different than what we had in the ’80s. But now, thanks to people like Dean, we have the computing power they need to thrive. “I’ve always believed that human learning is the result of relatively simple rules combined with massive amounts of hardware and massive amounts of data. And we now have that,” says Sebastian Thrun, the ex-Googler who oversaw the company’s self-driving car project, which can also benefit from the neural nets built by Dean and crew. “This is the chance for us to change the model of learning from very shallow, very confined statistics to something extremely open-ended.”
In the ’90s, Dean was a researcher at DEC, which made the big computing systems that ran the world’s businesses in those days. But as the DECs of the world were imploding, he moved to Google, and there he was among the small team who realized we could generate far more computing power by stringing together thousands of relatively small machines.
He helped create software that could store and process data across all these machines as if they were one big computer. With names like Bigtable and MapReduce, these tools were the secret weapons that enabled the company’s search engine to instantly serve hundreds of millions of people across the globe. Based on the research that Google later published, other companies like Facebook and Twitter and Yahoo began using similar tools. And now, drawing on many of the same ideas that allow such tools to juggle data across thousands of machines, people like Dean can finally construct neural networks that work.
They can reliably identify the voice commands you bark into Android phones—or recognize the faces in images you post to the Google+ social network. At Microsoft similar neural nets underpin a new Skype tool that instantly translates from one language to another. Baidu is using such AI technology to target ads on its search engine, something that has significantly boosted revenue.
In some ways the brainlike algorithms that drive these systems aren’t that different from those Dean played with as a college senior. The thing is, though, researchers now have a better understanding of how to build networks that operate on multiple levels, mimicking the multiple layers of neurons that operate in the brain. “It’s not a true reconstruction of what neurons do. But it’s an abstract notion of how we believe neurons work in the brain,” Dean says. “If you have lots and lots of these neurons and they’re all trained to pick up on different types of patterns, and there are other neurons that pick up on patterns that those neurons themselves have built on, you can build very complicated functions and systems that do pretty interesting things.”
Deep learning is the catchall term for this, and in the years to come it will remake far more than just Skype and other Internet products. Like Thrun, Dean says deep learning can improve our self-driving cars, helping them better understand the world around them. And if it can help cars learn, it can help other machines learn as well. In other words, sentient robots are probably coming down the road.
Check out the full Next List here.
