Amazon has become the latest tech giant that's giving away some of its most sophisticated technology. Today the company unveiled DSSTNE (pronounced "destiny"), an open source artificial intelligence framework that the company developed to power its product recommendation system. Now any company, researcher, or curious tinkerer can use it for their own AI applications.
It's the latest in series of projects recently open sourced by large tech companies all focused on a branch of AI called deep learning. Google, Facebook, and Microsoft have mainly used these systems for tasks like image and speech recognition. But given Amazon's core business, it's not surprising that the online retailer's version is devoted to selling merchandise.
"We are releasing DSSTNE as open source software so that the promise of deep learning can extend beyond speech and language understanding and object recognition to other areas such as search and recommendations," the Q&A section of Amazon's DSSTNE GitHub page reads. "We hope that researchers around the world can collaborate to improve it. But more importantly, we hope that it spurs innovation in many more areas."
Along with the idealistic rhetoric, open sourcing AI software is a way for tech industry rivals to show off and one-up each other. When Google released its TensorFlow framework last year, it didn't offer support for running the software across multiple servers at the same time. That meant users couldn't speed up their AI computations by stringing together clusters of computers the same way Google could running a more advanced version of the system internally.
That created an opening for other software companies like Microsoft and Yahoo to release their own open source deep learning frameworks that support distributed computing clusters.
Google has since caught up, releasing a version of TensorFlow that supports clusters earlier this year. Amazon claims its system takes distribution one step further by enabling users to to spread a deep learning problem not just across multiple servers, but across multiple processors within each server.
Amazon also says DSSTNE is designed to work with sparser data sets than TensorFlow and other deep learning frameworks. Google uses TensorFlow internally for tasks such as image recognition, where it can rely on the Internet's vast store of, say, cat photos to train its AI to recognize images of cats. Amazon's scenarios are quite different. The company does sells millions of different products. But the number of examples of how the purchase of one product relates to the purchase of another are relatively few by comparison to cats on the Internet. To make compelling recommendations—that is, to recommend products that customers are more likely to click on and buy—Amazon has a strong incentive to create a system that can make good predictions based on less data. By open sourcing DSSTNE, Amazon is increasing the likelihood that some smart person somewhere outside the company will help the company think of ways to make the system better.
