"Every purchase you make with a credit card, every magazine subscription you buy and medical prescription you fill, every Web site you visit and e-mail you send or receive, every academic grade you receive, every bank deposit you make, every trip you book and every event you attend—all these transactions and communications will go into … a virtual, centralized grand database,” the New York Times columnist warns.
On the heels of Mark Zuckerberg’s numerous government testimonies and sustained criticism over the Cambridge Analytica scandal, the author of this Times column must be talking about Facebook—right? Or perhaps the web’s broader, ad-based business model?
Not so: The William Safire column, "You Are a Suspect,” was published in the Times in 2002—two years before Facebook was created. And Safire isn’t talking about social networks or digital advertising—he’s discussing Total Information Awareness, a US Defense Advanced Research Projects Agency (Darpa) program that proposed mining vast amounts of Americans’ data to identify potential national security threats. The virtual grand database was to belong to the Department of Defense, which would use it to identify behavior patterns that would help to predict emerging terrorist threats.
Today, we’re voluntarily participating in the dystopian scenario Safire envisioned 16 years ago, with each bit of data handed to companies like Facebook and Google. But in this system, private companies are our information repositories—leaving us to reckon with the consequences of a world that endows corporations with the kind of data once deemed too outrageous for the government.
The Total Information Awareness project, run by Admiral John Poindexter, was a meta-program. It was designed to aggregate signals generated via other programs run out of Darpa’s Information Awareness Office. The programs focused on a range of capabilities, including information analysis, decision-support tools, language translation, data mining, and pattern recognition. When the component parts were combined, they would form a comprehensive picture of people and their behaviors. The purpose was to detect signals that could be used to identify terrorist behavior and head off attacks; the inspiration was the fact that the government had failed to connect the dots left by the 9/11 terrorists as they planned their attack.
Concern about the program was bipartisan and widespread. The Cato Institute warned of the potential for a surveillance society and raised Fourth Amendment concerns. The ACLU called it a virtual dragnet that would require the government “to collect as much information as it can about everyone—and these days, that is a LOT of information … Not only government records of all kinds but individuals' medical and financial records, political beliefs, travel history, prescriptions, buying habits, communications (phone calls, emails and web surfing), school records, personal and family associations, and so on.” The US Senate, led by senators Ron Wyden and Byron Dorgan, voted unanimously to defund the program shortly after it was announced; some of the technological underpinnings were reshuffled, sent to other parts of the government that weren’t focused on the activities of US citizens.
But as Total Information Awareness was being disassembled in Washington, DC, a similar system emerged, and began to gather momentum, in Silicon Valley. Within a few years, top industry trend reports and VC blog posts began to talk up the power (and economic promise) of “Big Data” and “Social Mobile Local.”
Today, you can probably name several companies that have access to data on every purchase you make with a credit card, every magazine subscription you buy, every website you visit and email you send or receive, every trip you book, and every event you attend—all the various types of data that Safire referenced, but gathered to identify behavior patterns that help predict what ads you’ll click on.
In the private sector, startups and large companies alike began to tout how well they could gather, store, and mine data; it was a popular business model. The fact that it was happening was hardly a secret—personalized ads within Gmail, universal logins, and retargeted ads following us from site-to-site were readily apparent to even minimally savvy users. The biggest tech companies in the world succeeded because they built products users loved—their users voluntarily opted in to giving up their information and behavioral data. Widespread tracking and data aggregation became the norm, as universal logins, tracking pixels, forays into linked products (Gmail, for example), and acquisitions of startups enabled the tech platforms to build comprehensive profiles of users with ease.
At a Congressional hearing last month, Senator Maria Cantwell broached the similarity between TIA and private information-gathering in an exchange with Facebook CEO Mark Zuckerberg:
“Have you heard of Total Information Awareness?” Senator Cantwell asked. “Do you know what I’m talking about?”
“No, I do not,” Zuckerberg replied.
It wasn’t unreasonable for Zuckerberg to have never heard of the program; after all, it was proposed when he was in high school. Cantrell went on to explain the initiative: data mining on a vast scale, with the potential for unprecedented surveillance, control, and identification. She brought up WhatsApp and Palantir as other examples of private-company data harvesters, which she called a “major trend in an information age.” Her line of inquiry, drawing parallels between several companies and TIA, raises important questions: How are private-sector entities amassing a level of power that Americans denied to Darpa? What are the consequences of this tool existing outside the bounds of public oversight? And where do we go from here?
There are, of course, a few critical differences between Total Information Awareness and the platforms run by social media giants. First, users voluntarily opt in to the companies' terms of service. People ostensibly know what they’re giving up in exchange for free email, messaging, and ways to share pictures and connect with friends. Second, the stakes are very different: The government has the power to arrest you; Facebook and Google don’t.
Total Information Awareness was nakedly about surveillance and how it could be used both defensively and offensively. Facebook, on the front end, is about likes and photos and status updates and community. Google is a great search engine and popular email provider. But the backends of our private companies are eerily similar to TIA, and the same threats exist—but without any offsetting public oversight. This is dangerous. Now multiple governments, not just our own, can tap into a vast data trove to predict our behavior and target us. In fact, as we’ve come to learn, other nations have already misused it, such as Russia, which used precision targeting and copious quantities of data to reach American citizens.
The private platforms that amassed the data collection and mining mechanisms necessary for total information awareness are ironically under no obligation to use it for the original national security and counterterrorism purposes that inspired (and purportedly justified) TIA—and many digital civil liberties organizations don’t believe they should. But more disturbingly, it seems that our social platforms may in fact be doing the opposite: They may be inadvertently radicalizing extremists. It seems increasingly possible that there is something to the idea that social signals can identify early indications of radicalization, or pick out those who are likely to be receptive or sympathetic. In response, our recommendation engine and search functions may well be inadvertently using those signals to further facilitate the process—the exact opposite of detection and intervention.
So where do we go from here? How do we address the profoundly powerful tools now in private hands? First, we need civic groups to scrutinize powerful companies the same way they scrutinize government. Some of the groups most vocally critical of Total Information Awareness—the Electronic Frontier Foundation, the Open Technology Institute—have been largely silent about Facebook's and Google’s amassing of similar power. Slate's April Glaser recently wrote an excellent piece examining the nuances and politics at play here, from financial conflicts of interest to worldviews. As she put it, we need the watchdogs to bark.
But civil society alone can’t win this fight. We need scrutiny from everyday internet users too. As a country, we were extremely bothered by the government aggregating databases and mining for certain behavioral signatures that might reveal extremist leanings. The idea that there could be effective oversight of such an invasive program was dismissed as absurd. But just a few years later, we were remarkably quick to give away the contents of those same databases to large corporations. We don’t trust our government—too many scandals, too much abuse—but we haven’t fully reckoned with the fact that we’re turning our personal information over to powerful private interests with no oversight.
- A new look inside Theranos' dysfunctional corporate culture
- Ketamine offers hope—and stirs up controversy—as a depression drug
- PHOTO ESSAY: Unreal views of the trippy colors in Ethiopia's Danakil Desert
- Nyan Cat, Doge, and the art of the Rickroll—here's all you need to know about memes
- Seakeeper’s super spinning system keeps ships stable at sea
- Hungry for even more deep dives on your next favorite topic? Sign up for the Backchannel newsletter
