—
If Tony Tether wanted to understand why DARPA’s support for a centralized database might touch off a furor over privacy, he need not have looked far. In June 1975, more than twenty-five years before the Total Information Awareness imbroglio, a series of sensational news reports warned of a new Orwellian-sounding computer technology that would be used to create dossiers on individual Americans. “What this technology means to you is this: The Federal Government now has the means to put together a computer file on you, or almost any American, within a matter of minutes,” Ford Rowan, a correspondent for NBC Nightly News, reported. “The key breakthrough in the new computer technology was made by a little known unit of the Defense Department—the Advanced Research Projects Agency, ARPA.”
The technology involved was the ARPANET. Various news reports claimed that the government was using the ARPANET to create centralized dossiers through a secret network that linked the White House, the CIA, the Defense Department, the FBI, and the Treasury Department. None of that was true. Although government agencies were beginning to use elements of computer-networking technology, the ARPANET in 1975 was primarily linking academic institutions. But the reports came on the heels of several years of post–Vietnam War debate about surveillance abuses of the intelligence agencies, and the spread of computers and national data banks, which led to the Privacy Act of 1974. Almost three decades later, these same concerns would be played out following the 9/11 attacks, and it was a DARPA-sponsored computer scientist who first identified them.
On October 12, 2001, just one month after the attacks, the members of DARPA’s Information Science and Technology study group, or ISAT, met to conduct their annual brainstorming session. The atmosphere at the meeting had darkened considerably from previous years. The group’s chairman came into the room and cued up a slide. On the overhead screen flashed Osama bin Laden’s 1998 fatwa against Jews and crusaders. “I don’t want to affect anyone’s thinking, but here’s what we’re up against this year,” he told the assembled scientists.
ISAT was established in the 1980s, during the days of the Strategic Computing Initiative, as an advisory group for DARPA focused specifically on computer science. Unlike the JASONs, the elite group of scientists who had helped create the McNamara Line in Vietnam, ISAT did not operate as an independent group; it advised only DARPA. One of the ISAT members, Eric Horvitz, had already been thinking about how to use computers to help sift through large amounts of data to predict future events. But that year, Horvitz, a prominent artificial intelligence expert working at Microsoft, saw an opportunity to apply this work to the nexus of data mining and privacy. His vision was for something called selective revelation.
Under Horvitz’s scheme, the government would collect data—be it intelligence, law enforcement, or commercial information—and put it into a centralized database. Humans would not have any direct access to it. Instead, computer algorithms would sift through the personal data, looking for patterns that might indicate a possible terrorist attack. When such a pattern was spotted, the government, in Horvitz’s thinking, would obtain a search warrant that would allow for “selective revelation” of personal data. “The idea basically was, how could you minimize the revelation of personal data while supporting analysis that was deemed important to find needles in haystacks?” he said.
The data-mining system operated like a locked black box. Inside, “you can basically have standing queries of interest with automated computer agents walking over rafts of data,” Horvitz said. “You collect data but no one is allowed to look at this data except computer programs, and whenever you find some troubling or concerning things, then the system alerts an operator and it says, ‘I’ve found something which could be a problem.’ ”
Inevitably, a human would have to review the results, but the system would monitor and log every time someone peeked inside. Horvitz described his concept as a system that “watches the watchers,” meaning that those with access to the database would themselves be monitored through random audits. It would function, he believed, like a “hall of mirrors,” where anyone is potentially watching everyone else. Horvitz’s idea for a “hall of mirrors” soon made its way to the desk of John Poindexter, who had just moved into his office at DARPA. Poindexter wanted DARPA to sponsor privacy research as part of Total Information Awareness, to see what safeguards could be incorporated into a data-mining system, and Horvitz’s proposal was to study precisely that.
Horvitz’s concept was also very close to how Poindexter saw the database searches working for Total Information Awareness. In Poindexter’s vision, the computer algorithms would be focused not necessarily on a specific terrorist event—those were too rare to be anticipated—but on patterns of activities that might indicate that terrorists were preparing an attack. Rather than going to the Foreign Intelligence Surveillance Court, or some other judicial authority to seek data on a specific person, law enforcement or an intelligence agency could seek authorization to search for a specific pattern. As Poindexter described it: “If your search for that pattern comes back with 100,000 hits, it’s not discriminating enough. You refine the pattern; you get that approved [by the judicial authority]. Maybe with that version you get ten hits. At that point, you go back to the court, through an automated system, and say, ‘Okay, we got, ten hits. Now we want permission to find out the details of those ten hits.’ ”
Because Poindexter believed that having a privacy mechanism would be important for automated searches, he offered to fund the ISAT proposal. He even attended the summer study meeting in 2002 held at the offices of the Institute for Defense Analyses in Alexandria, Virginia. ISAT invited two privacy advocates to observe the meeting, including Marc Rotenberg, the head of the Electronic Privacy Information Center. Poindexter and Rotenberg had clashed previously in the 1980s, when Poindexter was in the White House and Rotenberg worked as a congressional staffer. Rotenberg, then a counsel to the Democratic Vermont senator Patrick Leahy, had opposed a White House National Security Decision Directive, which had offered NSA assistance to private companies on cyber security. Opponents on the Hill, including Rotenberg, saw it as an NSA intrusion. “It was a Big Brother is going to watch you, and all this crap,” Poindexter said.
At the ISAT meeting in 2002, Rotenberg sat quietly, and when Poindexter approached him during the break, the conversation was civil. Poindexter thanked him for coming and said DARPA genuinely hoped to gather ideas on balancing security and privacy. According to Poindexter, Rotenberg said he understood, adding that he believed there needed to be more oversight. Poindexter took that as an encouraging sign. He was wrong, however.
Rotenberg saw the meeting from a completely different perspective. In his view, Poindexter and others involved in the project had no understanding of what privacy meant. Officials like Poindexter believed that internal audit mechanisms—the members of the surveillance state monitoring its own—would protect privacy. Rotenberg argued that the Privacy Act of 1974 was rooted in the public’s right to have control over the data collected about them and not just how that data is used. His concern about the DARPA program only deepened when he participated in another workshop related to Total Information Awareness. This workshop, held at Stanford, looked at a proposed DARPA program to create “eDNA,” which would make it possible to track every keystroke on the Internet back to a specific user. “Perfect surveillance” is how Rotenberg described it. “And completely crazy.”
Despite Rotenberg’s privacy concerns, DARPA’s plans were moving forward. DARPA earlier that year issued an open call for proposals in areas of interest to the Information Awareness Office, including what Poindexter called the “privacy protection appliance.” And with the floodgates open after 9/11 for counterterrorism spending, Genoa’s budget alone was set to more than double, from $70 million in 2002 to about $150 million for 2003. Over the first half of 2002, Poindexter worked to put together the initial Total Information Awareness prototype, reaching out to other defense and intelligence agencie
s, which were invited to establish “nodes” on the data network. The central node would be controlled by DARPA, but the distributed nodes would allow the different agencies to access the network and test out the program’s tools. The NSA, not surprisingly, had the most nodes of any agency.
By the late summer of 2002, the pieces were falling into place, although the picture from the outside probably looked different from how those inside saw it. The Information Awareness Office had as its head the bête noire of Iran-contra, an office seal that featured an icon of Illuminati-inspired conspiracy theories, and an ambitious vision for a centralized database. The new office was ready for its public debut, and for that it headed to Disneyland.
—
DARPATech, the semiannual DARPA conference, had once been a staid technical meeting held in cities like Denver and Dallas, but Tony Tether wanted to make a public splash, so in 2002 he moved the venue to Disneyland in California. “We have had a lot of fun creating this symposium,” Tether said in the keynote speech. “Part of the fun was each office finding a Disneyland-like theme for what it does.” (The theme for the Information Awareness Office featured the Star Wars androids, C-3PO and R2-D2.)
Just as Tether had not seen anything wrong with proposing that Poindexter go to credit card companies to collect financial data, he did not see a problem with allowing Poindexter to roll out the Information Awareness Office in a fantasy world populated by life-size versions of Goofy and Mickey Mouse. And so, in August 2002, in front of an audience of DARPA-supported researchers and a handful of journalists, Poindexter introduced the Information Awareness Office:
One of the significant new data sources that needs to be mined to discover and track terrorists is the transaction space. If terrorist organizations are going to plan and execute attacks against the United States, their people must engage in transactions and they will leave signatures in this information space. This is a list of transaction categories, and it is meant to be inclusive. Currently, terrorists are able to move freely throughout the world, to hide when necessary, to find sponsorship and support, and to operate in small, independent cells, and to strike infrequently, exploiting weapons of mass effects and media response to influence governments. We are painfully aware of some of the tactics that they employ. This low-intensity/low-density form of warfare has an information signature. We must be able to pick this signal out of the noise. Certain agencies and apologists talk about connecting the dots, but one of the problems is to know which dots to connect. The relevant information extracted from this data must be made available in large-scale repositories with enhanced semantic content for easy analysis to accomplish this task. The transactional data will supplement our more conventional intelligence collection.
Poindexter’s speech acknowledged potential concerns about privacy but promised to address them. “There are ways in which technology can help preserve rights and protect people’s privacy while helping to make us all safer,” he assured the audience.
DARPATech came and went, generating only modest press coverage, mostly focused on Tether’s announcement that DARPA would sponsor a “robot race” in the California desert, a demonstration of self-driving cars. Nobody seemed to take note of Poindexter’s speech, which did not necessarily introduce anything new. More than two years earlier, Brian Sharkey had talked about “Total Information Awareness” at a prior DARPATech meeting, though it was at the time a concept more than a program. As for Poindexter, his presence at a small defense conference hardly seemed newsworthy, particularly for the mostly science- and tech-focused members of the press who attended.
Yet Rotenberg, the privacy advocate, had been discussing his concerns with the New York Times technology reporter John Markoff. In November, Markoff’s article described Total Information Awareness as “a vast electronic dragnet, searching for personal information as part of the hunt for terrorists around the globe—including the United States.” The article quoted Poindexter’s Disneyland speech, as well as Rotenberg. The DARPA program was described as building a “system of national surveillance of the American public.”
Even that article did not garner much attention. The next week, the New York Times columnist William Safire declared war on Total Information Awareness, calling it an affront to the American way of life. “Every purchase you make with a credit card, every magazine subscription you buy and medical prescription you fill, every Web site you visit and e-mail you send or receive, every academic grade you receive, every bank deposit you make, every trip you book and every event you attend—all these transactions and communications will go into what the Defense Department describes as ‘a virtual, centralized grand database,’ ” he wrote. “To this computerized dossier on your private life from commercial sources, add every piece of information that government has about you—passport application, driver’s license and bridge toll records, judicial and divorce records, complaints from nosy neighbors to the F.B.I., your lifetime paper trail plus the latest hidden camera surveillance—and you have the supersnoop’s dream: a ‘Total Information Awareness’ about every U.S. citizen.”
Safire’s column was a conflation of fact and fantasy: Total Information Awareness was a research project using make-believe data, and there were plenty of legal reasons why some of that data might be excluded even assuming the technology was eventually adopted. On the other hand, Safire’s description was a reasonable portrayal of the scope of Poindexter’s ambition. The column touched off the classic firestorm amplified by the Washington echo chamber: an avalanche of articles appeared, many quoting the Safire piece. The agency’s spokesperson told Tether just to ignore it and the furor would eventually die down. DARPA officials steadfastly refused to comment on Poindexter’s work or the programs. As more articles tumbled forth, Tether was shocked, a reflection of how oblivious he was to the potential public perception of DARPA’s work. Total Information Awareness was, in his view, just a research program. Yet the articles were treating the project as if it were an operational system collecting everybody’s medical records. “I am reading these things and I know what is going on, but I start to think, holy cow, maybe I don’t know what is going on! But we didn’t respond,” he later told a group of reporters. “Where we screwed up is that we didn’t respond. We literally did not take an active stance of coming out and saying, ‘Hey, bullshit,’ you know, and went around and made sure that everybody understood what we are doing until it was almost too late.”
It was not just the press, however. Congress began to demand briefings, and that created a whole new problem. Poindexter, who had been convicted of lying to Congress, was not a good choice to send to the Hill to answer questions coming from incensed lawmakers. Poindexter suggested that his deputy, Bob Popp, do the briefings, but Tether insisted on speaking to lawmakers himself. It was a disaster. “Tony didn’t understand the programs well enough to explain them in detail,” Poindexter lamented. “It wound up looking like we were much more secretive than we wanted to be.”
As the critics circled Poindexter, the controversy surrounding Total Information Awareness grew, yet Tether refused to budge an inch. After all, DARPA had top cover from the defense secretary and the vice president. For a while, the Pentagon’s leadership backed DARPA, too. But as the articles proliferated, Congress demanded that the Pentagon provide a full report on the Information Awareness Office and all its projects. When congressional staffers combed through the report, one project in particular caught their attention: a small research study for something called the FutureMAP, which looked at the potential of using the “wisdom of crowds,” represented by free-market investors, to predict future political events. FutureMAP actually had its origins with a researcher from the National Science Foundation, who was interested in the predictive capabilities of open markets. DARPA had awarded contracts for very preliminary work, which would fund researchers to test whether having people wager real money on future political events would make for accurate predictions.
Net Exchange, one of the companies in
volved, created a website that featured “colorful examples,” like “the assassination of Arafat, and a missile attack from North Korea.” It was an early effort at crowdsourcing predictions, something that would be commonplace ten years later. For Congress, which was eager to come up with any excuse to get rid of Poindexter and his programs, FutureMAP was enough. “The idea of a federal betting parlor on atrocities and terrorism is ridiculous and it’s grotesque,” Senator Ron Wyden, a Democrat from Oregon, said.
The FutureMAP episode was almost a verbatim repeat of the opposition to AGILE toward the end of the Vietnam War, when members of Congress trawled DARPA reports in the hopes of finding something embarrassing. “The congressmen and senators that had been critics of the program all along became incensed that—as they described it—that I wanted to establish a betting parlor,” Poindexter said. “At that point, I told Tony it was time for me to leave, and I did.”
Poindexter’s resignation letter to Tether in August 2003 ran five single-spaced pages, expounding on DARPA’s origins, its purpose, and its achievements. He reviewed the history of the Information Awareness Office, providing a detailed and unapologetic defense of its activities, and blasted the “charged political environment of Washington, where glib phrases, ‘sound bites,’ and symbols” are used instead of debate. He expressed hope that Congress might salvage some of his office’s work.
The Imagineers of War Page 36