Data and Goliath

by Bruce Schneier

  This is not a new problem, nor one limited to the NSA. Recent US history illustrates many episodes in which surveillance has been systematically abused: against labor organizers and suspected Communists after World War I, against civil rights leaders, and against Vietnam War protesters. The specifics aren’t pretty, but it’s worth giving a couple of them.

  • Through extensive surveillance, J. Edgar Hoover learned of Martin Luther King’s extramarital affairs, and in an anonymous letter he tried to induce him to commit suicide in 1964: “King, look into your heart. You know you are a complete fraud and a great liability to all of us Negroes. White people in this country have enough frauds of their own but I am sure they don’t have one at this time anywhere near your equal. You are no clergyman and you know it. I repeat you are a colossal fraud and an evil, vicious one at that. You could not believe in God. . . . Clearly you don’t believe in any personal moral principles. . . . King, there is only one thing left for you to do. You know what it is. You have just 34 days in which to do it (this exact number has been selected for a specific reason, it has definite practical significance). You are done. There is but one way out for you. You better take it before your filthy, abnormal fraudulent self is bared to the nation.”

  • This is how a Senate investigation described the FBI’s COINTELPRO surveillance program in 1976: “While the declared purposes of these programs were to protect the ‘national security’ or prevent violence, Bureau witnesses admit that many of the targets were nonviolent and most had no connections with a foreign power. Indeed, nonviolent organizations and individuals were targeted because the Bureau believed they represented a ‘potential’ for violence—and nonviolent citizens who were against the war in Vietnam were targeted because they gave ‘aid and comfort’ to violent demonstrators by lending respectability to their cause. . . . But COINTELPRO was more than simply violating the law or the Constitution. In COINTELPRO the Bureau secretly took the law into its own hands, going beyond the collection of intelligence and beyond its law enforcement function to act outside the legal process altogether and to covertly disrupt, discredit and harass groups and individuals.”

  Nothing has changed. Since 9/11, the US has spied on the Occupy movement, pro- and anti-abortion activists, peace activists, and other political protesters.

  • The NSA and FBI spied on many prominent Muslim Americans who had nothing to do with terrorism, including Faisal Gill, a longtime Republican Party operative and onetime candidate for public office who held a top-secret security clearance and served in the Department of Homeland Security under President George W. Bush; Asim Ghafoor, a prominent attorney who has represented clients in terrorism-related cases; Hooshang Amirahmadi, an Iranian American professor of international relations at Rutgers University; and Nihad Awad, the executive director of the largest Muslim civil rights organization in the country.

  • The New York Police Department went undercover into minority neighborhoods. It monitored mosques, infiltrated student and political groups, and spied on entire communities. Again, people were targeted because of their ethnicity, not because of any accusations of crimes or evidence of wrongdoing. Many of these operations were conducted with the help of the CIA, which is prohibited by law from spying on Americans.

  There’s plenty more. Boston’s fusion center spied on Veterans for Peace, the women’s antiwar organization Code Pink, and the Occupy movement. In 2013, the city teamed with IBM to deploy a video surveillance system at a music festival. During the same time period, the Pentagon’s Counterintelligence Field Activity spied on all sorts of innocent American civilians—something the Department of Defense is prohibited by law from doing.

  Echoing Hoover’s attempt to intimidate King, the NSA has been collecting data on the porn-viewing habits of Muslim “radicalizers”—not terrorists, but those who through political speech might radicalize others—with the idea of blackmailing them.

  In 2010, DEA agents searched an Albany woman’s cell phone—with permission—but then saved the intimate photos they found to create a fake Facebook page for her. When they were sued for this abuse, the government speciously argued that by consenting to the search of her phone, the woman had implicitly consented to identity theft.

  Local authorities abuse surveillance capabilities, too. In 2009, the Lower Merion School District, near Philadelphia, lent high schoolers laptops to help them with their homework. School administrators installed spyware on the computers, then recorded students’ chat logs, monitored the websites they visited, and—this is the creepiest—surreptitiously photographed them, often in their bedrooms. This all came to light when an assistant principal confronted student Blake Robbins with pictures of him popping pills like candy. Turns out they were candy—Mike and Ike, to be exact—and the school was successfully sued for its invasive practices.

  Aside from such obvious abuses of power, there’s the inevitable expansion of power that accompanies the expansion of any large and powerful bureaucratic system: mission creep. For example, after 9/11, the CIA and the Treasury Department joined forces to gather data on Americans’ financial transactions, with the idea that they could detect the funding of future terrorist groups. This turned out to be a dead end, but the expanded surveillance netted a few money launderers. So it continues.

  In the US, surveillance is being used more often, in more cases, against more offenses, than ever before. Surveillance powers justified in the PATRIOT Act as being essential in the fight against terrorism, like “sneak and peek” search warrants, are far more commonly used in non-terrorism investigations, such as searches for drugs. In 2011, the NSA was given authority to conduct surveillance against drug smugglers in addition to its traditional national security concerns. DEA staff were instructed to lie in court to conceal that the NSA passed data to the agency.

  The NSA’s term is “parallel construction.” The agency receiving the NSA information must invent some other way of getting at it, one that is admissible in court. The FBI probably got the evidence needed to arrest the hacker Ross Ulbricht, aka Dread Pirate Roberts, who ran the anonymous Silk Road website where people could buy drugs and more, in this way.

  Mission creep is also happening in the UK, where surveillance intended to nab terrorists is being used against political protesters, and in all sorts of minor criminal cases: against people who violate a smoking ban, falsify their address, and fail to clean up after their dogs. The country has a lot of cameras, so it “makes sense” to use them as much as possible.

  Other countries provide many more examples. Israel, for instance, gathers intelligence on innocent Palestinians for political persecution. Building the technical means for a surveillance state makes it easy for people and organizations to slip over the line into abuse. Of course, less savory governments abuse surveillance as a matter of course—with no legal protections for their citizens.

  All of this matters, even if you happen to trust the government currently in power. A system that is overwhelmingly powerful relies on everyone in power to act perfectly—so much has to go right to prevent meaningful abuse. There are always going to be bad apples—the question is how much harm they are allowed and empowered to do and how much they corrupt the rest of the barrel. Our controls need to work not only when the party we approve of leads the government but also when the party we disapprove of does.

  CURTAILING INTERNET FREEDOM

  In 2010, then secretary of state Hillary Clinton gave a speech declaring Internet freedom a major US foreign policy goal. To this end, the US State Department funds and supports a variety of programs worldwide, working to counter censorship, promote encryption, and enable anonymity, all designed “to ensure that any child, born anywhere in the world, has access to the global Internet as an open platform on which to innovate, learn, organize, and express herself free from undue interference or censorship.” This agenda has been torpedoed by the awkward realization that the US and other democratic governments conducted the same types of surveillance they have criticized i
n more repressive countries.

  Those repressive countries are seizing on the opportunity, pointing to US surveillance as a justification for their own more draconian Internet policies: more surveillance, more censorship, and a more isolationist Internet that gives individual countries more control over what their citizens see and say. For example, one of the defenses the government of Egypt offered for its plans to monitor social media was that “the US listens in to phone calls, and supervises anyone who could threaten its national security.” Indians are worried that their government will cite the US’s actions to justify surveillance in that country. Both China and Russia publicly called out US hypocrisy.

  This affects Internet freedom worldwide. Historically, Internet governance—what little there was—was largely left to the United States, because everyone more or less believed that we were working for the security of the Internet instead of against it. But now that the US has lost much of its credibility, Internet governance is in turmoil. Many of the regulatory bodies that influence the Internet are trying to figure out what sort of leadership model to adopt. Older international standards organizations like the International Telecommunications Union are trying to increase their influence in Internet governance and develop a more nationalist set of rules.

  This is the cyber sovereignty movement, and it threatens to fundamentally fragment the Internet. It’s not new, but it has been given an enormous boost from the revelations of NSA spying. Countries like Russia, China, and Saudi Arabia are pushing for much more autonomous control over the portions of the Internet within their borders.

  That, in short, would be a disaster. The Internet is fundamentally a global platform. While countries continue to censor and control, today people in repressive regimes can still read information from and exchange ideas with the rest of the world. Internet freedom is a human rights issue, and one that the US should support.

  Facebook’s Mark Zuckerberg publicly took the Obama administration to task on this, writing, “The US government should be the champion for the Internet, not a threat.” He’s right.

  8

  Commercial Fairness and Equality

  Accretive Health is a debt collection agency that worked for a number of hospitals in Minnesota. It was in charge of billing and collection for those hospitals, but it also coordinated scheduling, admissions, care plans, and duration of hospital stays. If this sounds like a potential conflict of interest, it was. The agency collected extensive patient data and used it for its own purposes, without disclosing to patients the nature of its involvement in their healthcare. It used information about patient debts when scheduling treatment and harassed patients for money in emergency rooms. The company denied all wrongdoing, but in 2012 settled a Minnesota lawsuit by agreeing not to operate in Minnesota for two to six years. On the one hand, the fact that Accretive was caught and punished shows that the system is working. On the other hand, it also shows how easy it is for our data to be mishandled and misused.

  Stories like this demonstrate the considerable risk to society in allowing corporations to conduct mass surveillance. It’s their surveillance that contributes to all of the offenses against civil liberties, social progress, and freedom that I described in the previous chapter. And in addition to enabling government surveillance, corporate surveillance carries its own risks.

  SURVEILLANCE-BASED DISCRIMINATION

  In a fundamental way, companies use surveillance data to discriminate. They place people into different categories and market goods and services to them differently on the basis of those categories.

  “Redlining” is a term from the 1960s to describe a practice that’s much older: banks discriminating against members of minority groups when they tried to purchase homes. Banks would not approve mortgages in minority neighborhoods—they would draw a red line on their maps delineating those zones. Or they would issue mortgages to minorities only if they were buying houses in predominantly minority neighborhoods. It’s illegal, of course, but for a long time banks got away with it. More generally, redlining is the practice of denying or charging more for services by using neighborhood as a proxy for race—and it’s much easier to do on the Internet.

  In 2000, Wells Fargo bank created a website to promote its home mortgages. The site featured a “community calculator” to help potential buyers search for neighborhoods. The calculator collected the current ZIP code of the potential customers and steered them to neighborhoods based on the predominant race of that ZIP code. The site referred white residents to white neighborhoods, and black residents to black neighborhoods.

  This practice is called weblining, and it has the potential to be much more pervasive and much more discriminatory than traditional redlining. Because corporations collect so much data about us and can compile such detailed profiles, they can influence us in many different ways. A 2014 White House report on big data concluded, “. . . big data analytics have the potential to eclipse longstanding civil rights protections in how personal information is used in housing, credit, employment, health, education, and the marketplace.” I think the report understated the risk.

  Price discrimination is also a big deal these days. It’s not discrimination in the same classic racial or gender sense as weblining; it’s companies charging different people different prices to realize as much profit as possible. We’re most familiar with this concept with respect to airline tickets. Prices change all the time, and depend on factors like how far in advance we purchase, what days we’re traveling, and how full the flight is. The airline’s goal is to sell tickets to vacationers at the bargain prices they’re willing to pay, while at the same time extracting from business travelers the much higher amounts that they’re willing to pay. There is nothing nefarious about the practice; it’s just a way of maximizing revenues and profits. Even so, price discrimination can be very unpopular. Raising the price of snow shovels after a snowstorm, for example, is considered price-gouging. This is why it is often cloaked in things like special offers, coupons, or rebates.

  Some types of price discrimination are illegal. For example, a restaurant cannot charge different prices depending on the gender or race of the customer. But it can charge different prices based on time of day, which is why you see lunch and dinner menus with the same items and different prices. Offering senior discounts and special children’s menus is legal price discrimination. Uber’s surge pricing is also legal.

  In many industries, the options you’re offered, the price you pay, and the service you receive depend on information about you: bank loans, auto insurance, credit cards, and so on. Internet surveillance facilitates a fine-tuning of this practice. Online merchants already show you different prices and options based on your history and what they know about you. Depending on who you are, you might see a picture of a red convertible or a picture of a minivan in online car ads, and be offered different options for financing and discounting when you visit dealer websites. According to a 2010 Wall Street Journal article, the price you pay on the Staples website depends on where you are located, and how close a competitor’s store is to you. The article states that other companies, like Rosetta Stone and Home Depot, are also adjusting prices on the basis of information about the individual user.

  More broadly, we all have a customer score. Data brokers assign it to us. It’s like a credit score, but it’s not a single number, and it’s focused on what you buy, based on things like purchasing data from retailers, personal financial information, survey data, warranty card registrations, social media interactions, loyalty card data, public records, website interactions, charity donor lists, online and offline subscriptions, and health and fitness information. All of this is used to determine what ads and offers you see when you browse the Internet.

  In 2011, the US Army created a series of recruiting ads showing soldiers of different genders and racial backgrounds. It partnered with a cable company to deliver those ads according to the demographics of the people living in the house.

  There are other w
ays to discriminate. In 2012, Orbitz highlighted different prices for hotel rooms depending on whether viewers were using Mac or Windows. Other travel sites showed people different offers based on their browsing history. Many sites estimate your income level, and show you different pages based on that. Much of this is subtle. It’s not that you can’t see certain airfares or hotel rooms, it’s just that they’re ordered so that the ones the site wants to show you are easier to see and click on. We saw in Chapter 3 how data about us can be used to predict age, gender, race, sexual preference, relationship status, and many other things. This gives corporations a greater advantage over consumers, and as they amass more data, both on individuals and on classes of people, that edge will only increase. For example, marketers know that women feel less attractive on Mondays, and that that’s the best time to advertise cosmetics to them. And they know that different ages and genders respond better to different ads. In the future, they might know enough about specific individuals to know you’re not very susceptible to offers at 8:00 am because you haven’t had your coffee yet and are grouchy, you get more susceptible around 9:30 because you’re fully caffeinated, and then are less susceptible again by 11:00 because your blood sugar is low just before lunch.

  People are also judged by their social networks. Lenddo is a Philippine company that assesses people’s credit risk by looking at the creditworthiness of the people they interact with frequently on Facebook. In another weblining example, American Express has reduced people’s credit limits based on the types of stores they shop at.

  University of Pennsylvania law professor Oscar Gandy presciently described all this in 1993 as the “panoptic sort”: “The collection, processing, and sharing of information about individuals and groups that is generated through their daily lives as citizens, employees, and consumers and is used to coordinate and control their access to the goods and services that define life in the modern capitalist economy.” Those who have this power have enormous power indeed. It’s the power to use discriminatory criteria to dole out different opportunities, access, eligibility, prices (mostly in terms of special offers and discounts), attention (both positive and negative), and exposure.