We will always have new innovations, and people will find ways to misuse some, and to combine them in unanticipated ways. Some ideas will pass into oblivion like the pernicious RottenNeighbor.com. For a while, this website let you anonymously badmouth folks whose dogs allegedly pooped on your lawn or falsely label the guy down the street as a sex offender. Companies were not exactly eager to advertise on RottenNeighbor.com, and it now sits, dormant, though still registered and presumably ready to rise again if a viable business model suddenly appears. The “uncensored people reviews” at dirtyphonebook.com continue to provide an outlet for this kind of personal attack.
Creepiness is an elusive concept that taps into our primal fears and assumptions about the way things are and should be. Sigmund Freud pondered it, suggesting that creepy things, with part of their true nature hidden, remind us of our own deepest secrets and guilt over repressed impulses. Edgar Allan Poe achieved heights of creepiness in his short stories by playing on our fears of being buried alive or catching the plague.
Filmmakers often seek the fine balance that will send the audience away wondering darkly about a character’s true nature and motivation. Culture blogger Sarah Dobbs explains that movies can be scary with loud noises and sudden moves, but that truly unsettling cinema is a lot harder to achieve.
“To be one of the good horror movies,” she writes, “a film needs to establish a certain atmosphere; it needs to draw you in and make you care. It needs to give you something to think about when you’re trying to drop off to sleep at night; to make you wonder whether that creaking noise down the hallway was just the house settling, or something lurking in the shadows. Creepy stays with you. It gives you goosebumps.”9
Clowns, dolls, ghost stories, and even the words from the mouths of our young children can fascinate us in unsettling ways. The closer something is to our hearts and our highest values, the most disturbing it can be.
A famous discussion on the social news site reddit asked, “What’s the creepiest thing your young child has ever said to you?”
Here are two of the more disturbing examples:
I was tucking in my two-year-old. He said, “Goodbye, Dad.”
I said, “No, we say goodnight.”
He said, “I know. But this time it’s goodbye.”
Had to check on him a few times to make sure he was still there.
/u/UnfortunateBirthMark
And
When I was about three we had a cat that had stillborn kittens.
I asked my father if we could make crosses for them, which he did.
As he was making them I asked: “Aren’t those too small?”
Dad: “What do you mean?”
Me: “Aren’t we going to nail them to them?”
Dad (after several moments’ silence): “We’re not going to do that.”
Me: “Oh.”
/u/Tom_Zarek10
Cute? Innocent? Perhaps. But the creepiness stayed with these people long enough for them to share it on reddit. The fact that this thread is now up to around 15,000 comments speaks to our fascination with the unheimlich, a German word that literally means “the opposite of what is familiar or home-like.” By thinking about what unsettles us the most, we are able to confront and understand our greatest fears, and try to make rational decisions as citizens, software designers, creators, parents, and consumers.
So, what is creepy? We know it when we see it. The hairs stand up on our neck or we ask, “How do they know that?” Creepy things make us question our assumptions, and lie awake, wondering “what if?”
My study of hundreds of technologies that are creepy in various degrees has revealed some common factors that make people uneasy. At the end of this book, we will explore these “dimensions of technocreepiness” in the hope that we can avoid them in the future. We will even do a bit of “creep-proofing” to, as much as is humanly possible, protect ourselves from the worst ravages of invasive technologies. But for now, let us let those hairs do their job. We will begin our journey into technocreepiness at a rather unlikely place—the New York Public Library.
Intelligence Creep
There are twenty-nine steps from the corner of 41st Street and Fifth Avenue to the front entrance of the New York Public Library. I know this because, in the mid-1970s, I lugged a radio station’s tape recorder up every one of them. I might as well have been carrying a television set. Back then, “portable” meant that something had a handle. I was there to interview the keeper of an amazing new technology—the Kurzweil Reading Machine.
Billed as an aid to the blind, this bulky contraption was the world’s first functional text-to-speech synthesizer. Walter Cronkite used the machine to sign off on his January 13, 1976 newscast. I typed up a piece of paper with “For CBC Radio, this is Tom Keenan in New York.” The machine rattled this off for me in the same mechanical monotone that we now associate with the hacktivist group Anonymous.
I then asked the librarian, “What kind of things do people bring in to read on it?”
“Mostly pornography,” he replied.
I thought I had heard him wrong. He explained that “if somebody wants to hear a textbook on American History or something, there are plenty of volunteers who will read that. We’re seeing books like Lady Chatterley’s Lover, The Story of O, that sort of thing.”
A lot has changed since my first encounter with the Kurzweil Reading Machine. Instead of lugging a bulky tape recorder, I can now push a button on my smartphone and safely store my interview in the cloud. If I want to know how many steps I will face at the New York Public Library, I can simply count them on Google Street View. Anyone with Internet access can find all the pornography they could possibly want, and have it read to them in whatever exotic voice they desire.
The abundance and variety of Internet pornography illustrates a concept that Cullen Jennings, one of my former students and now a Cisco Fellow, expressed very well. “No matter how liberal or broad-minded you are,” he once said, “I guarantee I can find something on the Internet that will instantly offend you.” It is a small leap from “something that will offend you” to “something that will creep you out.”
Even though I have seen a lot of bizarre things since the 1970s, the image of tumescent guys hooked up to the Kurzweil Reading Machine at the public library has stuck with me, along with the gadget’s monotone voice.
A good friend and I used to split the generous “lead fees” which a certain national tabloid paid for ideas that turned into stories. Driven by empty wallets, and armed with a bottle of Jack Daniels, we could spin off quite a few plausible if sensational ideas in an evening. Of course, a tabloid tale is not a publishable story without a reputable expert to support it. This paper had a helpful list of “trained seals” who, for a fee, would happily confirm UFO sightings and authenticate photos of fictional monsters. Yet they had a big gap—they needed a bright young computer science professor, which was precisely my line of work.
Soon I came to be quoted on fantasy technology stories like “an amazing implanted chip will someday measure your caloric intake and release an appetite-suppressing hormone.” That feature attracted bags of mail from people desperately seeking to help me test this rather wacky idea. I sent the letters back suggesting that they look into some diet and exercise plans.
Perhaps I should have told them to wait. An article in the 2009 issue of MIT Technology Review describes a “small, stick-on monitor no bigger than a large Band-Aid” that can accurately monitor your caloric intake.11 Some smart scientist will undoubtedly invent the appetite suppression technique and make our fictitious dieter’s dream patch a reality.
In that weekly tabloid, I also mused that “someday computers will speak to you in the voice of your choosing. It might be Marilyn Monroe’s or Clark Gable’s or the voice of your long-dead mother.” Since then, science has shown that the sound of your mother’s voice indeed does have a profound physiological effect on you.
In studies of mother/daughter dyads, Leslie
Seltzer found that hearing Mom’s voice raised girls’ oxytocin levels, calming them down.12 Email and SMS messaging did not have the same effect. Some have even speculated that an artificially intelligent program that sounded like your mother, used her favorite expressions, and had an intimate knowledge of your life story, could be a powerful way to calm, interrogate, or even control you.
Back in 1966, Massachusetts Institute of Technology professor Joseph Weizenbaum unleashed ELIZA onto the world, hoping to illustrate the fundamental differences between human and computer brains. A fairly simple “chatterbot,” ELIZA could mimic the conversation style of a Rogerian psychotherapist, faking the answers when it did not understand what you just said.
Weizenbaum observed that a session with a psychotherapist was one of the few environments where it was perfectly acceptable to turn a question back on the speaker, or provide a canned response like “So, how does that make you feel?”
Although Weizenbaum intended ELIZA to be a parody with a point, many people enjoyed pouring out their personal problems to the program. Some even said they preferred it to a human therapist. When Weizenbaum told his secretary, who had been using the software, that he had access to the logs of all her conversations with ELIZA, she was outraged at this invasion of her privacy.13
ELIZA got herself into some humorous situations. In the age before massive hard disks, computer data was usually stored on 2,400-foot reels of magnetic tape which had to be manually mounted by an operator. The tape mount request came up as a flashing message on the operator’s console.
However, some of MIT’s computers did not have a human operator available. Pranksters configured one of them to use ELIZA to respond to operator requests. “Can you please mount a tape for me?” was met with “Is it because of your mother that you want me to mount a tape for you?”14 Profanity usually ensued from the hapless human.
The idea of programs that can mimic human conversation has spawned some creepy chatterbots at major companies. In 2006, the Coca-Cola Bottling Company of Atlanta, GA, “hired” a virtual representative named Hank. Hank was intended to, among other things, deflect phone calls to the company’s main switchboard from students doing reports on the history of Coca-Cola. He straddled the line between being a computer program and a person quite adroitly for his era.
I had some great talks with Hank. We discussed his personal life (“virtual representatives are not allowed to have relationships”) and he vehemently denied that Coca-Cola has ever contained cocaine.
Other people also tormented Hank with questions like15:
Q. Do you like snorting coke?
A. Of course! I like all of the products of The Coca-Cola Company.
Q. Are you a Mormon?
A. We are not in a position to comment on the ownership of stock. The holdings of our share owners are confidential. In accordance with our current governmental regulations, however, we include in our annual proxy statement a list of those share owners who are beneficial owners of five percent or more of our company stock, and the Mormon Church is not among them.16
History does not record when Hank was downsized from a full-fledged virtual representative to a “frequently asked questions” webpage.17 That seems to be all that is left of him today, though he is still wearing his trademark telemarketer headset.
If you could talk with Hank about such personal matters, why not unburden your venal and even mortal sins to a piece of technology? On-line confession does seem to have some kind of creepy visceral appeal to people. In 1984, an art project called “Interactive Catholic Confessional”—based loosely on ELIZA—was put on display. “Visitors entered a confessional space, knelt before the computer and went through the process of Catholic confession,” according to a posting at the University of Nevada at Reno. “The computer gave each user an appropriate penance for their sins.”18
The concept of computerized confession is still with us. In 2013, a Jewish congregation in Florida urged its members to atone for their sins through anonymous, but very public, tweets which would scroll by on a screen during the Yom Kippur services.19
On a more secular level, reddit has a whole subreddit called r/confession, and entire websites like www.truuconfessions.com (“your anonymous best friend”) thrive on this compulsion to share guilty secrets. Here, you can learn who is lusting after his cousin’s wife and who “kicked a child (who probably deserved it).” These posts make fascinating reading, but of course their real purpose is catharsis for those who write them. An amazing number of people seem to spend a lot of time poring over these stories, “upvoting” and “downvoting” them, and adding their own commentary. In one sense it is a new way of communicating with a higher power, even if this higher power is only a transient, anonymous online community.
The line between machine and human thinking is definitely blurring, as is well illustrated by the triumph of IBM’s Watson over the best human Jeopardy players. Virtual assistants like Apple’s Siri and Microsoft’s Cortana are mining our smartphones and emails to do some of our thinking for us.20 We can feel the hot breath of our technology pushing us relentlessly towards that much-touted “singularity”—the day when our creations will be smarter than us in ways that really count.21
In the mid-1800s, Augusta Ada King, Countess of Lovelace, studied the work of Charles Babbage, who designed a precursor to modern computers. Because she wrote down the steps to compute the Bernoulli numbers on Babbage’s never-constructed Analytical Engine, Lovelace is often called the first computer programmer.
She is also known for her famous “Objection” to the idea that a machine can possess creativity. “The Analytical Engine has no pretensions to originate anything,” she wrote in 1842. “It can do whatever we know how to order it to perform”22 (her italics).
If she were alive today, Lovelace might have trouble maintaining her position as Watson trounced her in her choice of intellectual games. But we do know that, putting aside quantum computers, neural networks, and other specialized technologies, mainstream computers still sequentially execute instructions that were designed by their human masters. If Lovelace’s Objection is as true as ever, why do technologies do things that amaze us and give us creepy spinal shivers?
One explanation of this apparent paradox is that many computer programs have already surpassed the comprehension of any one human mind. This was actually true of the operating system, OS/360, made for IBM’s mainframe computers in the 1960s and 1970s. It had so many modules and complexities that it took a team of systems programmers to build it, and nobody purported to know every inch of it. Mix in the creative input of today’s very bright designers and programmers, and you get a continuous stream of technologies that amaze, delight, confound, and, increasingly, disturb or even frighten us.
Sometimes we have trouble detecting the boundary between machine and human intelligence. Most people recognize that the “Recommended” suggestion list from Amazon.com comes from a robot. But what about the earnest email appeal from a friend who claims he is stranded abroad without funds. The sender seems to know intimate details about your mutual relationship. It is probably a nasty robo-scam, but how can you be sure?
In his signature essay on the subject, Sigmund Freud tackled the psychological aspects of our discomfort with things that may or may not be human. Using the example of the doll in the first act of Offenbach’s opera, Tales of Hoffman, he acknowledges that “doubts whether an apparently animate being is really alive” can invoke The Uncanny. Freud goes on to suggest that what we truly dread here is an Oedipus-style gouging out of our eyes, or, this being Freud, a symbolic castration.23
Japanese robotics professor Masahiro Mori coined the term “uncanny valley effect” to explain why we become uneasy when non-human things exhibit human-like behavior.24 Perhaps nothing embodies the spirit of the uncanny valley better than BINA48.
Figure 2. (top) BINA48 from the front. Courtesy of Robert Koier.
Figure 3. (bottom) BINA48 from the rear. Courtesy of Terase
m Movement Foundation.
Martine Rothblatt, a serial entrepreneur, lawyer, and researcher, has created an extremely lifelike humanoid robot in collaboration with robotics engineer Dave Hanson. In addition to having a convincing and expressive face made of a polymer called “Frubber,” BINA48 has an uncanny ability to display human mannerisms. New York Times reporter Amy Harmon, sent to Vermont to interview BINA48, reports a profound moment as BINA48 looked her in the eyes and said “Amy!”
“Maybe it was the brightening of the sun through the skylight enabling her to finally match up my image with the pictures of me in her database,” Harmon writes. “Or were we finally bonding?” The spell was broken by BINA48’s jarring next remark, which was to change the subject: “You can ask me to tell you a story or read you a novel.”25
BINA48 has cameras in her eyes and is equipped with face finding and facial recognition software. As their cost plummets to virtually zero, digital cameras are turning up almost everywhere. They now seem to be present at the best, worst, and creepiest moments of our lives.
Camera Creep
On April 19, 2013, law enforcement agents used a thermal imaging camera, combined with a tip from a citizen, to locate Dzhokhar Tsarnev in the aftermath of the Boston Marathon bombings. The image of a human form crouched under a tarp sped around the globe. The imaging technology was praised for leading to the result almost everyone was hoping for: the live capture of a desperate fugitive.
Figure 4. Fugitive Dzhokhar Tsarnev, hiding under a tarp. Courtesy of Massachusetts State Police Air Wing.
Thermal imaging cameras are not new. They have been used for years by firemen (who look for cool spots since burning walls are much hotter than trapped humans) and by house inspectors probing for heat-wasting leaks.
They also play a role in tracking down marijuana grow-ops and finding people and objects hidden in walls and vehicles. Yet, suddenly thermal imaging was front page news, seeming to give law enforcement superpowers. From their helicopter, the Massachusetts State Police found a needle in a haystack, using what seemed like a kind of x-ray vision.
Technocreep Page 2