Because Internet

by Gretchen McCulloch

  One use of emoji that’s explicitly beat-related comes when each word is followed by a clapping hands emoji, as in WHAT ARE YOU DOING . This started as an emoji representation of a beat gesture common among African American women. Comedian Robin Thede described the “double clap on syllables” in a Nightly Show segment on “Black Lady Sign Language.” But as writer Kara Brown put it when the gesture started making mainstream news headlines, “This—this clapping on every word for emphasis—is something that I have done since I was a cantankerous youth.” In 2016, it started spreading to mainstream Twitter users unaware of its offline, African American origins. But whether online or offline, it’s a beat.

  * * *

  —

  The emoji combinations story also explains a puzzle that I encountered when analyzing the SwiftKey data: the case of the missing eggplant emoji. We knew that people were fond of the eggplant emoji as a phallic symbol: heck, you can buy eggplant emoji plush toys and keychains. And yet, when we looked at the top two hundred most common sequences of two, three, and four different emoji, the eggplant was nowhere to be found. We did find other, less famous, sexual combinations, such as the tongue emoji with water droplets , or the pointing finger and the ok sign . But the eggplant emoji only showed up as pure repetition in our Top 200 lists. Same for the smiling pile of poo, another classic emoji that you can buy as endless novelty items: people were happy to repeat it , but reluctant to combine it. What gives?

  This mysterious absence of these classic emoji makes sense if we think about the difference between how emblems and co-speech gestures deal with sequences. Co-speech illustrative gestures are fluid, going smoothly from one into the other, with lots of possible shapes and variations for essentially the same meaning. If you describe the path of where you’ve gone today, you’ll use many gestures in a row and you could easily gesture it slightly differently when you tell me about it now and when you told someone else about it a few minutes ago. Same with illustrative emoji: you can depict “Happy Birthday” or the weather with different sequences on different days, and that works fine. Emblems, on the other hand, are discrete, individual gestures: they can repeat, but they don’t combine. You can applaud for a long time or flip someone off repeatedly, but you can’t un-applaud someone or un-flip someone the bird, even if you combine them with the widely understood head shake that means no. In the same way, both the eggplant and the smiling poo emoji are emblem emoji: they have conventional meanings not immediately obvious from their literal origins, and they don’t combine readily either. That’s why we tend not to see them in interesting emoji sequences like we saw the birthday party emoji. Sending someone all of the possible birthday party emoji is extra festive: great! But sending someone all of the possible phallic emoji (say, the eggplant and the cucumber and the corncob and the banana ) is NOT extra sexxaayy: that’s a weird salad. There are multiple kinds of gestures and multiple kinds of emoji. Paying attention to how emoji fit with each other can give us a renewed appreciation for the gestures we make every day.

  How We Got Emoji

  When we think of emoji as gesture, it’s clear why they caught on so quickly. But it leaves us with the inverse question: What took us so long to figure out a way to write our gestures?

  Well, we tried.

  Writing used to have illustration all the time. Medieval scribes illustrated their manuscripts with everything from the classic illuminated capital letters to a bizarrely popular motif of knights fighting giant snails at swordpoint. It was really the printing press that made us think that books should be composed primarily of walls of text: letters became significantly easier to produce than drawings. After all, once you’ve cast a set of metal letters, you can type any arrangement of words you could possibly want, but each new picture has to be engraved from scratch for printing. In theory, early printers could have created small, versatile metal drawings, too. In practice, they tended to be conservative about making new pieces of type: the first English printers imported their presses from Continental Europe, where no one used the English letter þ (thorn), so English printers substituted either the “th” letter sequence (which won out in most places) or the similar-looking letter “y” (which survives in a few limited contexts like Ye Olde Tea Shoppe). If printers weren’t willing to cast a genuinely important letter, well, you can see why pictures were banished to book jackets and frontispieces and children’s books. But the other factor preventing us from obtaining a delightful inventory of Renaissance emoji was psychological. What we thought we wanted out of writing was still very different: printing was a formal context, and handwriting was still around for informal doodles. We didn’t yet have the sense that we could demand emotional expression in the same place as our standardized typesetting.

  The chief variety of written gesture for a long time was the manicule, or printer’s fist ☞, a pointing finger drawn or typeset in the margins of manuscripts to call attention to a particular passage. It was in widespread use from the twelfth to eighteenth centuries, used by medieval monks adding notes, printers calling attention to corrections or additions, and Victorian readers highlighting passages they wanted to remember. It only fell out of use around the same time that the stylized arrow shape was developed in the early nineteenth century.

  Informal writing, however, retained a considerable array of ways to ornament our text: doodles were popular with authors from Lewis Carroll, who himself drew a series of sketches for the original handwritten version of Alice’s Adventures in Wonderland, to Sylvia Plath, who drew in both her own diaries as well as the margins of books she owned. (She was especially fond of cows.) Even if you lacked their doodling skills, you could express your aesthetic sensibility by ornamenting your personal correspondence with different-colored ink and monogrammed, bordered, textured, or even scented paper. You could also borrow images from other people, by cutting out printed photographs and quotes and gluing them onto your pages, as people did with commonplace books in the eighteenth and nineteenth centuries, and with printed stickers and scrapbooks in the modern era. Some of the 1970s postcards that we saw in previous chapters had handwritten smiley faces and doodled animals.

  Early computers weren’t much better than printing presses, as we saw in the previous chapter, with even fewer character and font options. But people did make borders, words, and artwork using punctuation symbols, a style known as ASCII art after the ninety-five printable characters defined by the ASCII encoding system of early computers, and later extended once more characters were available to mean any sort of art created with text-based symbols. (Text-based art itself is older, dating back to the limited graphics capabilities of typewriters.) The ASCII art below, for example, uses slashes, backslashes, underscores, and the occasional parenthesis and apostrophe to make hollow letters reading “ASCII art” and a slightly wider array of symbols including double quotes and equals signs to make a simple bunny rabbit. More ambitious examples could contain thousands of symbols and portray elaborate shading or an entire scene.

  One major advance in internet gesture happened as the result of a serious miscommunication on the Carnegie Mellon University computer message system. Most of the time, the message system was pretty serious: announcements of talks in the computer science department, lost and found items, and heated discussions about politics and which keyboard layout was the best. But one day in September 1982, messageboard users started goofing off by posting absurd hypothetical questions about the physics of elevators in free fall. What would happen, wondered one person, if you put a helium balloon in an elevator and cut the cable? Or what if, wondered a second person, you put a bunch of pigeons in a free-falling elevator? Okay but what if, wondered a third, the birds were breathing the helium? Would their cheeping get higher-pitched? A fourth person had an idea for a similar experiment: What if you put a tiny drop of mercury with a lit candle in the free-falling elevator?

  Alas, I am a linguist and can provide the answer to none of these quest
ions. What I care about is what happened next. First, the setup: someone continued the joke: “WARNING! Because of a recent physics experiment, the leftmost elevator has been contaminated with mercury. There is also some slight fire damage. Decontamination should be complete by 08:00 Friday.” Then, the problem: other people logged into the message system and saw only the fake warning, minus its necessary context. A few hours later, someone had to get back on and clarify that the warning was fake: “My apology for spoiling the joke but people were upset and yelling fire in a crowded theatre is bad news. . . .”

  Finally, the solution: the CMU users switched to brainstorming ways to indicate that a particular message was intended as a joke (this wasn’t the first time a user’s attempt at humor had been taken seriously). Various options were proposed—putting an asterisk * or a percent sign % or an ampersand & in the subject line, posting all messages with a numerical 0–10 “Humor Value,” creating a separate messageboard just for jokes, or using the sequence {#} “because it looks like two lips with teeth showing between them” or the sequence __/, which looks like a smiling mouth. But the idea that caught on was a suggestion by a professor named Scott Fahlman. Here’s the original message that he posted, dug up from dusty 1980s archives, from back when computer records were preserved on reel-to-reel tapes:

  19-Sep-82 11:44 Scott E Fahlman :-)

  From: Scott E Fahlman

  I propose that the following character sequence for joke markers:

  :-)

  Read it sideways. Actually, it is probably more economical to mark things that are NOT jokes, given current trends. For this, use

  :-(

  The idea of a simplified smiling face already had a considerable history, so Fahlman’s sideways proposal was straightforward to interpret. It was also easy to type, and was thus quickly picked up by other participants in the message thread, and within a couple months people were using sideways text faces beyond Carnegie Mellon and had come up with a wide variety of creative expansions on the sideways idea, including non-face examples like the heart <3 and the rose @>-->--. Many of the more elaborate examples, like sideways portraits of famous people, circulated more in lists of clever faces than in actual usage. (It’s unclear when a person would ever need to invoke Abraham Lincoln by smiley, but here he is ==(:-)= complete with tall hat and beard.) A few classics like :-) :-( ;-) :’-( :-P, and later their noseless variants :) :( ;) :’( :P, remained the most popular.

  Symbols like :-) were named emoticons, a combination of the words “emotion” and “icon.” One useful side effect of emoticons is that they let you incorporate the facial part into your running text, right alongside your words, rather than using a large, unwieldy image that has to go on a new line—even if it’s made out of the same ASCII characters as the rest of your message. Just like gestures and facial expressions fit seamlessly with spoken words, punctuation-based emoticons can directly accompany typed words.

  Filling an important niche, the text-based emoticons grew and changed. The meaning of the basic smile shifted after Scott Fahlman’s original proposal, from indicating a joke to indicating a more general positive sentiment, a marker of sincerity: “that’s great :)” is sincere, not sarcastic. The nose fell out of favor among younger people: in 2011, a study of emoticons on Twitter by linguist Tyler Schnoebelen found that noseful emoticons were used by people who also tended to tweet to celebrities like Pepe Aguilar, Ashton Kutcher, and Jennifer Lopez, whereas those who tweeted noseless emoticons tended to prefer to tweet to Justin Bieber, Miley Cyrus, the Jonas Brothers, and Selena Gomez. (For the benefit of readers from the future who don’t have a degree in Early Twenty-First Century American Pop Culture, I’ll point out that Justin Bieber and the like were very popular among teenagers in 2011, while Kutcher, Lopez, et al. were an older set of celebrities. This strongly suggests that younger people were dropping the noses in their emoticons.)

  Around the same time as emoticons were developing in the United States and on English-speaking networks, another form of digital face was developing on an early Japanese computer network known as ASCII Net. They were called kaomoji, from the Japanese kao (顔, “face”) and moji (文字, “character”). Kaomoji are like emoticons, but you don’t have to turn your head sideways to read them, allowing for virtually any pair of symbols to be used to represent the shape of the eyes, not just symbols like :) and =) that are already found in a pair. Classic kaomoji such as ^_^ (happy), T_T (crying), and o.O (wide-eyed) are nearly as old as emoticons—there are claims of them appearing on ASCII Net as early as 1985 or 1986.

  The emphasis on the eyes was important for kaomoji because of a broader cultural difference in how emotions are represented. When researchers show East Asian and Western Caucasian people photos of faces displaying different emotions, the Asian participants tend to make conclusions about the emotions based on what people are doing with their eyes, whereas the Western participants look to the mouth to read emotions. This tendency is reflected in the different conventions for portraying emotions in manga and anime versus Western cartoons, and it shows up again in the stylized faces of emoticons and kaomoji. Happy :) and sad :( emoticons can have the same eyes but must have different mouths, whereas happy ^_^ and sad T_T kaomoji can have the same mouths but must have different eyes. Some kaomoji have caught on more broadly among English speakers, especially those that narrate actions of the whole body rather than relying solely on the eyes, such as shruggie ¯_(ツ)_/¯ since 2014, flower-in-hair (◕‿◕✿) since 2013, and table flip (╯°□°)╯︵ㅗ—ㅗ since 2011. But the kaomoji that purely convey emotion through the eyes seem to require a certain minimum level of fluency with a set of cultural conventions that most English speakers simply don’t have (unless they’re manga or anime fans).

  By the late 1990s, you could include images on your website just by digging out the connection cable for your newfangled digital camera or combing through other people’s GeoCities pages for exactly the right “under construction” gif to borrow. In Japan, something new had caught on beyond kaomoji: sending picture messages back and forth on cellphones. Unfortunately, it was impractically popular, because it took a lot of data to send and receive them. So in 1997, the Japanese cellphone carrier SoftBank found a solution: What if they encoded some common pictures the way they encoded text characters? After all, when you text a friend the letter A, your phone doesn’t send pixel-by-pixel each tiny dot in the grid that would make up a picture of the letter A, your phone just sends one short number code like 0041 and your friend’s phone knows that 0041 makes an A and displays it. If you could send a simple number like 2764 to display a heart , things would go much faster than sending a whole image file. So designers at SoftBank created short number codes for ninety small pictures, including icons for weather, transit, time, and sports apps, as well as hearts, hands, and a few faces that looked a lot like the existing kaomoji. This was the origin of the emoji that we started talking about earlier.

  Although the word “emoji” resembles the English “emoticon” (“emotion” + “icon”), the word actually comes from the Japanese e (絵, “picture”) and moji (文字, “character”), the same moji as in kaomoji. This coincidence did probably help the word catch on among English speakers, but typing the symbols wasn’t quite as straightforward. These small, easy-to-send pictures quickly became popular in Japan, and other Japanese cellphone carriers got busy adding their own sets of emoji. But here they ran into a problem. The whole point of emoji was to save space by assigning number codes to small pictures, but different phone manufacturers were using different sets of images and different number codes for them. So if you had a phone with DoCoMo and you texted a heart emoji to your friend whose phone was from SoftBank, your friend might see an indecipherable box, nothing at all, or worse yet, an entirely different symbol like an umbrella or a music note. (One common point of confusion was that people who thought they were sending a Taurus zodiac sign from DoCoMo phones would end up
appearing to have sent a picture of a normal cow when received on a KDDI phone. Which could be...awkward.)

  The organization that’s in charge of standardizing the number codes for normal letters and numbers and punctuation characters is called the Unicode Consortium. The Unicode Consortium is a small committee of people who live at the intersection of tech geek and font nerd, and are mostly employees of major tech companies trying to make sure that, say, when you copy and paste an apostrophe from one program to another, or type an apostrophe on one device and view it on a different one, it doesn’t mysteriously change into ’ instead. This problem is fairly rare and confined to punctuation symbols for English, which was privileged to have its letters ubiquitously encoded very early. But the names for this problem in other languages speak to the frustration: Japanese mojibake, “character transformation” (that’s the same moji as in emoji); Russian krakozyabry, “garbage characters”; German Zeichensalat, “character salad”; and Bulgarian majmunica, “monkey’s (alphabet).” Multiply that by all the symbols in all the scripts of all the languages around the world, add in special symbols for mathematical notation and music notes and over six hundred styles of arrows (seriously), and you have the unglamorous but very important monkey-salad-garbage-transformation job that Unicode’s been doing since 1987.