Emblem gestures have precise forms and stable meanings. They may seem universal because they often cover different territories than languages do: the middle finger, or digitus impudicus, was also considered rude in ancient Greece and Rome, while the palm-inwards V sign means “up yours” in some English-speaking nations but not others. But ultimately, emblems are arbitrary and culturally specific: obscene emblems from around the world include thumbs up (“sit on this” in many Arabic-speaking countries), the ok sign (“asshole” in many Latin American countries), an open hand thrust forward (the Greek mountza), a fist with the thumb between the index and middle finger (the Russian and Turkish “fig” gesture), and a gesture known as the bras d’honneur or Iberian slap (common to many Romance-speaking countries), which consists of extending one arm palm-up in a fist while the other hand is placed on the upper arm at the crook of the elbow. Perform one of these inside its region and you’ll get anything from a rude gesture in return to legal prosecution for obscenity; perform it outside its region and no one will care. (An American recently told me about how, when visiting Japan, she was surprised to see people casually using their middle finger to press buttons on elevators and microwaves.) Perform one of them in a subtly wrong way (try flipping someone off with your palm facing towards them, instead of towards yourself), and you’ll get laughed out of town.
The kinds of emoji that end up in emoji trend piece articles (“10 Emoji You Should Be Using Right Now!”) sometimes also have taboo meanings. The eggplant emoji is a prime example: widely used as a phallic symbol, it’s a natural heir to the obscene gesture list above. The smiling pile of poo emoji is another: in deciding whether to include it in Gmail, the Japanese engineers had to explain its importance to the head office. They described it as: “It says ‘I don’t like that,’ but softly,” and “‘That’s unfortunate, and I would like to punctuate my comment with a reiteration that I am displeased at what has just been expressed.’ It’s the anti-like.” (For me, what made the poo emoji “click” was parsing it as “a bit shitty.”) But what’s crucial about emblem emoji is that they also have precise forms: some designers initially implemented the poo emoji without the smiling face, but this leaves out an essential aspect of its meaning. When emoji were first catching on internationally, a surprisingly big problem was emoji fragmentation: different app or device manufacturers were displaying the same underlying emoji with different designs. Platforms had not anticipated how much people would hate it when sending a lady in a red dress could result in their friends seeing a disco man or a blobby figure with a rose in its mouth: designers thought they were free to put their own company’s spin on the general idea of “dancer.” People felt as foolish sending the wrong dancer emoji as you would giving the middle finger backwards or crossing the wrong two fingers, and companies eventually backed down: the Emojipedia blog hailed 2018 as “The Year of Emoji Convergence.” If we think of these emoji as emblems, we know that the range for variation is tiny indeed.
Thinking of certain types of emoji as emblems can also make it clearer what they’re doing with respect to language. The key feature of emblems is that they’re nameable gestures, and dictionaries naturally have entries for gestures like “wink” and “thumbs up” in their status as English words. Similarly, the names of certain emoji (already English words or phrases) are taking on additional connotations which originate with emoji but don’t require the emoji itself. I’ve noticed a few people using nonculinary “eggplant” with no mention of the emoji at all: in one headline, a singer “Mistakenly Shared Photo Of His Eggplant On Instagram.” If this usage continues, it will clearly be necessary to add a new subdefinition under “eggplant” (joining the existing euphemistic senses of words like “banana” and “sausage”). But this doesn’t mean that dictionaries necessarily need to keep an inventory of all emoji, even the non-emblem ones, any more than they already do for the non-emblem gestures.
Emoji aren’t the only way to express emblems in internet communication. An early fan of Snapchat described the appeal of sending messages overlaid on top of a photo you’ve just taken as “like texting, but you get to use your face as the emoticon instead of an actual emoticon.” In other words, texting but with emblematic gestures. Animated gifs, while technically looping, silent animated image files that can display any image, are often used in practice to display emblems. The most engaging gifs have been found to be those with faces in them, a feature that’s come to be reflected in user interfaces. When you go to insert a gif on Twitter, the built-in categories of gif that you’re offered are nameable, stylized gestures by humans, cartoon characters, or occasionally animals, such as applause, eww, eye roll, facepalm, fistbump, goodbye, happy dance, hearts, high five, hug, kiss, mic drop, no, omg, ok, popcorn, scared, shocked, shrug, sigh, wink, and yawn. Certain gifs are so emblematic that they can be invoked by name, without an image file at all, just like a thumbs up or an eggplant emoji: when you want to convey your excitement in observing other people’s drama, you can send a gif of Michael Jackson eating popcorn in a darkened movie theater, eyes avidly glued to the screen, but you can also simply say #popcorngif or *popcorn.gif*.
It’s not an accident that the most iconic popcorn.gif is one of Michael Jackson. Both gestural and digital emblems participate in cycles of appropriation from African American culture: the high five came from a Jazz Age gesture known as the low five or “giving skin” and spread via sports teams, and the fistbump came from the “dap” among black soldiers during the Vietnam War. Similarly, the painting fingernails emoji entered the mainstream by its association with the black drag queen expression “throwing shade,” for giving a subtle, cutting insult. In an article called “We Need to Talk About Digital Blackface in Reaction GIFs,” Lauren Michele Jackson pointed out that black people are overrepresented in gifs used by nonblack people, especially those that show extreme emotion. She linked this stereotype to the exaggerated acting of minstrel shows and scholar Sianne Ngai’s term “animatedness” to describe the long-standing tendency to see black people’s actions as hyperbolic.
* * *
—
There are other gestures that don’t have conventional names, which I’d been ignoring entirely because they’re hard to describe in words. But we use them even more than we do emblems, because they come with us in practically everything we say. You’d probably gesture when saying, “Keep going that way and turn at the lights” and “The fish was THIS big” and “The person sitting next to me just kept going on, and on, and on . . .” but your gestures don’t have specific names, only descriptions. For directions, you’d point to indicate which way is “that way” and which direction to turn at the lights; for indicating size, you might put your hands flat, facing each other some distance apart; and for “going on and on,” you might make some sort of open hand shape, moving repeatedly in a rough circle.
If you try talking with your hands tied down (recruit a friend to make sure you get untied!), you’ll probably have a hard time with it. Researchers have done this: they showed people cartoons of Wile E. Coyote chasing Road Runner and asked them to describe them to someone else. Half the time, the describers had their hands fastened to the chair, ostensibly to take physiological measurements, but really to see what happened when they weren’t able to gesture. The researchers found that when you can’t gesture, you have a harder time telling the visual and spatial parts of a story: people talked slower, paused more, and were more likely to say “um” and “uh.”
Every culture that’s been studied has gesture, and we gesture along with our speech even when it’s communicatively useless, such as when we’re talking on the phone. Even people who have been blind since birth do it, even when they’re talking with people who they know are also blind. But it’s not so much about an irresistible temptation to wink or flip someone the bird: the gestures we can’t help doing are the ones without specific names. So linguists think that this other kind of gesture, called co-speech or illustrative
gesture, is more about the thinking of the speaker than the understanding of the listener. Sure enough, people who are encouraged to gesture do better at solving math problems and mental rotation.
The next time you’re in a restaurant, have a look around you at the groups of people at other tables. You probably won’t see a lot of emblems, but you’re guaranteed to see some co-speech gestures. Look at some people at a table far enough away that you can’t hear them: you can tell who’s speaking when by who’s gesturing. You can probably get a sense of how well they’re getting along, whether they’re laughing merrily or you’re about to witness an awkward fight, but the content of their conversation remains private because the meaning of co-speech gestures is more dependent on their surrounding speech. For example, the thumbs up sign could also be used as a co-speech gesture to indicate “up there.” But “up there” could just as easily be illustrated by the index finger or whole hand pointing up, using one or both hands, the eyes or eyebrows pointing up, or any of these things combined—none of which work as a substitute for the thumbs up emblem.
We can see this same flexibility at play when it comes to illustrating birthday greetings with emoji. People wish others a happy birthday using the cake with candles , the slice of cake , the balloon , the wrapped gift , the bouquet of flowers , or general positive emoji such as hearts, sparkles, happy faces, confetti, and positive hand shapes like the thumbs up or fistbump. These emoji showed up in the SwiftKey dataset in a wide variety of combinations and orders. When you’re illustrating your speech, you’re more willing to accept a range of options as suitable for “birthday” or “beach” or “fun” or “danger.” The birthday cakes display variously as chocolate, vanilla, or strawberry, with an inconsistent number of candles, across different platforms, and yet “The Year of Emoji Convergence” did nothing to make them more similar.
People were bothered by variation in the form of the dancer emoji because it was serving as an emblem, and emblems add their own separate meaning to the words they go with. But people were unruffled by variation in the form of the birthday cake emoji because it’s an illustration, and illustrative emoji instead highlight and reinforce a topic that’s already present. It’s okay if illustrators aren’t quite exactly on target: the surrounding words will provide enough context to interpret them correctly. For the emblem emoji, we tend to know exactly what we’re looking for because we’ve seen other people using it first. For illustrative emoji, we often go browsing through our keyboards instead.
But sometimes, when we go fishing for an illustration, we realize in surprise that there’s nothing at all suitable: “What?? How is there no ______ emoji??” The problem here is that emoji were added through a hodgepodge of historic compatibility and individual request, not designed as a systematic attempt to cover all areas of semantic space. (More on this later, when we get to the history of emoji.) Birthdays are a well-filled-in area of emoji illustration, but other domains are less so, especially those beyond emoji’s original roots in Japan or first transplant in the United States.
The illustrative, co-speech emoji are interpretable at face value: it takes cultural knowledge of birthday traditions to interpret a cake and a balloon, sure, but it doesn’t take any particular internet literacy to know that sending these items as emoji is intended to evoke birthday wishes. Illustrator emoji are readily used even by people who are less familiar with internet cultural norms: it’s easy to add a cat emoji to “Have you fed the cat yet?” But many an inadvertent grocery-store innuendo has been texted by someone who was treating the eggplant as an innocent illustrator rather than a suggestive emblem.
* * *
—
The final emoji puzzle is how we use them in combination with other emoji. One type of sequence, which gets a lot of attention, is the retelling of familiar stories in emoji: Emoji Dick is a retelling of Moby-Dick in emoji, the #EmojiReads hashtag featured people’s renditions of stories like Lord of the Flies and Les Misérables, and Emoji Karaoke is a game where you see who can come up with the best emoji version of a song before it stops playing. It’s easy to see how these fit in with the idea of emoji as gesture: they’re like playing digital charades or pantomiming to a friend across a loud bar. Emoji pantomime and other “stunt” uses of emoji are fun, but what I wanted to know was, is this truly a reflection of the generic, day-to-day use of emoji? How do emoji interface with our normal, casual writing?
To answer this, I got SwiftKey’s engineers to run two queries. The first thing I wanted to know was, what percent of things that people write are describable as emoji stories: emoji-only utterances that are at least five or ten emoji long? If emoji storytelling were truly common, if (as the headlines would have it) emoji were taking over from English, we should find lots of emoji-only messages. We did not. The vast majority of messages were text only. Of the ones that contained emoji at all, the vast majority put them alongside words. And of the messages that were only emoji, the majority were just one or two emoji long—presumably replies to something else. Less than one in a thousand messages were long enough that they might qualify as a potential emoji story. In fact, the only people I’ve been able to find who regularly communicate with extended emoji sequences are preliterate children. Many parents have told me about how their two- to five-year-old kids enjoy texting them messages full of dinosaur or animal emoji—but then these same kids start sending words instead of emoji once they learn how to read.
So, long emoji sequences weren’t common, but our dataset was still really large. What did these rare sequences of potential emoji storytelling look like?
This was my second question. I got SwiftKey’s engineers to extract the most common sequences of two, three, and four emoji. This is a common way of analyzing a large body of writing. The difference between a mere list of words and a story is that a story’s words are arranged in sentences and paragraphs: a collection of narratives has subpatterns among its common words that reflect the basic structure of the language itself. If you look at the most common sequences of two, three, and four words for the half-billion words of the Corpus of Contemporary American English, you get sequences like “I am,” “in the,” “I don’t,” “a lot of,” “I don’t think,” “the end of the,” “at the same time,” “as well as,” “for the first time,” “one of the most,” and “some of the.” They’re not riveting prose by themselves, but you can feel how well they’d work as glue to hold a more interesting story together. If people are commonly writing stories in emoji, looking for patterned emoji sequences is how we’d find them. We might expect, for example, a lot of the red circle with a line through it to indicate negation, or that emoji representing people, like or , would often be followed by the arrow emoji , indicating a person going somewhere.
Instead, what we found was repetition. Looking at the top two hundred sequences of each length, about half were pure repetition, such as two tears of joy emoji , three loudly sobbing emoji , or four red heart emoji . Those that weren’t simple repetition were often complex repetition, such as snow around a snowman , the see-no-evil, hear-no-evil, speak-no-evil monkeys , and kiss faces with kiss marks . Even the most heterogeneous strings of emoji were always thematically similar, such as heart eyes and kiss face or single tear and loudly crying , strings of related objects like birthday or fast food , and strings of hearts in different colors or sizes such as .
Emoji aren’t behaving like words in this respect: there are no repetitions at all in the top two hundred sequences of two, three, and four words in the Corpus of Contemporary American English. We don’t even get strings of all nouns or all adjectives the way we got strings of thematic emoji. Sure, words are sometimes used repetitively (very very very, higgledy-piggledy) and emoji nonrepetitively ( to mean “no snow” or to mean “I like pizza”). But nothing like this shows up in the top lists. You might use several similar adjectives to reinforce each other (such as “big bad wolf”), but you wouldn’t intertwine them in co
mplex repetition (there’s no “big bad big wolf”). Furthermore, emoji ordering wasn’t very important (different kinds of hearts or birthday-related emoji appeared in all sorts of orders), whereas word ordering is often important (“bad big wolf” and “red small car” just feel wrong). It’s essential to look at what’s common if we want to understand how emoji fit into our communicative systems; after all, what distinguishes emoji from any other set of little pictures is that billions of people use them on a daily basis. The true emoji question is what billions of people are currently doing with emoji, not what an advertiser or a philosopher thinks they could hypothetically do with them. The part of communication where we repeat stuff all the time isn’t in our words, it’s in our gestures.
Look over to your imaginary co-patrons at the imaginary restaurant again. Here’s someone with a hand loosely curled, palm upwards, moving it up and down to make a point. Someone else nods vigorously. Another person’s finger loops the air emphatically several times. Someone taps the table in quiet boredom. A politician on TV brings an open hand down on the podium over and over again to drive home a point. These repetitive gestures are known as beats. You can do any shape of gesture in the style of a beat, whether it’s repeatedly flipping someone off, pointing emphatically several times, or simply an open-handed beat that comes along for the ride in our regular conversations. What’s important about the beat is the rhythm: when you stutter out loud, your beat gestures stutter with you. When you hold a vowel for a loooong time, your beat gestures hold a silent scream for just as long.
Emoji have the same rhythmic tendency as beat gestures. That’s what the repetition is telling us. We type because we might also blow multiple kisses, we type because when we give the thumbs-up gesture, we sometimes do it rhythmically or hold it up for several seconds to emphasize it. Like how we can extend the letters of a word for emphasis, even when the result is unpronounceable (“sameeeee”), we can repeat even those emoji that don’t have direct gestural correlates, like the skull or the smiling pile of poo or the sparkle heart , because we’ve generalized this behavior to the category as a whole.
Because Internet Page 17