The Numerati
Page 5
Following World War II, however, retail took a halfcentury detour into mass industrialization. Shoppers were handed carts and instructed to find their own stuff. Whether they were pushing those carts through Ikea or Wal-Mart, they had entire warehouses to explore. And the merchandise was cheap, in part because the stores had eliminated the middleman—the shopkeeper at the local store who knew the customers by name. They mastered a startling new efficiency, which came from manufacturing and distributing with martial precision. That’s what the brainiacs and their computers were focused on: operations. The customers? As we made our way from the massive lots through the equally massive stores, we were processed like card-carrying herd animals.
Now retailers are changing. Accenture’s Smith calls it “back to the future.” Instead of deploying millions of shopkeepers to twenty-first-century counters, they’re relying on automatic machinery, from video cameras to newfangled customer loyalty cards. The operation runs on data, our data. The goal is to follow our footsteps in much same the way that e-tailers track our clicks. In the marketplace of the Numerati, we’ll define ourselves as shoppers in ever-greater detail simply by going about our business in a store. When the stores get to know us, they’ll recognize us the moment we walk in the door—just the way the corner grocer used to. And just like that grocer, they’ll know our week-to-week routines and our not-so-secret cravings. They may calculate that we’re probably running short on cat kibbles, and they won’t forget that we spike a gallon or two of eggnog every holiday season. (And wouldn’t it taste better with premium Jamaican rum this year?) The automatic systems will calculate not only what we’re likely to buy but also how much money we make for the store. Many of them will learn how to lavish big spenders with special attention and nudge cheapskates toward the door.
AN OLD shopping cart is parked next to the wall at Accenture’s lab, high above downtown Chicago. The offices are chock full of tech gadgetry. Blinking video cameras hang from the ceilings, staring down on the researchers. (They’re guinea pigs in a new surveillance system designed to track shoppers and workers.) In one nook of the lab is a large, always-on video connection with another Accenture lab in Silicon Valley. Around lunchtime in Chicago, you can see the California contingent coming to work, steaming coffee cups in hand. You hear their phones ringing and their footsteps echoing across the lobby 2,000 miles to the west. All of this gadgetry is backed by a wraparound view of Chicago’s skyscrapers, with Lake Michigan shimmering in the distance. In this technology showcase, the shopping cart looks out of place and a little forlorn. But it reminds Rayid Ghani and his small team of researchers of their key mission: to predict the behavior of people like my wife, and you, and me as we make our way through stores.
Ghani made a splash in 2002 with a study of how a clothing retailer like The Gap or Eddie Bauer could automatically build profiles of us from the things we buy. This sounds simple, but it adds a thick layer of complexity to data mining. If you unearth an old receipt gathering dust in your bedroom, you’ll see that one afternoon a few months ago you bought, say, one pair of gray pants, two cotton shirts, and some socks. What can the retailer possibly learn about you from this data? That you’re a human being with a body and, presumably, two feet? They take that much for granted. That you spend an average of $863 per year in the store? That’s a tad more interesting. But if each one of the items you bought carried a bit more contextual information, what computer scientists call a layer of “semantic” detail, much more of you would pop into focus.
Let’s say the pants are tagged as “urban youth.” With this bit of knowledge, the system can move beyond your spending habits and start to delve into your personal tastes—much the way Amazon.com calculates the kind of reader you are from the books you buy. A clothing system with semantic smarts can send you coupons for garments that appeal to urban youth. It can track the proclivities of this “tribe” (that’s a word marketers adore). And depending on the store’s privacy policy, it might decide to sell that data to other companies eager to market songs or cars to the same group. Some, as we’ll see later, might even use tribal data to push members toward one political candidate or another. Complications? No doubt. Maybe you’re a 55-year-old woman who bought that pair of pants for your 16-year-old son. Maybe he hated them. That’s not really you in the receipt, and it’s not him either. Faced with such complexity and contradictions, machines need smart and patient teachers to guide them in making sense of us.
That’s how Rayid Ghani views himself—as a personal tutor for the idiot savants we know as computers. Ghani is short, a bit round, and quick to smile. He’s one of the friendliest tutors his students could hope for (not that they’d notice). A Pakistani who studied at the computer science powerhouse Carnegie Mellon, Ghani would seem to fit right in with the Numerati. But in their rarefied ranks, he’s missing a standard ingredient: a doctorate. Having “only a master’s” in his circle is viewed as a handicap. But the 29-year-old outsider has grown accustomed to clawing his way upward. The son of two college professors in Karachi, Pakistan, he applied to American colleges fully aware that he could afford only those offering a full scholarship. He landed at the University of the South, in Sewanee, Tennessee. Ghani calls it “a liberal arts college in the middle of nowhere.” Hardly the ideal spot for a budding computer scientist, it is better known for its theology school. But one summer, Ghani won an internship at Carnegie Mellon, in Pittsburgh. He plunged into a world where classmates were teaching cars to drive by themselves and training computers to speak and read. He developed a passion for machine learning. Upon graduation from Sewanee, he proceeded to a master’s program at CMU. Ghani was in a hurry. He started publishing papers nearly as soon as he arrived. And when he got his master’s, he decided to look for a job “at places where they hire Ph.D.’s.” He landed at Accenture, and now, at an age at which many of his classmates are just finishing their doctorate, he runs the analytics division from his perch in Chicago.
Ghani leads me out of his office and toward the shopping cart. For statistical modeling, he explains, grocery shopping is one of the first retail industries to conquer. This is because we buy food constantly. For many of us, the supermarket functions as a chilly, Muzak-blaring annex to our pantries. (I would bet that millions of suburban Americans spend more time in supermarkets than in their formal living room.) Our grocery shopping is so prodigious that just by studying one year of our receipts, researchers can detect all sorts of patterns—far more than they can learn from a year of records detailing our other, more sporadic purchases. (Most of us, for example, buy zero cars and zero TV sets in any given year.)
Three years ago, Ghani’s team at Accenture began to work with a grocery chain. (They’re not allowed to name it.) This project came with a windfall: two years of detailed customer records. The stores left out names, ages, and other demographic details, but none of that mattered. The 20,000 shoppers Ghani and his colleagues studied were simply numbers. But by their behavior in the stores, each number produced a detailed portrait of a shopper.
Let’s assume you’re one of those nameless shoppers. What can researchers learn about you? As it turns out, plenty. By the patterns of your purchases, and the amount you spend week after week, they can see if you’re on a budget. They can calculate your spending limit. If they add some semantic tags to the data, they can draw other conclusions. When they see you starting to buy skim milk, or perhaps those miracle milk shakes, they can infer that you’re on a diet. And they have no trouble seeing when you lapse. That carton of Ben & Jerry’s in your cart, or the big wheel of Roquefort, is a giveaway. But wait! Maybe it’s the holiday season, or your birthday. A few more weeks of receipts will spell out whether you’re just cheating a little or in free fall. All of this they can do with the kind of statistical analysis an eighth grader could understand.
It gets a bit more complicated when they calculate your brand loyalty. Let’s say you like Cherry Coke. You lug home a 12-pack every week. How much would Pepsi have to slash the
price of its Wild Cherry Cola to entice you to switch? Ghani and two colleagues, Katharina Probst and Chad Cumby, watch how the shoppers respond to sales and promotional giveaways. They score each shopper on brand loyalty, and even loyalty to certain products within a brand. Some people, they’ve found, are loyal to certain foods, such as Kraft’s macaroni and cheese. But does that loyalty extend to other Kraft products? For a certain group of shoppers, it does. The Accenture team takes note.
What they have on their hands is an enormous catalog of the eating habits of a small group of urban Americans in the first years of this century. Anthropologists of a certain bent would feast on it. But what good does it do a supermarket to know that you, for example, have a $95 weekly budget, are fiercely loyal to Cheetos, and flirted with the Atkins diet last barbecue season? What can they do with all that intelligence when they don’t do business with you until you show up, loyalty card in hand, at the checkout counter? At that point, you’ve done your shopping. The chance to offer you promotions based on your profile has passed. Sure, they can throw a few coupons in your bag. Maybe you’ll remember them on your next visit, but probably not. This is why, until now, supermarkets have virtually ignored the records of individual shoppers. They had little opportunity to put them to use.
The real breakthrough will come when retailers can spot you grabbing an empty cart and pushing it into the store. This has been a grocers’ dream for decades. In a previous life in the 1990s, that sad little shopping cart at Accenture was a proud prototype of a “smart cart,” one that allowed shoppers to swipe their loyalty cards through a computer attached to the cart, which would then lead them to bargains. “Everyone tried to do it,” Ghani says. The attempts fell flat. The computers were too pricey, the analytics primitive. But computers are far cheaper now. Companies like Accenture are betting they can make systems so smart that shoppers will view the new smart cart as a personal assistant.
The first of such smart carts are just starting to roll. Stop & Shop is testing them in grocery stores in Massachusetts. Carts powered by a Microsoft program are taking their first turns in ShopRite supermarkets along the East Coast. The German chain Metro is launching them in Düsseldorf. And Samsung-Tesco, a Korean-British venture, has them operating in Seoul. A few things we know even at this early stage. For one, a computer on a shopping cart can ill afford to make dumb mistakes. This sounds axiomatic, but the fact is, we’ve long given grocery stores the benefit of the doubt when they offer us fliers and coupons that don’t match our needs or wants, since they don’t pretend to know them. But if a shopper has been buying skim milk for a year and the personalized cart insists on promoting half-and-half, the shopper may well view the smart cart as idiotic (and revert to the traditional dumb cart that specializes in rolling).
The other extreme? If these carts get too smart, we’ll likely view them as creepy. I can just imagine rolling through my neighborhood Kings, when the cart starts flashing a message: STEVE: Hurry to aisle three for bargains on two of your favorite FUNGAL MEDICATIONS, plus this bonus SELECTION for the fungus you’re most likely to contract NEXT! At that point, I’d be inclined to push it out to the street and under the wheels of an oncoming truck.
Setting aside such troubling scenarios, here’s what shopping with one of these carts might feel like. You grab a cart on the way in and swipe your loyalty card. The welcome screen pops up with a shopping list. It’s based on the patterns of your past purchases. Milk, eggs, zucchini, whatever. Smart systems might provide you with the quickest route to each item. Or perhaps they’ll allow you to edit the list, to tell it, for example, never to promote cauliflower or salted peanuts again. This is simple stuff. But according to Accenture’s studies, shoppers forget an average of 11 percent of the items they intend to buy. If stores can effectively remind us of what we want, it means fewer midnight runs to the convenience store for us and more sales for them.
Things get more interesting when store managers begin to manipulate our behavior. Rayid Ghani opens his laptop and shows me the supermarket control panel that he and his team have built. “Let’s say you want four hundred shoppers to switch to a certain brand of frozen fish,” he says. With a couple of clicks, the manager can see how many shoppers at the store buy this item. They sit in groupings known in marketing lingo as “buckets”—in this case, the frozen-fish bucket. Let’s say it includes 5,000 shoppers. Among that group are those who buy rival brands of frozen fish. They’re the target audience, and they sit in three smaller buckets, say, 1,000 shoppers per rival brand. Of those shoppers, one-third appear to be brand loyalists. It would likely take big discounts to pry them from the fish they usually buy. But the others, some 2,000, are more flexible when it comes to brands. They switch easily and often.
These buckets, as you can see, are getting increasingly refined. Now we’re down to the brand-fickle buyers of certain types of frozen fish. Ghani plays at the controls. If he cuts the price by just 50 cents a pound—and sends word of the discount to their smart carts—he can entice a projected 150 of them to jump to the target brand. Ghani lowers the price by another 75 cents. At that level, an additional 300 bargain-hunters would line up to buy the fish. The manager can play with endless variables. He can adjust the formula to raise profits, to goose sales, to promote brands, to slash inventory. It’s a virtual puppet show, all of it based on probability. The puppets, needless to say, are mathematical representations of us.
Let’s say you’re notoriously fickle when it comes to brands. Even the smallest fluctuations will push you from Cheerios to Wheaties and back again. If the manager is interested in slashing inventory, you’re likely to be in the first bucket he picks up. You’re an easy sell. But if the goal is to switch your allegiance from one brand to another, you’re a lousy bet. No offense, but you’re disloyal, at least in this context. You’ll pocket the discount and abandon the brand the very next time you can save a dime. The manager might fare better promoting the discount to those who stick to brands a bit longer than you do. Naturally, they’re in another bucket.
You may also lose out on discounts if you hew to a weekly budget. Let’s say you spend about $120 a week on groceries. The system calculates that you’re on a budget because, say, 87 percent of the time you spend between $113 and $125 a week. If you’re not restricted to a formal limit, you might as well be. Assume that the manager is eager to get rid of a mountain of detergent moldering in the warehouse. He’s offering jumbo boxes at two for the price of one. Should he send the word to your screen? Maybe not, Ghani says. The reason is simple. For every dollar you spend on discounted products, that’s one less dollar you have in your budget to spend at full price. That hurts profits. To get rid of that detergent, it’s smarter to target people in freer-spending buckets.
Among the most unpleasant buckets a manager must confront are those loaded with “barnacle” shoppers. That term comes from V. Kumar, a consultant and marketing professor at the University of Connecticut. Barnacles, from a retailer’s perspective, are detestable creatures. We all know a few of them. They’re the folks who drive from store to store, clipped coupons in hand, buying discounted goods—and practically nothing else. Kumar calls them barnacles because, like the mollusks clinging to a ship, they hitch free rides and contribute nothing of value. In fact, they cost the retailer money. With all the consumer data pouring in, Kumar says, it’s becoming a snap to calculate a projected profit (or loss) for each customer. Kumar, who sells his advice to Ralph Lauren and Procter & Gamble, says that retailers should “fire” customers who look likely to drag down profits.
This doesn’t mean hiring musclebound bouncers to block these shoppers at the door. But retailers can take steps in that direction. They can start by removing barnacles from their mailing lists. Increasingly, they’ll also have the means to make adjustments inside the store. If bona fide barnacles are pushing smart carts through a supermarket, for example, it might make sense to fill their screens with off-putting promotions for full-priced caviar and truffles. (Discouragi
ng unwanted shoppers is far easier on the Internet. Already, online merchants are assailing their barnacles with advertisements. And if these bargain hunters click to browse the pages of a book or gawk at the free photos on a paid-porn site, they get shunted to the slowest servers, so that they wait and wait.)
If you think about it, barnacles thrive in markets where we’re all treated alike. They feast on opportunities that the rest of us, for one reason or another, miss. But now retailers are gaining tools not only to spot barnacles but also to discriminate against them. Barnacles, of course, are the first to notice when this happens. It’s their nature to keep their eyes wide open. And you can bet that they’ll challenge this type of discrimination in court. In a class-action suit in 2005, lawyers representing some 6 million subscribers to Netflix, the film-by-mail rental service, charged that the service was taking longer to send movies to its most active customers. Those were the film buffs who paid a flat monthly fee of $17.99 for limitless rentals and tried to see as many movies as they could for their money. This involved watching a movie or two the very day they arrived in the mail and rushing to mail them back the next morning. (I know the routine; for my first few months on Netflix, I was an eager barnacle.) Netflix officials admitted that they favored less active (and more profitable) customers with prompt mailings. And in a settlement, they gave millions of subscribers a free month of service. But, significantly, they did not vow to change their barnacle-punishing ways. They simply adjusted the wording in their rental contracts.
Barnacles aren’t the only creatures in Kumar’s menagerie. He also warns retailers about “butterflies,” customers who drop in at the store on occasion, spend good money, and then flit away, sometimes for months or years on end. They’re unreliable, and retailers are warned to avoid lavishing attention on them. “You shouldn’t chase the butterflies,” the professor says. However, by studying their patterns of behavior, smart retailers may learn which butterflies they can turn into reliable customers—a bucket that Kumar calls “true friends.”