by Luke Dormehl
Epagogix’s route to Hollywood was something of an unusual one. Meaney, a fortysomething Brit with a mop of thick black hair and a face faintly reminiscent of a midcareer Orson Welles, had a background in risk management. During his career Meaney was introduced by several mathematician friends to what are called neural networks: vast, artificial brains used for analyzing the link between cause and effect in situations where this relationship is complex, unclear or both. A neural network could be used, for example, to read and analyze the sound recordings taken from the wheels of a train as it moves along railway tracks. Given the right information it could then predict when a particular stretch of track is in need of engineering, rather than waiting for a major crash to occur. Meaney saw that neural networks might be a valuable insurance tool, but the idea was quickly shot down by his bosses. The problem, he realized, is that insurance doesn’t work like this: premiums reflect the actuarial likelihood that a particular event is going to occur. The better you become at stopping a particular problem from happening, the lower the premiums that can be charged to insure against it.
Movies, however, could benefit from that kind of out-of-the-box thinking. With the average production costs for a big budget movie running into the tens or even hundreds of millions of dollars, a suitably large flop can all but wipe out a movie studio. This is exactly what happened in 1980, when the infamous turkey Heaven’s Gate (singled out by Guardian critic Joe Queenan as the worst film ever made) came close to destroying United Artists. One studio boss told Meaney that if someone was able to come up with an algorithm capable of stopping a single money-losing film from being made each year, the overall effect on that studio’s bottom line would be “immeasurable.” Intrigued, Meaney set about bringing such a turkey-shooting formula to life. He teamed up with some data-analyst friends who had been working on an algorithm designed to predict the television ratings of hit shows. Between them they developed a system that could analyze a movie script on 30,073,680 unique scoring combinations—ranging from whether there are clearly defined villains, to the presence or lack of a sidekick character—before cranking out a projected box office figure.
To test the system, a major Hollywood studio sent Epagogix the scripts for nine completed films ready for release and asked them to use the neural network to generate forecasts for how much each would make. To complicate matters, Meaney and his colleagues weren’t given any information about which stars the films would feature, who they were directed by, or even what the marketing budget was. In three of the nine cases, the algorithm missed by a considerable margin, but in the six others the predictions were eerily accurate. Perhaps the most impressive prediction of all concerned a $50 million film called Lucky You. Lucky You starred the famous actress Drew Barrymore, was directed by Curtis Hanson, the man who had made the hit movies 8 Mile and L.A. Confidential, and was written by Eric Roth, who had previously penned the screenplay for Forrest Gump. It also concerned a popular subject: the world of high-stakes professional poker. The studio was projecting big numbers for Lucky You. Epagogix’s neural network, on the other hand, predicted a paltry $7 million. The film ended up earning just $6 million. From this point on, Epagogix began to get regular work.
Illustration of a simplified neural network. The cat’s cradle of connections in the middle, labeled the “intermediate neurons,” is the proprietary “secret sauce” that makes Epagogix’s system tick.
A fair question, of course, is why it takes a computer to do this. As noted, Will Smith does something not entirely dissimilar from his kitchen table at the start of every week. Could a person not go through the 30,073,680 unique scoring combinations and check off how many of the ingredients a particular script adhered to? The simple answer to this is no. While it would certainly be possible (albeit time-consuming) to plot each factor separately, this would say nothing about how the individual causal variables interact with one another to affect box office takings. To paraphrase a line from George Orwell’s Animal Farm, all numbers might be equal, but some numbers are more equal than others.
Think about it in terms of a successful movie. In March 2012, The Hunger Games was released at cinemas and rapidly became a huge hit. But did The Hunger Games become a hit because it was based on a series of books, which had also been hits, and therefore had a built-in audience? Did it become a hit because it starred the actress Jennifer Lawrence, who Rolling Stone magazine once referred to as “the coolest chick in Hollywood”?7 Or did it become a hit because it was released at a time of year when a lot of young people were out of school or college and were therefore free to go to the cinema? The best that anyone can say about any one of these questions is: maybe. The Hunger Games was based on a successful series of books, it did star a hot young actress popular with the film’s key demographic, and it was released during the spring break holiday in the United States when large numbers of young people were on holiday. But the same could be said for plenty of other films that don’t go on to become massive hits. And although The Hunger Games ultimately took more than $686 million in cinemas, how does anyone know whether all of the factors mentioned resulted in positive gains? Could it be that there was a potential audience out there who stayed home because they had heard that the film was based on a book, or because it starred Jennifer Lawrence, or because they knew that the cinema would be full of rowdy youths fresh out of school? Might the film have earned a further $200 million if only its distributors had known to hold out until later in the year to release it?
These are the types of questions Epagogix seeks to quantify. A studio that employs Meaney and his colleagues will send Epagogix a shooting script, a proposed cast list, and a note about the specific time of year they plan to release their film. In return, they receive a sealed brown envelope containing the neural network’s report. “We used to send reports that were this thick,” Meaney says, creating a gap between his thumb and his forefinger to indicate a dossier the thickness of an average issue of Vogue magazine. Unconvinced that they were being read all the way through, the company now sends just two or three pages, bound by a single staple. “You might think that studios would want more than that, but in fact we spent a lot of time trimming these down,” he continues.
The last page of the report is the most important one: the place where the projected box-office forecast for the film is listed. There is also a second, mysterious number: usually around 10 percent higher than the first figure, but sometimes up to twice its value. This figure is the predicted gross for the film on the condition that certain recommended tweaks are made to the script. Since regression testing is used to analyze each script in forensic detail, Meaney explains that the neural network can be used to single out individual elements where the potential yield is not where it should be—or where one part of the film is dragging down others. Depending upon your disposition and trust in technology, this is the point at which Epagogix takes a turn for either the miraculous or the unnerving. It is not difficult to imagine certain screenwriters would welcome the types of notes that might allow them to create a record-breaking movie, while others will detest the idea that an algorithm is telling them what to do.
It’s not just scriptwriters who have the potential to be confused either. “One of the studio heads that we deal with on a regular basis is a very smart guy,” Meaney says. “Early on in our dialogue with him he used to ask questions like, ‘What would happen if the main character wears a red shirt? What difference does that make in your system?’ He wasn’t trying to catch us out; he was trying to grasp what we do. I was never able to explain to his satisfaction that it all depends. Is this a change that can be made without altering any of the other variables that the neural network ranks upon? Very seldom is there a movie where any significant alteration doesn’t mean changes elsewhere.” To modify a phrase coined by chaos theory pioneer Edward Lorenz, a butterfly that flaps its wings in the first minute of a movie may well cause a hurricane in the middle of the third act.
The studio boss to whom Meaney refers was likely picking a purposely arbitrary detail by mentioning the color of a character’s shirt. After all, who ever formed their opinion about which movie to go and see on a Saturday night, or which film to recommend to friends, on the basis of whether the protagonist wears a blue shirt or a red shirt? But Meaney was nonetheless bothered by the comment: not because the studio boss was wrong, but because he wanted to reassure himself that saying “it depends” wasn’t a cop-out. That evening he phoned one of his Epagogix colleagues back in the UK. “Quick as a flash, they said to me, ‘Of course it depends.’ Think about Schindler’s List,” Meaney recalls. “At the end of the film you get a glimpse of color and it’s an absolutely pivotal moment in the narrative. In terms of our system it suddenly put the question into an historical context. In that particular film it makes the world of difference, while in another it might make no difference at all. Everything’s relative.”8
Parallel Universes
One way to examine whether there really are universal rules to be found in art would be to rewind time and see whether the same things became popular a second time around. If the deterministic formula that underpins companies like Epagogix is correct, then a blockbuster film or a best-selling novel would be successful no matter how many times you repeated the experiment over again. Avatar would always have grossed in excess of $2 billion, while John Carter would always have performed the exact same belly flop. Wolfgang Amadeus Mozart was always destined for greatness, while Antonio Salieri was always doomed to be an also-ran. This would similarly mean that there is no such thing as an “unlikely hit,” since universal truths would state that there are rules that define something as a success or a failure. Adhere to them and you have a hit. Fail to do so and you have a flop.
A few years ago, a group of researchers from Princeton University came up with an ingenious way of testing this theory using computer simulations. What sparked Matthew Salganik, Peter Dodds and Duncan Watts’s imagination was the way in which they saw success manifest itself in the entertainment industry. Much as some companies start out in a crowded field and go on to monopolize it, so too did they notice that particular books or films become disproportionate winners. These are what are known as “superstar” markets. In 1997, for instance, Titanic earned nearly 50 times the average U.S. box-office take for a film released that year. Was Titanic really 50 times better than any other film released that year, the trio wondered, or does success depend on more than just the intrinsic qualities of a particular piece of content? “There is tremendous unpredictability of success,” Salganik says. “You would think that superstar hits that go on to become so successful would be somehow different from all of the other things they’re competing against. But yet the people whose job it is to find them are unable to do so on a regular basis.”
To put it another way, if Goldman’s Law that nobody knows anything is right, is this because experts are too stupid to realize a hit when they have one on their hands, or does nobody know anything because nothing is for certain?
In order to test their hypothesis, Salganik, Dodds and Watts created what they referred to as an “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market.”9 This consisted of an online music market, a bit like iTunes, but featuring unknown songs from unknown bands. The 14,341 participants recruited online were given the chance to listen to the songs and rate them between 1 (“I hate it”) and 5 (“I love it”). They could then download those songs that they liked the most. The most downloaded tracks were listed in a Top 40–style “leader board” that was displayed in a prominent position on the website.
What made this website different from iTunes was that Salganik, Dodds and Watts had not created one online music market, but many. When users logged on to the site, they were randomly redirected to one of nine “parallel universes” that were identical in every way with the exception of the leader board. According to the researchers’ logic, if superstar hits really were orders of magnitude better than average, one would expect the same songs to appear in the same spot in each universe.
What Salganik, Dodds and Watts discovered instead was exactly what they had suspected: that there is an accidental quality to success, in which high-ranking songs take an early lead for reasons that seem inconsequential, based upon those taste-makers who sample it first. Once this lead is established, it is exacerbated through social feedback. A bookshop, for instance, might notice that one particular book is proving more popular than others, and therefore decide to order more copies. At this stage, the book may be selling 11 copies for every 10 sold by its next most popular rival—a marginal improvement. But when the new copies arrive they are displayed in favorable places around the shop (on a table next to the front door, for example) and soon the book is selling twice as many copies as its closest rival. To sell even more, the bookshop then decides to try to attract new customers by lowering its own profit margins and selling the book at a reduced price. At this point the book is selling four times as many copies as its closest rival. Because customers have the impression that the book is popular (and therefore must be good) they are more likely to buy it, thereby driving sales up even more.
This is what psychologists call the “mere-exposure” effect. At a certain juncture a tipping point is reached, where people will buy copies of the book so as not to be left out of what they see as a growing phenomenon, in much the same way that we might tune in to an episode of a television show that has gained a lot of buzz, just to see what all the fuss is about.
In Salganik, Dodds and Watts’s experiment the songs that were ranked as the least popular in one universe would never prove the most popular in another, while the most popular songs in one universe would never prove the least popular somewhere else. Beyond this, however, any other result was possible.
The Role of Appeal
As you may have detected, there was a problem posed by the formulation of Salganik, Dodds and Watts, one that they acknowledged when it came time to write up their findings. In an experiment designed to determine the relationship between popularity and quality, how could any meaningful conclusions be drawn without first deciding upon a quantifiable definition for quality? “Unfortunately,” as the three researchers gravely noted in their paper, “no generally agreed upon measure of quality exists, in large part because quality is largely, if not completely, a social construction.” To get around this issue (which they referred to as “these conceptual difficulties”) Salganik, Dodds and Watts chose to eschew debate about the artistic “quality” of individual songs altogether, and instead to focus on the more easily measurable characteristic of “appeal.”
A song’s appeal was established through the creation of one more music market, this time with no scoreboard visible. Lacking the presence of any obvious social feedback mechanisms, Salganik, Dodds and Watts theorized that whichever song turned out to be the most popular in this scenario would do so based wholly on the merits of its objective appeal. What those were didn’t matter. All that mattered was that they existed.
This market-driven reading of “appeal” over “quality” is (no pun intended) a popular one. “There are plenty of films that, to me, might be better than Titanic, but in the marketplace it’s Titanic that earns the most,” says Epagogix’s Nick Meaney. If what emanates from his company’s neural network happens to coincide with what Meaney considers a great work of art, that is wonderful. If it doesn’t, it’s better business sense to recommend studios fund a film that a lot of people will pay money to see and not feel cheated by, rather than one that a few critics might rave about but nobody else will watch. Netflix followed a similar logic to Meaney when in 2006 it implemented its (now abandoned) $1,000,000 open competition to ask users to create a filtering algorithm that markedly improved upon Netflix’s own recommender system. Instead of an “improvement” being an algorithm that directed users toward relatively obscure critical favorites like Yasujirô Ozu’s 1953 master
piece Tokyo Story or Jean Renoir’s 1953 La Règle du jeu, Netflix judged “better recommendations” as recommendations that most accurately predicted the score users might give to a film or TV show. To put it another way, this is “best” in the manner of the old adage stating that the best roadside restaurants are those with the most cars parked outside them.
The advent of mass appeal is a fairly modern concept, belonging to the rise of factory production lines in the late 19th and early 20th century. For the first time in history, a true mass market emerged as widespread literacy coincided with the large-scale move of individuals to cities. This was the birth of the packaged formula, requiring the creation of products designed to sell to the largest number of people possible. Mass production also meant standardization, since the production and distribution process demanded that everything be reduced to its simplest possible components. This mantra didn’t just apply to consumer goods, but also to things that didn’t inherently require simplification as part of their production process. As the authors of newspaper history The Popular Press, 1833–1865 observe, for example, the early newspaper tycoons “packaged news as a product [my emphasis] to appeal to a mass audience.”10
In this way, art, literature and entertainment were no different from any other product. In the dream of the utopianists of the age, mass appeal meant that the world would move away from the elitist concept of “art” toward its formalized big brother, “engineering.” Cars, airplanes and even entire houses would roll off the factory conveyor belt en masse, signaling an end to an existence in which inequality was commonplace. How could it, when everyone drove the same Model T Ford and lived in the same homes? Art was elitist, irrational and superficial; engineering was collectivist, functional and hyperrational. Better to serve the democratized objectivity of the masses than the snobbish subjectivity of the few.11