by David Boyle
It has all the makings of a fairy tale. If you choose the wrong measure, you sometimes get the opposite of what you wanted. And any measure has to be a generalization that can’t do justice to the individuals that are included.
Counting paradox 3: Numbers replace trust, but make measuring even more untrustworthy
When farmers and merchants didn’t trust each other to provide the right amount of wheat, they used the standard local barrel stuck to the wall of the town hall, which would measure the agreed local bushel. When we don’t trust our corporations, politicians or professionals now, we send in the auditors – and we break down people’s jobs into measurable units so that we can see what they are doing and check it. If doctors hide behind their professional masks, then we measure the number of deaths per number of patients, their treatment record and their success rate, and we hold them accountable. When politicians look out of control, we measure their voting records and their popularity ratings – just as the TV commentators break down a sporting performance into opportunities, misses, aces, broken services and much else besides.
It wasn’t always like that. Previous generations realized that we lose some information every time we do this – information the numbers can’t provide. They realized, like James Anyon, that we could never measure what a doctor does so well that we could do it for them. They still have experience that slips through the measurement, so we still have to accept the word of the professionals to get to the truth. The British establishment used to be quite happy to accept the word of the professionals if they were ‘trustworthy gentlemen of good character’. But from the outside, that trust looked like a cosy nepotistic conspiracy. And probably it was.
It was this kind of political problem which led to the growth of cost-benefit analysis. This was originally used by French officials to work out what tolls to charge for new bridges or railways, but it was taken up with a vengeance in twentieth-century America as a way of deciding which flood control measures to build. After their Flood Control Act of 1936, there would be no more federal money for expensive flood control measures unless the benefits outweighed the costs. Only then would the public be able to see clearly that there was no favouritism for some farmers rather than others. It was all going to be clear, objective, nonpolitical and based on counting.
Even so, the professionals clung to their mystery as long as they could, just as doctors fought the idea of scientific instruments that would make the measurements public and might lead people to question their diagnoses – the stethoscope was acceptable because they were the only ones who could listen to it. Even the US Army Corps of Engineers – in charge of the flood control analysis – tried to keep the mystery alive. ‘It is calculated according to rather a complex formula,’ a Corps official told a Senate committee in 1954. ‘I won’t worry you with the details of that formula.’ It couldn’t last. The more they were faced with angry questioning, the more their calculations had to be public.
But how far do you go? Do you, as they did for some flood control schemes, work out how many seagulls would live in the new reservoir, and how many grasshoppers they would eat, and what the grain was worth which the grasshoppers would have eaten? Do you work out what these values might be in future years? Do you value property when no two estate agents can name the same price? ‘I would not say it was a guess,’ one of their officials told the US Senate about property values. ‘It is an estimate.’ And after all that, it is economists who persuaded the US Secretary of the Interior, Bruce Babbitt, to start demolishing the dams their predecessors laboriously calculated.
So here’s the paradox. Numbers are democratic. We use them to peer into the mysterious worlds of professionals, to take back some kind of control. They are the tools of opposition to arrogant rulers. Yet in another sense they are not democratic at all. Politicians like to pretend that numbers take the decisions out of their hands. ‘Listen to the scientists,’ they say about BSE or genetically-modified food. ‘It’s not us taking the decisions, it’s the facts.’ We are submitting these delicate problems to the men in white coats who will apply general rules about individual peculiarities. It is, in other words, a shift from one kind of professional to another, in the name of democracy – from teachers and doctors to accountants, auditors and number-crunchers. And they have their own secretive rituals that exclude outsiders like a computer instruction manual.
But when it comes to auditing the rest of us in our ordinary jobs, auditing undermines as much trust as it creates – because people have to defend themselves against the auditors. Their lives – usually their working lives – are at stake, and their managers will wonder later why the figures they spent so much to collect are so bizarrely inaccurate. And as we all trust the companies and institutions less, we trust the auditors less too.
Counting paradox 4: When numbers fail, we get more numbers
Because counting and measuring are seen as the antidote to distrust, then any auditing failure must need more auditing. That’s what society demanded the moment the Bank of Credit and Commerce International had collapsed and Robert Maxwell had fallen off his yacht into the Bay of Biscay. Nobody ever blames the system – they just blame the auditors. Had they been too friendly with the fraudsters? Had they taken their eyes off the ball? Send in the auditors to audit the auditors.
If the targets fail, you get more targets. Take the example of a large manufacturer that centralizes its customer care to one Europe-wide call centre. After a while, they find that the customers are not getting the kind of care they were used to before. What does the company do, given that it can’t measure what it really needs to – the humanity and helpfulness of their service to customers? They set more targets – speed answering the telephone, number of calls per operator per day. They measure their achievements against these targets and wonder why customers don’t get any happier. ‘People do what you count, not necessarily what counts,’ said the business psychologist John Seddon.
Counting paradox 5: The more we count, the less we understand
Numbers are the international tools of scientists. They allow experts to ‘speak one and the same language, even if they use different mother tongues’, said the philosopher Karl Popper – whose libertarian beliefs made him enthusiastic about anything which could break down totalitarian regimes. The auditors look for measurements with no human content, like the metre, (one 10 millionth of the distance from the pole to the equator) in their search for pure objectivity. It is a scientific dream of the kind you often get after revolutions, in this case the French. Could taking decisions like this usher in an era of facts after all that political confusion? Answer: no.
This is because everyone would have to count in exactly the same way, in laboratory conditions, taking no account of local variation or tradition, so the figures are not as informative as they might be. Decisions by numbers are a bit like painting by numbers. They don’t make for great art. When you reduce something to figures or the bottom line, you lose information, and the Tower of Babel comes tumbling down again.
This is an international language based on centrally imposed definitions and understanding. It’s a kind of modern imperialism, with all the respect for local understanding of the glass towers of the international architectural style. Can we really believe the European Union was getting accurate comparisons in their recent mega-survey asking 60,000 people if they ‘could make ends meet’? As if a German, a Greek and a Brit would all understand the same by that. When Pepsi had their slogan ‘Come alive with the Pepsi generation’ translated into Chinese, it was understood as ‘Pepsi brings your ancestors back from the grave’. Microscopic differences in definition have big effects. It’s chaos theory all over again.
But the paradox also works the other way round. When the Berlin Wall came down, the global exchange controls were swept away, and almost everything around the world could suddenly be measured in money. The process drives out tyrants, privilege and secrecy – just as money is supposed to do. But the real reason money has
this effect is because it is a counting system: it’s the numbers which count, not the money. So we have a parallel process which you might call the ‘globalization of fact’. In a world without borders and without trust, everything must be translated into figures.
Counting paradox 6: The more accurately we count, the more unreliable the figures
Number-crunchers can’t miss anything out if they’re going to be absolutely precise. So you get a peculiar phenomenon when the cost-benefit experts spend enormous efforts getting a figure absolutely correct – only to throw something else in which is simply plucked from the air.
This has a long tradition, back to the French statistician Adolphe Jullien in 1844, who worked out precisely the cost of moving one unit of traffic on their new rail system. He finally came up with a wonderfully exact figure of 0.01254 francs per kilometre. But what about administration and the interest on capital? Ah yes, he says – but these were more difficult to assess, so he arbitrarily doubled the figure. Or the US Corps of Engineers who would spend months on the exact cost-benefit of new waterways they liked, then shoved in a notional $600,000 for national defence and $100,000 for recreation.
Many figures are an unusual amalgam of the precise and the arbitrary. It’s like Lewis Carroll’s story about the little boy who comes up with a figure of 1,004 pigs in a field. ‘You can’t be sure about the four,’ he is told. ‘And you’re as wrong as ever,’ says the boy. ‘It’s just the four I can be sure about, ‘cause they’re here, grubbing under the window. It’s the thousand I isn’t pruffickly sure about.’
Counting paradox 7: The more we count, the less we can compare the figures
‘It has not gone unnoticed that crime has increased parallel with the number of social workers,’ said the doyen of moral conservatives, Dr Rhodes Boyson, in 1978. He was speaking after the fearsome, bible-bashing Chief Constable of Greater Manchester James Anderton had made a challenging speech about crime figures for England and Wales. There had been 77,934 recorded crimes in 1900, he said. In 1976, there were 2,135,713. The country was horrified.
The Daily Mirror replied with a classic critique of these kinds of statistics: ‘It is equally true that crime has increased parallel with speeches from Dr Boyson,’ they said. All you can do with statistics is to show that trends tend to happen together. Nobody can prove that one rise caused another. The point is that Anderton’s frightening figures looked scientific, unanswerable and objective – an absolute test of Britain’s criminality.
They weren’t. In 1900, the recording of crime was pretty informal, and different crimes horrified people. What would have been seriously violent now, might then just have been charged as a simple assault or drunkenness. Crimes against property, or against the aristocracy, really upset the late Victorians and Edwardians. But violence seems to have been regarded primarily as what the lower classes did to each other, and it concerned society less. These were the days when one in four London policemen was assaulted every year, and when the streets echoed to the antics of gangs like the Somers Town Boys or the notorious Manchester Scuttlers. And when policemen in particularly violent areas tried to make an arrest, they were liable to be surrounded by large threatening crowds shouting ‘Boot them!’.
It used to be different then. And if you doubt it, you can read the complaints of the chaplain of Newgate Gaol while Bentham was calculating the world – that all the boys in prison kept a mistress. That includes those aged nine and ten. Simple figures can’t possibly compare such different worlds. The past is a foreign country, as L. P. Hartley once said: they do things differently there.
Every generation believes that crime is getting worse. ‘The morals of children are tenfold worse than previously,’ the social reformer Lord Ashley told the House of Commons in 1843, using a bogus statistical style that would be much copied. At the same time, the communist pioneer Friedrich Engels had calculated that crime had risen over seven times, since the Battle of Trafalgar in 1805. A generation later, in 1862, British society was horrified by the outbreak of ‘garottings’ – a kind of mugging by throttling the victim. By the 1890s, it was ‘hooligans’, hitting old ladies in the street and brandishing pistols – which were then available by mail order from the department store Gamages. Then there was the sudden 90 per cent rise in bag-snatching between the wars, made to seem worse because police statistics stopped reporting such thefts as ‘lost property’.
The same thing happened with vandalism in the 1970s, when police figures stopped distinguishing between major damage and minor damage (defined as less than £20). When the distinction was ended in 1977, the minor offences were added to the list, and vandalism figures doubled in a single year. This added about 200,000 incidents to the total number of crimes in the UK – just before Anderton unveiled his statistics.
Counting paradox 8: Measurements have a monstrous life of their own
Stalin announced his first Soviet five-year plan in 1928, an enormous undertaking planned to increase gross industrial output by 235.9 per cent and labour productivity by 110 per cent. But don’t be fooled by these figures, which were completely spurious. The fake precision was to lend a pseudo-scientific air to the whole enterprise. The actual effect of the plan was to reduce real per capita income by half, and starve millions on what Stalin referred to as the ‘agricultural front’. Even so, he declared the first five-year plan a success 12 months early in 1932, and the second one started right away.
The figures were widely believed, even in the West. But not only were nearly all of the figures falsified (something you can do during a reign of terror) but they carried with them a terrible authoritarianism to try to force them to be true. Which is why strikes had to be redefined as ‘sabotage’, and why after 1939, employees had to be fired if they were once more than 20 minutes late for work. It was also why one in eight of the Soviet population was either shot or sent to a labour camp during Stalin’s reign. Figures are frightening sometimes.
Counting paradox 9: When you count things, they get worse
In quantum physics, the mere presence of the observer in sub-atomic particle experiments can change the results. In anthropology, researchers have to report on their own cultural reactions as a way of offsetting the same effect. And once you start looking at numbers you keep falling over a strange phenomenon, which is that the official statistics tend to get worse when society is worried about something. For the sake of argument, I’ll call it the ‘Quantum Effect’.
Why, for example, did the illegitimacy figures shoot up only after the war babies panic in 1915? At the time of the panic, the number of illegitimate births was actually astonishingly low – and the number of marriages strangely high. After the panic, the illegitimacy rate suddenly increased. Why was the number of homes unfit for human habitation in the UK in 1967 (after the TV film Cathy Come Home) almost twice the figure for 1956 – despite over a decade of intensive demolition and rebuilding?
The garrotting scare in the 1860s was the same. The story began during the silly season in August 1862, and public horror got so bad that Punch advertised a range of fearsome neck-guards with metal spikes to protect your neck. But the increase in the crime statistics came immediately afterwards, once the Garrotters Act had brought back flogging for adults. The tragic death of Stephen Lawrence in a racist attack led to widespread concern about race attacks in London. But after the public inquiry on the subject in 1998, Metropolitan Police figures of race attacks leaped from 1,149 to 7,790 in one year.
It was the same with the sex abuse statistics. They toddled along in the UK at the 1,500-a-year mark until 1984, when an unprecedented wave of publicity on both sides of the Atlantic catapulted the issue to the top of the public agenda. Between 1984 and 1985, the National Society for the Prevention of Cruelty to Children reported a 90 per cent increase in reported cases. And in the following year they reported a similar rise. Child abuse campaigners would say that the actual rate of child abuse is never reflected properly in the statistics. They may well be right, the same
would be true of the figures for racist attacks. All I am saying is that the actual statistics wouldn’t have told you anything, except how strongly the public felt about it at the time. So often, the statistics start rising after the panic, rather than the other way round, as an eagle-eyed society tries to stamp out the unforgivable. That’s the quantum effect.
It’s difficult to know quite why the figures go up. Sometimes the definitions change to reflect greater public concern. Sometimes people just report more instances of it because it is in the forefront of their minds. Sometimes, maybe, what we fear the most comes to pass.
III
If you wander through the East End of London, you can still catch a glimpse of the best British example of political numbers having a life of their own. By the end of the Second World War, up to a third of all of the homes in Britain had been damaged, and almost half a million couldn’t be lived in at all. The Attlee government failed to build more than 170,000 replacements a year, and it was the future Prime Minister Harold Macmillan who launched into the 1951 general election campaign, as Conservative housing spokesman, with a pledge to build 300,000 new homes a year.
There was nothing very unusual about this. Politicians have made statistical promises for generations, but Macmillan was different in that he actually achieved it. By reducing the building standards and cutting the space requirement by 10 per cent, there were 300,000 homes being built every year by 1953, and Macmillan had made his reputation as ‘Supermac’.