In 2007, DFI’s Good Hospital Guide gave the Mid Staffordshire NHS Trust an HSMR of 127, the fourth highest in the country. The trust thought that inaccurate coding might be the cause of the high HSMR: a review of case notes led to recoding in several cases. The trust also commissioned two epidemiologists from Birmingham University, Professor Richard Lilford and Dr M. A. Mohammed, to examine the statistical methods used to calculate the HSMR. Lilford and Mohammed were known critics of the HSMR: they argued that it was susceptible to all sorts of biases, such as the accuracy of coding, local GP care, the proportion of admissions that are emergency, and local access to hospice beds. They concluded that this method could not possibly measure ‘avoidable’ deaths, and thus the quality of care. Many other statisticians and health informatics experts agreed with them. Stafford Hospital and its HSMR might have remained a local or, at least, regional concern had it not been for a woman called Julie Bailey. Her eighty-six-year-old mother, Bella, died at the hospital on 8 November 2007, after being admitted eight weeks earlier with ‘an inflamed hiatus hernia’. The Bailey family was shocked by the poor nursing care at the hospital. Bailey complained to the hospital CEO Martin Yeates, but received no response. She wrote a letter to the local newspaper and was contacted by several other families who had similar experiences; the response prompted Bailey to set up a local pressure group called ‘Cure the NHS’. This group, consisting of local people whose relatives had died at Stafford Hospital, held its first meeting in December 2007 at Breaks, the café run by Bailey.
Meanwhile, the Healthcare Commission, aware of the high HSMR at Stafford Hospital, carried out an investigation between March and October 2008; the report was published in March 2009. The Commission was not impressed by the hospital’s explanation for the high HSMR – namely, that it was primarily due to poor coding. Details of the report were leaked to the press before publication. Among these details was an estimated 400 to 1,200 excess (and by extrapolation ‘avoidable’) deaths over a period of more than a decade. These figures did not appear in the final, published report, but were leaked to the press, who put these numbers in their headlines; they are, to this day, still presented as evidence. When the Commission’s report was published, Prime Minister Gordon Brown and Health Secretary Alan Johnson apologized to the patients and families who had experienced poor care at the hospital. CEO Martin Yeates was suspended and the chairman Toni Brisby resigned. Alan Johnson was replaced by Andy Burnham on 5 June 2009, and on 21 July 2009, he announced that there would be a further, independent inquiry into Stafford Hospital, which would be chaired by Robert Francis QC. The report of this Inquiry was published on the 24 February 2010.
The Inquiry devoted much time to mortality rates at the hospital, and Francis relied greatly on Jarman’s evidence. The report contains the now-famous table of the HSMRs at Stafford Hospital from 1996 to 2008, with the final figure of an excess of 1,197 ‘observed’ over ‘expected’ deaths. Francis expressed doubts about the HSMR statistics, pointing out that later figures for Mid Staffordshire showed ‘an astonishing apparent recovery’: the HSMR for 2008/9 was down to 89.6, and in the Good Hospital Guide 2009, Stafford was one of the top 14 hospitals. While acknowledging Sir Brian Jarman’s reputation and eminence, Francis suggested that statistics such as the HSMR should be produced by an independent government-funded agency, not by a commercial enterprise such as DFI. Jarman, however, was careful to point out to the Inquiry that all ‘excess’ deaths were not necessarily ‘avoidable’:
We recognize that mortality alerts and HSMRs cannot be used as a direct tool for discovering failings in hospitals… What the data does… is pose the question: what is the explanation for your high mortality for the particular diagnosis or procedure that has alerted that month? We make it very clear in the alert letter that we send to Trusts that we draw no conclusions as to what lies behind the figures.
Francis finally ruled that no conclusions could be drawn from the HSMR figures:
Taking account of the range of opinion offered to the Inquiry, including a report from two independent experts, it has been concluded that it would be unsafe to infer from the figures that there was any particular number or range of numbers of avoidable or unnecessary deaths at the Trust.
The story, and Robert Francis, didn’t end there. In June 2010, the new (Conservative) Health Secretary Andrew Lansley announced that there would be another public inquiry, which would again be chaired by Francis. Cure the NHS had complained that the first Inquiry was not public, and Francis himself thought its remit too narrow. Francis reopened for business in November 2010, and another report was published in February 2013. Yet again, he heard evidence from Sir Brian, but Jarman, by this stage, seems to have tested even Francis’s patience: ‘The Inquiry received voluminous and detailed evidence and analyses from Professor Jarman.’ Francis 2 covered much the same territory as Francis 1, such as the by now infamous table ‘provided by Professor Brian Jarman’ on HSMRs at Stafford from 1996 to 2009. Professor Sir Bruce Keogh expressed major concerns about the HSMR:
Potentially ‘avoidable’ deaths cannot be identified a priori, with the result that there can be no ‘gold standard’ against which to assess the sensitivity and specificity of purely statistical measures such as HSMRs; and that no single measure can possibly encapsulate all aspects of the quality of care offered by a hospital… I have major reservations with the presentation of simplistic ‘league tables’ of HSMRs.
Ian Kennedy also criticized the HSMR: ‘Professor Sir Ian Kennedy told the Inquiry that, at the time, the HSMR was not widely accepted as predicting risk or underperformance.’ Francis clearly did not enjoy the protracted haggling over the HSMR methodology; the report is littered with phrases such as ‘to the non-statistician’ and ‘if I understand correctly’. He concluded, somewhat wearily, much as he had after the first Inquiry, that ‘to this day, there is no generally accepted means of producing comparative figures, and unjustifiable conclusions continue to be drawn from the numbers of deaths at hospitals and about the number of avoidable deaths’. The media and politicians must have skimmed through Francis’s report of 783,710 words; they maintained their belief that there had been 1,200 ‘avoidable’ deaths at Stafford. The Guardian was the only newspaper which later admitted – in 2015 – that this figure was entirely bogus, and that its own reporting of the numbers had been incorrect.
Stafford Hospital was probably no worse, and no better, than many other NHS district general hospitals. Many local people deeply resented Julie Bailey’s campaign; she received anonymous hate mail, and her mother’s grave was vandalized. She left Stafford in 2013, several months after the publication of the second Francis Report. Stafford, like many hospitals, struggled with poor morale, inadequate staffing and facilities, the pressure of targets and the ever-increasing demands of an ageing and frailer population. The witness statements made for harrowing reading, but was Stafford really a rotten apple, an outlier? I witnessed, during the fourteen years I worked in the NHS, episodes of cruelty and neglect as bad as those described at Stafford, in hospitals that were regarded as ‘centres of excellence’. What happened at Stafford and what happened, and still happens, in many other NHS hospitals was the result of a toxic combination of decades of managerialist totalitarianism, understaffing and the steady decline in the professional standards of doctors and nurses. Robert Francis, the eminent and sweetly reasonable silk, didn’t understand that acute medical wards are constantly on the edge of chaos, understaffed and leaderless; that most beds are occupied by highly dependent elderly patients; that senior and experienced nurses have abandoned these wards for easier roles; that form-filling and box-ticking have been prioritized over patient care. His two inquiries were triggered by bogus statistics, but Francis seemed to find it difficult to believe that the method which produced these statistics could be produced by a man as eminent as Professor Sir Brian Jarman.
Less than three weeks before the publication of Francis 2, Sir Bruce Keogh announced that he would be conductin
g a review of urgent and emergency care in the NHS, focusing particularly on fourteen hospitals, like Stafford, with a high HSMR. His report was published in July 2013. He wrote to Jeremy Hunt, secretary of state for health: ‘However tempting it may be, it is clinically meaningless and academically reckless to use such statistical measures [as the HSMR] to quantify actual numbers of avoidable deaths.’ Keogh told Hunt where the real problems lay: ‘Some hospital trusts are operating in geographical and academic isolation… the lack of value and support being given to frontline clinicians, particularly junior nurses and doctors… the imbalance that exists around the use of transparency for the purpose of accountability and blame rather than support and improvement.’ Keogh also commissioned a review into the relationship between the HSMR and ‘avoidable’ deaths, which was carried out by Nick Black (professor of health services research) and Lord Ara Darzi. Their review was published in the British Medical Journal in 2015. They randomly selected 100 deaths from 34 trusts, and found that the proportion of avoidable deaths was low (3.6 per cent), with no significant association with HSMRs. A similar (if smaller) external review of deaths at Stafford Hospital – commissioned by the Mid Staffordshire NHS Foundation Trust – had been carried out by Dr Mike Laker (of Newcastle University) in 2009. He reviewed 120 case notes and interviewed 50 families. He concluded that poor care had led directly to death in ‘perhaps one’.
The letter from Keogh to Hunt was written on 16 July 2013. Two days earlier, the Sunday Telegraph published an article called ‘Labour’s “denial machine” over hospital death rates’:
Professor Sir Brian Jarman, of Imperial College London, worked on a government review which will this week show that 14 hospital trusts have been responsible for up to 13,000 ‘excess deaths’ since 2005.
… Prof Sir Bruce Keogh will describe how each hospital let its patients down badly through poor care, medical errors and failures of management, and will show that the scandal of Stafford Hospital, where up to 1,200 patients died needlessly, was not a one-off.
The media clearly loved Jarman; here is a 2014 profile by the Guardian: ‘Ever ready to take a journalist’s phone call and with a pithy quote for every occasion, the eighty-year-old remains a reporter’s friend. He is also a spiky proponent of his brainchild: the hospital standardized mortality ratio (HSMR).’ A month after the publication of the Keogh report, Sir David Spiegelhalter wrote a commentary in the British Medical Journal. Spiegelhalter is a distinguished statistician: Winton professor for the public understanding of risk at Cambridge University, fellow of the Royal Society and president of the Royal Statistical Society. He examined the media outrage over hospital deaths:
So where did the ‘13,000’ come from? It is the difference between the observed and ‘expected’ number of deaths in the 14 trusts between 2005 and 2012. The Telegraph claims this number is based on research by Professor Brian Jarman, one of the Keogh team, and the numbers can be derived from data on the HSMR available on Jarman’s website. It should have been fairly predictable that such a briefing to journalists would be misleadingly reported… we would expect at any time that around half of all trusts would have ‘higher than expected’ mortality, just by chance variability around an average… The difference between the observed and expected number of deaths has been called ‘excess deaths’, a term used in the Bristol Royal Infirmary Inquiry: as head of that statistical team, I deeply regret this use as it so readily translates, whether through ignorance or mendacity, into ‘needless deaths’… Like the ‘1,200’ at Mid Staffs, ‘13,000’ threatens to become a ‘zombie statistic’ – one that will not die in spite of repeated demolition.
Spiegelhalter’s contrition over his clumsy use of the term ‘excess deaths’ at Bristol is telling. Professor Sir Brian Jarman, however, is not given to such expressions of public contrition, and was undaunted by the thrashing of his statistical baby by Sir Bruce Keogh and many others. If anything, it stiffened his resolve. In September 2017, the Mail on Sunday carried a piece entitled: ‘NHS buries 19,000 “suspect” deaths: expert demands urgent probe into “avoidable” fatalities amid shock claims dozens of hospitals across Britain are “potentially unsafe”’:
Professor Sir Brian Jarman says his shocking findings mean there are dozens of ‘potentially unsafe’ hospitals that should be investigated over high death rates, but which are being overlooked… He calculated that there were 32,810 ‘unexpected’ deaths in English hospitals over the past five years. But using the NHS’s preferred method, only 13,627 were classed as such – a difference of 19,183 deaths.
Jarman’s prominent status in the British medical establishment has given him a public platform from which he terrifies the general public through his regular briefing of journalists. Francis indulged him and stopped short of direct criticism only because he is so eminent. It is too late for Jarman to admit that he was wrong: he has invested his entire career and academic credibility in the HSMR.
Robert McNamara, however, was big enough to admit in his old age that he had been wrong. During his long post-White House career, he repeatedly revisited his Vietnam experience to see what could be learned from it. At the age of eighty-five, he told an interviewer: ‘I’m at an age where I can look back and derive some conclusions about my actions. My rule has been: Try to learn. Try to understand what happened. Develop the lessons and pass them on.’ He devoted his later years to doing just that. He met with the North Vietnamese general Vo Nguyen Giap, and learned that the US had failed to understand their adversary: ‘We saw Vietnam as an element of the Cold War, not what they saw it as, a civil war.’ McNamara admitted that this failure ‘reflected our profound ignorance of the history, culture and politics of the people in the area and the personalities and habits of their leaders’. When he died in 2009, the Economist observed: ‘He was haunted by the thought that amid all the objective-setting and evaluating, the careful counting and the cost-benefit analysis, stood ordinary human beings. They behaved unpredictably.’
McNamara’s career is also a great example of the cult of managerialism which came to dominate, in the second half of the twentieth century, not just business, but also many other spheres of human activity, including health care, education and government. McNamara was a great exemplar of the new manager: ‘A trained specialist in the science of business management who is also a generalist moving easily from one technical area to another.’ Writing in the Harvard Business Review in 1980, Robert H. Hayes and William J. Abernethy blamed managerialism (at least in part) for America’s economic decline:
What has developed, in the business community as in academia, is a preoccupation with a false and shallow concept of the professional manager, a ‘pseudoprofessional’ really – an individual having no special expertise in any particular industry or technology who nevertheless can step into an unfamiliar company and run it successfully through strict application of financial controls, portfolio concepts, and a market-driven strategy.
The obsession with metrics in medicine is partly due to managerialism. The delusion that generic business methods can be easily applied to the complexities of health care has been perpetuated by famous managerialists such as Sir Gerry Robinson. Less than two weeks after the publication of the second Francis report, he wrote an opinion piece for the Daily Telegraph entitled: ‘Yes, we can fix the NHS’:
Imagine a McDonald’s in Leicester, say, where things are going wrong. Perhaps the wrong number of chicken nuggets are being handed out, or the washrooms aren’t supplied with soap. These problems would show up immediately via a weekly reporting system which compared its performance against every other McDonald’s in the country, and you’d have a senior manager down in days to sort out the problems.
To the metrics-driven managerialist, running the NHS is essentially no different from ensuring a uniformity of customer experience at the many McDonald’s outlets.
The target culture was partly to blame for Stafford. To meet financial targets to become a foundation trust, the hospital sacked 150 staff
and closed 100 beds (18 per cent of the total). The hospital had one of the lowest numbers of ward nurses in the country, with a deficit of 120 nursing posts. The Healthcare Commission Inquiry found that junior doctors were routinely diverted from duties on the general wards and redeployed to the emergency department, so the hospital would not be in breach of the four-hour target (the target for patients attending emergency departments to be seen, treated, and either admitted or discharged). This left the general wards unstaffed and dangerous. The politicians who expressed their shock and outrage over Stafford were often the very same ministers who had imposed these targets. Although John Major’s Conservative government started the process in the early 1990s with the Patient’s Charter, it was the Blair administration which really embraced targets. These targets initially focused on such matters as waiting times, cleanliness and average length of hospital stay. Ian Blunt, health services analyst at the Nuffield Trust, wrote: ‘One of the fundamental challenges to targets is that they measure what can be counted rather than what matters. This is particularly true when a target (one tiny slice of activity) is used to infer quality (which is the result of a complex array of care processes and interactions).’ Blunt noted that, when first introduced, NHS targets were generally achieved because of increased funding and central support. Targets, however, came with an inevitable and predictable downside: ‘Methods such as increasing the risk of managers being sacked and public “naming and shaming” led to dysfunctional behaviour such as “gaming” data, short-termism, bullying and obsessive checking and assurance activities.’ Targets can work if used sparingly in a few well-chosen areas, and backed up with additional resources. If more targets are added willy-nilly, they create a climate of confusion in hospitals about their priorities. Targets, which were intended to guide and promote good care, have become an end in themselves, often leading to a grotesque inversion of their original purpose.
Can Medicine Be Cured Page 17