Also Human

by Caroline Elton

  *

  My experience of sitting on an interview panel at a London medical school was not encouraging. I was shocked when another panel member marked an applicant down because she had only decided to apply to medical school in the last nine months. Implicitly this panel member was equating duration of the desire to apply for medicine with the robustness of that decision. This assumption runs counter to the results of a study that asked over two thousand eleven-year-olds about their career plans and mapped their responses to standardised tests of ability. The study found that 10% of the sample of eleven-year-olds expressed an interest in medicine, but this interest was not associated with high educational attainment or cognitive ability.30 Children in this sample seemed not to be making very robust career decisions (and why should they? Eleven years old is definitely too young to be making such a choice). But my colleague wanted to mark down an applicant because she had made her career choice as a seventeen-year-old, rather than as a younger child.

  Then there’s the whole issue of unconscious bias. There’s been little research that has looked specifically at the impact of unconscious bias in medical school admissions, but as it’s known to be a significant factor in interviews for other professions, there’s no reason why doctors would be exempt.31 Some medical schools such as the Icahn School of Medicine in New York are doing unconscious bias training for all interview panel members – and there is good evidence that this can make a difference.32 For example, in a recent study in a state medical school in the Midwest all 140 members of the admissions committee were taught about unconscious bias and took the implicit association test – an assessment of unconscious racial bias. All members were found to have significant levels of unconscious preferences for white people.33 Following this training nearly half of the committee members reported that they were conscious of their biases during the next admissions cycle and the class admitted that year was the most racially diverse in the history of the medical school.

  I also found from my experience as a panel member that traditional medical school interviews don’t explore a candidate’s motivation in any real depth – despite the fact that this is the personal quality that tops the list of the Medical Schools Council. This was definitely what had happened with Kevin, the doctor whose sister died of leukaemia.

  Medicine wasn’t actually Kevin’s first choice of career; he started off training to be an engineer and only considered switching to medicine after his sister became ill. Visiting her in hospital during her treatment, the idea of training as a doctor took hold. Sadly the treatment didn’t work for Kevin’s sister and a couple of years after diagnosis, she died. Immediately after her death, Kevin applied to medical school, even though the doctors treating his sister strongly advised him to wait a couple of years.

  So for Kevin, like some other doctors we’ve met in earlier chapters, the initial motivation to study medicine was intimately linked to illness within the family. But the interview panel didn’t dwell on this link, which is in stark contrast to what happens when you apply to train as a psychologist or psychotherapist. In interviews for those professions, one’s own motivation for choosing that particular career will be put under the microscope. Personal or family experience of mental illness won’t rule one out of the profession but there is an explicit expectation that an applicant should understand the links between things that have happened to them in the past, and their desire to help people experiencing mental health problems.

  In medical school interviews it’s not considered vital that an applicant has fully explored the way in which personal and family experience of illness might be related to their decision to train as a doctor. But such an understanding is essential because when people are faced with a tragedy, the desire to make sense of the experience, or to extract something constructive out of it, can be overwhelming. Somebody you love dies of cancer – you run a marathon to raise money for cancer research, or for the hospice that cared for your loved one at the end of their life. A cyclist gets crushed by a lorry – you campaign for separate cycling lanes, or heftier penalties for careless lorry drivers. You’ve just graduated with a good degree in engineering and your sister dies from leukaemia so you decide to capitalise on your scientific potential and train as a doctor.

  ‘By choosing medicine, something good could come out of my sister’s death’ was how Kevin explained it to me.

  Making a career choice like this isn’t necessarily problematic. However, it can be, as it was with Kevin. Because in Kevin’s case the human desire for something constructive to emerge from a personal tragedy overshadowed an appreciation of the ways in which he might struggle with medical work. Undoubtedly he had the academic potential – he had top A-level grades and a first-class degree in engineering. But Kevin told me that he found it extremely upsetting to witness human suffering and he struggled with the responsibility inherent in medical work. Even before Kevin started medical school he had doubts about whether he was psychologically suited to the profession; doubts which were not fully explored at the interview.

  *

  Panel interviews for medical school places are on the way out, anyway. In their place, applicants are assessed by ‘multiple sample’ methods that have been used for many years to assess the clinical competence of medical students later on in their training. So for example when a student’s ability to communicate effectively with a patient is assessed in medical school, they will be observed rotating through a large number of ‘stations’, each one consisting of a different communicative task. In one they might have to break bad news to an actor-patient whilst in another, they might have to placate an actor role-playing an angry relative.

  The reason that students are observed carrying out multiple tasks is that an individual’s performance is influenced far more by the specifics of the task than one intuitively expects; a candidate can do brilliantly in one station but far less well in another, even though one might have predicted that both tasks draw on a similar underlying skill. But it was not until 2004 that Professor Kevin Eva and his colleagues at McMaster University in Canada adapted these methods, which have become a ubiquitous feature of medical school education, for use in the initial selection of medical students.34

  What Eva ended up with was ten different stations, assessing different skills – the so-called multiple mini interview (MMI) format. Each station lasted eight minutes, followed by a two-minute interval during which interviewers completed standardised evaluation forms, and candidates read the details of the next station. Assessors stayed put, while candidates rotated between the stations. With this MMI format candidates are therefore observed for eighty minutes (ten stations each of eight minutes). As a comparison, with traditional panel interviews, each candidate was typically given no more than twenty minutes – so the MMI format quadruples the time spent with each candidate.

  In two of the ten stations, candidates were asked typical interview questions: ‘Why do you want to be a physician?’ and ‘What experiences have you had (and what insights have you gained from these experiences) that led you to believe you would be a good physician?’ But the other stations were very different. A candidate’s capacity for ethical decision making was assessed by giving them the following scenario:

  Dr Cheung recommends homeopathic medicines to his patients. There is no scientific evidence or widely accepted theory to suggest that homeopathic medicines work and the doctor doesn’t believe them to. He recommends homeopathic medicine to people with mild and non-specific symptoms such as fatigue, headaches and muscle aches, because he believes it will do no harm, but will give them reassurance.

  The candidate is instructed to consider the ethical problems that Dr Cheung’s behaviour might pose and discuss these issues with the interviewer. (I find it surprising that an obviously Chinese name was chosen. But perhaps this was done deliberately in order to see if candidates trotted out any racist stereotypes. Or, alternatively, to see whether an outstanding candidate was able to make subtle observations about the expec
tations that patients from different cultural backgrounds might bring to the clinical encounter.) The second station on ethical decision making asked the candidate to consider whether physicians should provide circumcisions for religious as opposed to medical reasons.

  In the two communication skills stations, actors were used. In one, the candidate was told:

  Your company needs both you and a co-worker (Sara, a colleague from another branch of the company) to attend a critical business meeting in San Diego. You have just arrived to drive Sara to the airport. Sara is in the room.

  What the candidate wasn’t told is that Sara (played by an actor) is somebody who has developed a fear of flying, following the September 11 attacks. In this particular station, the interviewer doesn’t ask any questions, but instead observes how empathic the candidate is, and how well they manage to communicate with their fictional ‘colleague’. The other communication skills station involved the candidate explaining to Tim (also played by an actor) that they had crashed into his BMW in an underground car park.

  A candidate’s critical thinking was assessed in one station by providing them with a short piece of information about artificial sweeteners, taken from the Internet, which they then had to critique. Another station assessed knowledge of the healthcare system; so, for example, they were asked their views on whether patients should be charged a small fee each time they went to visit the doctor, to discourage unnecessary visits.

  The day-to-day stuff of medical work certainly requires doctors to have a firm grounding in medical science but they also need to communicate well with their patients, and with the patients’ relatives, who can be distraught, or furious, or in despair. They need to work well with their colleagues who may be stressed and burnt-out, and make tough choices between competing priorities. They need to understand the implications and the inevitable constraints of the particular healthcare system in which they practise. And the MMIs try to assess whether the applicant might be able to carry out these sorts of task in future.

  *

  Thirteen years after they were introduced in Canada, MMIs have become standard practice throughout the world. In the UK, they were piloted by Dundee Medical School in 2008, and formally adopted in 2009. Ready-made MMIs don’t exist. Each time they have been incorporated into the admissions process the content of each station has to be worked out, matched to the precise priorities of each medical school. There’s often political work that needs to be done to bring the sometimes crusty members of the medical school’s admissions committee on board. Assessors and any actors need to be recruited and trained. And space needs to be found.35

  As an assessor on an MMI at a London medical school I have seen how the whole enterprise has to be coordinated like a military manoeuvre; it’s a huge amount of work. The questions have to change each year, so that the scenarios don’t leak out and coaching companies don’t get rich training candidates in model answers. But having sat on old-style interview panels and also MMIs, it’s clear to me that the latter assess a broader range of skills, and also allow each applicant to be observed for a longer period of time.

  Thirteen years after they were first introduced, sufficient evidence has accrued to demonstrate that MMIs improve the selection process. As an example (and there are many others from around the world), when Kevin Eva followed up the original cohort, the MMI scores were found to be the best predictor of scores on clinical performance assessments, clerkship ratings by their supervisors, and clinical aspects of the Canadian Medical Licensing exam.36

  In 2011, the great and the good in the world of medical education got together and produced a consensus statement and recommendations on medical student selection. Globally they concluded that: ‘There is evidence of the predictive validity of the multiple mini-interview … Furthermore there is evidence on this issue from outside of North America. There is not much evidence of the credibility of interviews, personal statements and letters of reference.’37

  Dundee was the first medical school in the UK to get the message, and thirty-one others have now followed. Only six medical schools in the UK are still using panel interviews for the 2017 recruitment round. Four of these institutions (Oxford, Cambridge, Imperial and University College London) also use the BMAT to assess ‘aptitude’. Except, as we now know, it assesses scientific knowledge – a metric that is inevitably influenced (like panel interviews) by the sort of school that the applicant had the luck, or misfortune, to attend.

  ‘Around the world, medical schools’ desire to apply the similarly rigorous standards of evidence to their admissions processes that they have typically applied to their clinical practice, have resulted in the typical interview being replaced by MMI,’ wrote an optimistic proponent of MMIs. Not everywhere, it would seem. Not only are there a small number of medical schools in the UK (and elsewhere) that are ignoring the compelling evidence that old-style methods don’t work. Some of these institutions are the very same ones that are carrying out research showing the flaws in the old methods – yet the medical school admissions panels don’t seem to take any notice of their colleagues’ published findings. As an example, Chris McManus at UCL has contributed to the global consensus statement on best practice in selection, but his medical school still uses panel interviews. He has highlighted the limitations of BMAT, but that’s what the admissions committee at UCL medical school has chosen to use.38

  In a similar vein, Donald Barr, a paediatrician writing in The Lancet described how research from the psychology department at University of California, San Francisco, showed that MCAT and undergraduate science scores failed to predict later clinical performance at medical school. But he sat on the admissions committee at UCSF Medical School without ever being informed of this research.39

  *

  A colleague once told me about a medical student who had been granted a place, despite failing to get the required grades, because his mother had unexpectedly died during the period when he was sitting his A-level exams. Fair enough, you might think. Except that it later transpired that his mother hadn’t died; the story was a lie. Yet he was still allowed to continue at medical school. A couple of years into the course when the student was failing badly he was summoned to a meeting with the Head of Year. On the day of the meeting he told his Head of Year that he couldn’t come because he had an urgent appointment with another faculty member. It later transpired that this was also a lie. But still he didn’t lose his place.40

  This was a number of years ago and my sense is that nowadays medical schools are more stringent about managing students who are dishonest. Unprofessional behaviour (such as lying) as a student has been shown to increase the risk of subsequent disciplinary action by the medical regulator once the student has qualified. Thus overlooking this sort of dishonesty is short-sighted in the extreme – particularly because you can’t get much of a handle on this personal quality by direct questioning at an interview. What is a dishonest person likely to do, if questioned about their honesty? Lie. So simply asking won’t work. And in an MMI format, the stakes are so high that, even if an applicant often resorts to lying in real life, they will probably tell the truth in that context. Instead, seemingly minor examples of dishonesty throughout the course need to be treated extremely seriously.

  But from my perspective, the Achilles heel of medical school selection is its inability to assess emotional resilience. Of the many struggling doctors I have encountered over the last decade, a small proportion probably didn’t have the cognitive ability to succeed in medicine, and a slightly larger proportion came to the conclusion that they weren’t sufficiently interested in medical science to continue in the profession. The remainder found medicine too emotionally demanding. Often (as we have seen in earlier chapters) their difficulties were due to temporary factors: personal illness, family bereavement, relationship breakdown, being placed in a particularly hostile work environment, lack of support from colleagues or being asked to carry out tasks for which they were inadequately trained. Sometimes the dist
ress was due to them being in the wrong branch of medicine. But for at least some of these doctors medicine, in any shape or form, was too much for them to manage.41

  Should the medical school selection process have weeded them out?

  All applicants to medical school have an occupational health assessment. In most cases health conditions will not bar somebody from becoming a doctor, because ‘reasonable adjustments’ to their training can be made. Health matters are also considered separately from the selection process. According to the guidance published by the GMC and the Medical Schools Council, a history of serious health issues, including mental health conditions, will not jeopardise a career in medicine unless the condition impinges on professional fitness to practise:

  Medical schools should explain that mental health conditions are common in medical students and that support is available. In almost every case, a mental health condition does not prevent a student from completing his or her course and continuing a career in medicine.42

  I’ve encountered many doctors who had already experienced episodes of mental illness prior to entering medical school, and occupational health allowed them to pursue medical training. Often this has worked well, but sometimes it hasn’t. And occupational health can’t be expected to screen out all those students who won’t cope with medicine before they start their course; some doctors only develop a mental illness a couple of years into medical training, so there would not have been any prior illness to declare when they first applied to medical school.

  What about the interview process? Is that a way of identifying those applicants who might not cope with medical training?

  There are exceptional cases where the answer is ‘yes’. For example, as part of an MMI I once assessed an applicant who shook so much with terror that she couldn’t utter a word. I don’t think I’m intimidating, and I didn’t have a similar effect on any other applicant; I took great pains to make all applicants feel as comfortable as possible. When it became apparent that she was unable to speak I gently told her to take her time, and that I would wait for an answer until she was ready. But she never got to the point where she was able to answer the question.